Computer system
Claim Score by NHIP
Abstract
The present invention provides techniques, including a method and system, for relocating data between storage systems. In one embodiment of the present invention a host collects usage information from a plurality of storage systems, and determines the relocation destination LU for data stored in the LU to be relocated. The host alters an LU logical position name table that determines matching between the logical position names of data and LUs. It also carries out data relocation between storage subsystems by shifting data stored in an origin LU to be relocated to a destination LU. In another embodiment relocation of files is provided.

Term
Term ended
Projected expiry passed 8 July 2022, 4.2 years ago.
- Priority
- Filed
- Published
- Projected expiry
- Today
43 claims: 14 independent, 29 dependent
- 1A system comprising a computer and a plurality of storage systems, said plurality of storage systems for providing a group of logical storage areas to the computer, said system comprising:the plurality of storage systems coupled together via a network;a storage system of said plurality of storage systems, comprising: a logical storage area of said group of logical storage areas;an acquisition module for acquiring usage of at least one storage system resource for the logical storage area;and the computer coupled to said plurality of storage systems, comprising: a gathering module for obtaining usage information on the usage of the storage system resource by said logical storage area.
- 7Broadest claimClaim Score 71, broad(NHIP)A system, comprising a computer and a plurality of distributed storage systems coupled together by a network, said system comprising:an acquisition means for acquiring, for storage areas of at least two storage systems of the plurality of distributed storage systems, usage information of storage system resources used for the storage areas, and a notifying means for notifying the computer, responsive to a request from the computer, of the usage information;and the computer comprising: a requesting means for requesting the storage systems to notify the computer of the usage information.
- 8A system comprising a computer and at least one storage system, comprising:a control means for controlling physical relocation of data among storage areas of the at least one storage system;a matching table for defining matching between a logical position indicating the logical position of data perceived by an application operating on the computer and a storage area of a storage system of the at least one storage system storing the data;and an updating means for updating the matching table so that the storage area of the storage system, which is the relocation destination of data relocated by the control means, matches the logical position of the data.
- 9A method for managing an information item used by an application program and located in a plurality of storage systems, said plurality of storage systems coupled together via a network, said method comprising:storing said information item in a first storage system of said plurality of storage systems;accessing said information item by said application program using a logical position name;and relocating said information item from said first storage system to a second storage system of said plurality of storage systems, wherein said application program accesses said information item in said second storage system by using said logical position name;
- 16A method for managing storage of a first storage system of a plurality of storage systems and a second storage system of said plurality of storage systems, wherein said plurality of storage systems are coupled together by a network, said method comprising:a manager module receiving a first logical volume usage for a first logical volume of said first storage system;said manager module receiving a second logical volume usage for a second logical volume of said second storage system;and displaying to a user storage usage information, including said first logical volume usage and said second logical volume usage.
- 20A method for relocating information from an origin logical unit (LU) of a plurality of logical units on a origin storage system to a destination LU of said plurality of logical units on a destination storage system, said origin storage system coupled to said destination storage system by a Fibre channel network, said method comprising:receiving said relocation instruction by said destination storage system;preparing a copy area management table comprising one or more origin LU No. of said origin LU, one or more destination LU No. of said destination LU, and a bit map of blocks of information to be relocated;and relocating of each block of information from said origin LU to said destination LU using said copy area management table.
- 23A relocation system for relocating information from an origin logical unit to a destination logical unit while said destination logical unit is being read to or written from by a host system, said relocation system comprising:a first disk array of a plurality of disk arrays, comprising said origin logical unit for storing an information item;a second disk array of a plurality of disk arrays, comprising said destination logical unit, said second disk array couple with said first disk array via a Fibre channel switch;and the host system for controlling relocating said information item from said origin logical unit to said destination logical unit, while concurrently reading to or writing from said destination logical unit.
- 24A method for relocating a file from a first storage system of a plurality of distributed storage systems to a second storage system of said plurality of distributed storage systems, wherein said plurality of distributed storage systems are part of a Storage Area Network (SAN), said method comprising:storing said file in an origin area on said first storage system, wherein a file position of said file in a metadata table refers to said origin area;reserving in said metadata table a destination area in said second storage system;copying said file from said origin area to said destination area;and modifying said metadata table such that said file position is modified to refer to said destination area.
- 28A system for a client relocating a file from a Fibre channel storage area network comprising a plurality of storage systems coupled with a host system to said client, said system, comprising:said client for requesting from a file system of said host system, access to said file;and said file system for determining a file location of said file in a storage system of said plurality of storage systems and returning said file location to said client;and wherein said client after receiving said file location, relocates said file directly from said storage system using said file location.
- 33A network storage system for providing a plurality of distributed storage systems viewed by a client as one virtual storage system, said network storage system comprising:said client for sending a request to a server system for a file;said server system, responsive to said request, and coupled to said plurality of distributed storage systems, for retrieving said file from said plurality of distributed storage systems, and sending said file to said client;and said plurality of distributed storage systems coupled together via Fibre channels and organized into an integrated area for access by said server system.
- 35A system for managing relocation of data between a plurality of logical units, comprising:a LU management table, comprising a unique LU No. for each logical unit (LU) of said plurality of logical units and LU information associated with each unique LU No.;a manager for receiving from a user a relocation origin LU No. and user requirements for a relocation destination LU;and a LU pool manager for determining a relocation destination LU using said LU management table and said user requirements.
- 38A method for sharing files between a client, a host, and a plurality of disk arrays, all coupled together by both a standard network and Fibre channels, said method comprising:receiving a request by said host from said client for access to a shared file, wherein said request includes a type of use and an intra-area address;determining by said host of a location of said shared file, said location including a disk array of said plurality of disk arrays;and said host allowing said client access to said shared file as determined by said type of use.
- 42A computer program product stored on a computer readable medium for managing an information item used by an application program and located in a plurality of storage systems, said plurality of storage systems coupled together via a network, said computer program product comprising:code for storing said information item in a first storage system of said plurality of storage systems;code for accessing said information item by said application program using a logical position name;and code for relocating said information item from said first storage system to a second storage system of said plurality of storage systems, wherein said application program accesses said information item in said second storage system by using said logical position name.
- 43A computer program product stored on a computer readable medium for managing storage of a first storage system of a plurality of storage systems and a second storage system of said plurality of storage systems, wherein said plurality of storage systems are coupled together by a network, said computer program product comprising:code for a manager module receiving a first logical volume usage for a first logical volume of said first storage system;code for said manager module receiving a second logical volume usage for a second logical volume of said second storage system;and code for displaying to a user storage usage information, including said first logical volume usage and said second logical volume usage.
Independent claims14
284 paragraphs in 5 sections, as filed
CROSS-REFERENCES TO RELATED APPLICATIONS
P-0001[0001] This application is related to and claims priority from Japanese Patent Application No. 2000-205510, filed on Jul. 6, 2000.
BACKGROUND OF THE INVENTION
P-0002[0002] The present invention relates generally to managing data stored in a storage system, and more particularly to relocating data in a computer system having a plurality of storage systems.
P-0003[0003] One conventional technique for relocation of data stored in a storage system such as a disk array system is described in JP-A-No. Hei 9-274544. The disk array system in this application refers to a system, in which a plurality of disk units are arranged in arrays, for accomplishing the reading/writing of data. The data is divided and stored among the individual disk units and read/written at high speed by operating the disk units in parallel. As described in D.A. Patterson, G. Gibson and R. H. Kats, “A Case for Redundant Arrays of Inexpensive Disks (RAID),” Proc. ACM SIGMOD, pp. 109-116, June 1988, different levels consisting of level <b>1</b> through level <b>5</b> are given, depending on the redundancy configuration of the disk array systems. In addition to these redundancy levels, a disk array system without redundancy may be referred to as level <b>0</b>.
P-0004[0004] Since disk array systems of different levels differ in cost, performance and characteristics, arrays (each being a group of disk units) of a plurality of levels are often mixed in architecting a disk array system. An array of each level in such a case is referred to as a parity group. As the disk units also differ in cost with performance, capacity and other factors, a plurality of types of disk units, differing in performance and capacity, may also be used in architecting a disk array system with a view to achieving the optimal cost performance.
P-0005[0005] Furthermore, in such a disk array system, since data may be distributed among a plurality of disk units, each logical storage area to be accessed by a host computer connected to the disk array system is matched with a physical storage area of a disk unit (address conversion). In such a disk array system, according to JP-A-No. Hei 9-274544, for a logical storage area, data may be relocated from one physical storage area to another physical storage area without changing the logical storage area address. Further, the load state due to accessing by the host computer to the different logical storage areas are managed, and the particulars of a relocation are determined so that the data be appropriately arranged after the relocation.
P-0006[0006] There are also techniques for transferring data between a host computer and storage systems such as that disclosed in M.T. O'Keefe, “Shared File Systems and Fibre Channel,” Proc. Sixth Goddard Conference on Mass Storage Systems and Technologies, pp. 1-16, March 1998. According to this technique, there is disclosed a SAN (Storage Area Network), i.e. a storage environment in which a plurality of host computers and a plurality of storage systems are connected by Fibre channels (FCs), in which high-speed interfaces, realize data sharing via the FCs. By transferring data via FCs in this way, loads on the host computers and the network can be reduced compared with usual transfers via a network.
P-0007[0007] A conventional technique for enabling a plurality of computers to share data in files held by storage systems connected to typical networks not using high-speed FCs, is an NFS (Network File System). When data is shared using an NFS, loads on the computer sharing data or on the network connecting the computers and storage systems are greater than in the aforementioned case of using FCs. However, since existing networks can be used, it has its own advantages in that the cost of new equipment can be smaller than where a FC network is to be newly laid and the management of file sharing and other factors can be easier.
P-0008[0008] As stated above, the technique described in JP-A-No. Hei 9-274544 makes possible relocation of data within a disk array system. However, it does not cover relocation of data between storage systems in computer system having a plurality of storage systems. In addition, since a disk array system is incapable of file recognition, the technique does not allow relocation of files.
P-0009[0009] On the other hand, a SAN (Storage Area Network) makes possible high-speed data transfers between storage systems using a FC switch. However, relocation of data by transferring data between storage systems by using a SAN entails the following problems.
P-0010[0010] In the prior art for a SAN, no consideration is given to the acquisition by the host computer, which is to control relocation of data, of necessary information for determining appropriate arrangement of data, such as the state of load on each storage area in each storage system due to accessing by the host computer. As a result neither the host computer nor its user can judge how data should be relocated to realize efficient arrangement of the data.
P-0011[0011] Furthermore, even if the user tried to relocate data in each storage system himself/herself, the burden on the user would be great because the user would have to check in detail and take charge of everything including the management of unused areas in the destination of data relocation.
P-0012[0012] Moreover, if data is transferred between storage systems, the data seen by an application, i.e., the destination address to be designated by the application to access the same data, will differ between before and after the relocation.
P-0013[0013] Also, data sharing on a typical network using an NFS involves the following problems.
P-0014[0014] In the prior art, when a host computer used for data sharing in an NFS network (hereinafter to be referred to as an “NFS server”), manages a plurality of storage systems, the NFS server itself cannot physically relocate data between the plurality of storage systems. As a consequence, it is very difficult to accomplish, by using an NFS server, fine differentiation and management of the storage areas in the storage systems, such as altering the physical positions of shared data for each computer.
P-0015[0015] Thus there is a need for a host computer, including an NFS server, to acquire from the storage systems, for example disk arrays, necessary information for appropriate arrangement of data and thereby alleviate the burden on the user of managing the data on the storage systems. There is also a need for relocation of data between different storage systems to be transparent to an application, i.e., the data location seen by an application is the same before and after the relocation. Lastly, there is a need for the relocation of data as files.
SUMMARY OF THE INVENTION
P-0016[0016] The present invention provides techniques, including a method and system, for relocating data between storage systems. Examples of storage systems include a client's PC hard disk, a server's hard disks or databases, or a disk array. In one embodiment of the present invention a disk array acquires usage of a disk unit in response to read/write from a host. The host collects usage from a plurality of disk arrays, and determines the relocation destination LU for data stored in the LU to be relocated, and alters an LU logical position name table that determines matching between the logical position names of data, which are the data positions for an application, and LUs. It also carries out data relocation between different disk arrays by shifting data stored in the LU to be relocated to the relocation destination LU.
P-0017[0017] In a first embodiment of the present invention, a computer is provided with a requesting means for requesting at least one storage system connected to the computer to notify the usage of the physical storage system resources of each logical storage area of the storage system. Further, the storage system is provided with an acquisition means for acquiring usage of physical storage system resources of each storage area of the storage system, and a notifying unit for notifying the computer, at a request from the computer, the usage of the physical storage system resources of each logical storage area of the storage system acquired by the acquisition means.
P-0018[0018] The usage of the physical storage system resources of each storage area includes, for instance, the usage of the physical storage space of the storage area and the usage of the processing time of the storage system spent in the processing of access to the storage space.
P-0019[0019] In this embodiment, the computer can use the information on usage of the physical storage system resources of each logical storage area of the storage system acquired from the storage system for, e.g., determining the particulars of appropriate arrangement of data from the viewpoint of load diffusion of storage system resources. Therefore, by using this information, the data can be appropriately arranged by, for instance, relocating the data among different storage systems.
P-0020[0020] In a second embodiment of the invention, a computer is provided with a control means for controlling physical relocation of data among the logical storage areas of the at least one storage system connected to the computer; a matching table for defining matching between a logical position indicating the logical position of data perceived by an application operating on the computer and a logical storage area of the storage system storing the data; and an updating unit for updating the matching table so that the logical storage area of the storage system, which is the relocation destination of data relocated by the control means, match the logical position of the data.
P-0021[0021] In this embodiment, even if, between before and after the relocation of data, the storage system or storage area storing the data varies, the logical position of the data does not vary. Thus, data can be relocated without allowing the logical position to vary between before and after the relocation of the data for an application accessing data according to its logical position.
P-0022[0022] In a third embodiment of the invention, a computer is provided with a managing means for managing a matching between a file and a logical storage area of the storage system connected to the computer in which the file is stored; and an extracting unit for extracting the usage of the physical storage system resources of the logical storage area in which each file is stored.
P-0023[0023] In this third embodiment, it is possible to obtain on the computer side the usage of the physical storage system resources of the storage area, in which a file is stored, and to determine the particulars of the relocation of the file. This enables files to be efficiently arranged.
P-0024[0024] These and other embodiments of the present invention are described in more detail in conjunction with the text below and attached figures.
BRIEF DESCRIPTION OF THE DRAWINGS
P-0025[0025]FIG. 1 is a diagram illustrating the configuration of a computer system of a first embodiment of the present invention.
P-0026[0026]FIG. 2 is a flow chart showing the procedure of read/write processing and usage acquisition processing in the first embodiment of the invention.
P-0027[0027]FIG. 3 is a table of items of logical/physical matching information used in the first embodiment of the invention.
P-0028[0028]FIG. 4 is a table of items of logical volume usage in the first embodiment of the invention.
P-0029[0029]FIG. 5 is a flow chart showing the procedure of usage collection processing in the first embodiment of the invention.
P-0030[0030]FIG. 6 is a list of parameters of logical volume information for use in the first embodiment of the invention.
P-0031[0031]FIG. 7 is a further list of parameters of logical volume information for use in the first embodiment of the invention.
P-0032[0032]FIG. 8 is a list of parameters of parity group information for use in the first embodiment of the invention.
P-0033[0033]FIG. 9 is a further list of parameters of parity group information for use in the first embodiment of the invention.
P-0034[0034]FIG. 10 is a list of parameters of usage information for use in the first embodiment of the invention.
P-0035[0035]FIG. 11 is a further list of parameters of usage information for use in the first embodiment of the invention.
P-0036[0036]FIG. 12 is a flow chart showing the procedure of relocation object determination processing in the first embodiment of the invention.
P-0037[0037]FIG. 13 is a flow chart showing the procedure of relocation processing in the first embodiment of the invention.
P-0038[0038]FIG. 14 shows an LU logical position name table for use in the first embodiment of the invention.
P-0039[0039]FIG. 15 shows another LU logical position name table for use in the first embodiment of the invention.
P-0040[0040]FIG. 16 is a flow chart showing the procedure of copy processing in the first embodiment of the invention.
P-0041[0041]FIG. 17 shows a copy area management table for use in the first embodiment of the invention.
P-0042[0042]FIG. 18 is a flow chart showing the procedure of processing of a command to read/write from or into a relocation destination LU in the course of copying shown in FIG. 16 in the first embodiment of the invention.
P-0043[0043]FIG. 19 is a diagram illustrating the configuration of a computer system of a second embodiment of the present invention.
P-0044[0044]FIG. 20 shows an LU area range table for use in the second embodiment of the invention.
P-0045[0045]FIG. 21 is a flow chart showing the procedure of read/write processing in the second embodiment of the invention.
P-0046[0046]FIG. 22 shows metadata for use in the second embodiment of the invention.
P-0047[0047]FIG. 23 is a flow chart showing the procedure of relocation object determination processing in the second embodiment of the invention.
P-0048[0048]FIG. 24 is a flow chart showing the procedure of relocation processing in the second embodiment of the invention.
P-0049[0049]FIG. 25 is a diagram illustrating the configuration of a computer system of a third embodiment of the invention.
P-0050[0050]FIG. 26 is a flow chart showing the procedure of processing by the application of a client to read from a file in the third embodiment of the invention.
P-0051[0051]FIG. 27 is a flow chart showing the procedure of processing by the application of a client to write into a file in the third embodiment of the invention.
P-0052[0052]FIG. 28 is a diagram illustrating the configuration of a computer system of a fourth embodiment of the invention.
P-0053[0053]FIG. 29 shows an LU management table in the fourth embodiment of the invention.
P-0054[0054]FIG. 30 is a flow chart showing the procedure of relocation object determination processing in the fourth embodiment of the invention.
P-0055[0055]FIG. 31 is a flow chart showing the procedure of build-up processing in the fourth embodiment of the invention.
P-0056[0056]FIG. 32 is a flow chart showing the procedure of processing to the relocation origin LU off line in the fourth embodiment of the invention.
P-0057[0057]FIG. 33 is a diagram illustrating the configuration of a computer system of a fifth embodiment of the invention.
P-0058[0058]FIG. 34 shows an LU area range table in the fifth embodiment of the invention.
P-0059[0059]FIG. 35 is a flow chart showing the procedure of processing by the application of a client to read from a file in the fifth embodiment of the invention.
P-0060[0060]FIG. 36 is a flowchart showing the procedure of processing by the application of a client to write into a file in the fifth embodiment of the invention.
P-0061[0061]FIG. 37 is a flowchart showing the procedure of processing to reply on accessible files in the fifth embodiment of the invention.
DESCRIPTION OF THE SPECIFIC EMBODIMENTS
P-0062[0062] First will be described a first embodiment of the invention.
P-0063[0063]FIG. 1 illustrates the configuration of a computer system for a first embodiment of the present invention.
P-0064[0064] As illustrated, a computer system in this embodiment comprises a host <b>100</b>, disk arrays <b>200</b>-<b>1</b> and <b>200</b>-<b>2</b>, a switch <b>500</b>, clients <b>800</b>-<b>1</b> and <b>800</b>-<b>2</b> and a local disk <b>190</b>.
P-0065[0065] The host <b>100</b> has a network interface <b>170</b> and a FC (Fibre Channel) interface <b>160</b>. The network interface <b>170</b> is connected via a network <b>700</b> to the clients <b>800</b>-<b>1</b> and <b>800</b>-<b>2</b> and the disk arrays <b>200</b>-<b>1</b> and <b>200</b>-<b>2</b>. The FC interface <b>160</b> is connected to the switch <b>500</b>, the disk arrays <b>200</b>-<b>1</b>/<b>2</b> and the local disk <b>190</b> via FCs <b>600</b>-<b>1</b> to <b>600</b>-<b>4</b>. Also, the host <b>100</b> has a file system <b>110</b>, an OS <b>120</b>, a manager <b>130</b> and an application <b>140</b>. The application <b>140</b> reads from and writes into the disk arrays <b>200</b>-<b>1</b>/<b>2</b> via the OS <b>120</b> and the file system <b>110</b>.
P-0066[0066] The host <b>100</b> and the clients <b>800</b>-<b>1</b>/<b>2</b> can be implemented on an electronic computer with a typical configuration, provided with, for instance, a CPU, a main storage, an external storage system(s), an input device, a display device and so forth. In this case, each section shown in FIG. 1 is realized as a process embodied on the electronic computer as the CPU executes a program loaded on the main storage. The program for implementing these sections on the electronic computer illustrated in FIG. 1 is stored in advance in an external storage system and loaded onto the main storage as required for execution by the CPU. Alternatively, the program may be stored on a portable storage medium, for instance a CD-ROM or a DVD-ROM, and after being transferred from the portable storage medium into the external storage system via a reading device, loaded onto the main storage as required for execution by the CPU. Or else, it may be received as appropriate via a communication medium, such as the network <b>700</b> and, after being stored into the external storage system, loaded onto the main storage as required for execution by the CPU.
P-0067[0067] In the local disk <b>190</b> is stored various items of management information including an LU (logical unit) logical position name table <b>191</b> and an intra-LU address logical position name table <b>195</b> to be used by the OS <b>120</b> and the file system <b>110</b>. The LU logical position name table <b>191</b> is a table that shows the matching or mapping between a logical position name the application <b>140</b> designates when accessing data in a disk array system <b>200</b>-<b>1</b> or <b>200</b>-<b>2</b> and a LU (which will be elaborated upon later) for storing the set of data specified by the logical position name.
P-0068[0068] The intra-LU address logical position name table <b>195</b> is a table that shows the matching or mapping between a logical position name the application <b>140</b> designates when accessing data in a disk array system and the intra-LU address of the set of data specified by the logical position name.
P-0069[0069] For example the disk array <b>200</b>-<b>1</b> has a control section <b>300</b>-<b>1</b>, a plurality of disk units <b>210</b>-<b>1</b> to <b>210</b>-<b>8</b>, a network interface <b>270</b>-<b>1</b> and an FC interface <b>260</b>-<b>1</b>. The network interface <b>270</b>-<b>1</b> is connected via the network <b>700</b> to the clients <b>800</b>-<b>1</b> and <b>800</b>-<b>2</b>, and the host <b>100</b>. The FC interface <b>260</b>-<b>1</b> is connected via the switch <b>500</b> and FCs <b>600</b>-<b>1</b>, <b>600</b>-<b>3</b> and <b>600</b>-<b>4</b> to the host <b>100</b> and the local disk <b>190</b>.
P-0070[0070] The control section <b>300</b>-<b>1</b> has a CPU <b>310</b>-<b>1</b> for the execution of processing, a memory <b>320</b>-<b>1</b> and a cache <b>330</b>-<b>1</b>. Into the memory <b>320</b>-<b>1</b> are stored logical/physical matching information <b>321</b>-<b>1</b>, a logical volume usage <b>322</b>-<b>1</b> and a copy area management table <b>323</b>-<b>1</b>. Details of these information items will be described later.
P-0071[0071] There are n (n is 2 or a larger integer) disk units <b>210</b>, e.g., <b>210</b>-<b>1</b>, <b>210</b>-<b>2</b>, <b>210</b>-<b>3</b>, and <b>210</b>-<b>4</b>, to constitute a RAID (disk array), and this set of n disk units, generically referred to as <b>210</b>, is referred to as a parity group, generically referred to as <b>220</b>, (for example disk units <b>210</b>-<b>1</b>, <b>210</b>-<b>2</b>, <b>210</b>-<b>3</b>, and <b>210</b>-<b>4</b> constitute parity group <b>220</b>-<b>1</b>). Note, in this description, for ease of documentation, a label with a dashed extension number refers to a specific item in the figures (e.g., <b>210</b>-<b>1</b> or <b>210</b>-<b>2</b>), while a number without a dashed extension (e.g., <b>210</b>) refers generically to the item and includes one or more of the dashed extensions (e.g. the label “disk units <b>210</b>” includes <b>210</b>-<b>1</b> and/or <b>210</b>-<b>2</b>). Possible configurations of a RAID include redundancy-involving ones such as a configuration in which redundant data (parity) generated from the contents stored in n-1 disk units, out of the disk units <b>210</b> contained in one parity group <b>220</b>, are stored into the remaining one disk unit, or a mirror disk (RAID 1) configuration in which contents stored in n/2 disk units are kept in a copied form by the remaining n/2 disk units. Each parity group <b>220</b> can be deemed to be one operational unit.
P-0072[0072] Incidentally, because the cost, performance, characteristics and other factors differ with the level of redundancy, the magnitude of n (the number of disk units) and the like, arrays (parity groups <b>220</b>) differing in level and n may be mixed in configuring disk arrays, for example, <b>200</b>-<b>1</b> and <b>200</b>-<b>2</b>. Regarding disk units <b>210</b> constituting the parity groups <b>220</b> as well, because of the difference in cost with performance, capacity and other factors, a plurality of types of disk units <b>210</b> differing in performance and capacity may be used to realize the optimal cost performance in configuring disk arrays <b>200</b>. In this embodiment, too, the attributes of different parity groups <b>220</b> constituting disk arrays <b>200</b> including performance, reliability and characteristics may be either the same or different.
P-0073[0073] Since the disk arrays <b>200</b>store data in the disk units <b>210</b> in a distributive way, a logical volume to be read from or written to by the host <b>100</b> is matched with physical addresses representing storage areas in the disk units <b>210</b> (address conversion), and the host <b>100</b> is thereby provided with the logical volume. Further, the disk arrays <b>200</b>, in converting addresses, can as well combine a plurality of logical volumes and provide them to the host <b>100</b> as a single logical unit (LU). Thus the disk arrays <b>200</b> provide to the host <b>100</b> an LU consisting of at least one logical volume. Then the host <b>100</b> reads from and writes into the LU.
P-0074[0074] In the above-described configuration in this embodiment, physical relocation of data, taking into account the usage of the disk arrays <b>200</b>, is made possible between the plurality of disk arrays <b>200</b>, for example, physical relocation of data between disk array <b>200</b>-<b>1</b> and disk array <b>200</b>-<b>2</b>. More specifically, the disk arrays <b>200</b> acquire the usage of the disk units <b>210</b> in terms of reading/writing by the host <b>100</b>. The host <b>100</b> collects the usage acquired by each of the plurality of disk arrays <b>200</b> as described above, and displays it to the user. Further, the host <b>100</b>, in response to an instruction from the user to whom the usage of the plurality of disk arrays <b>200</b> has been presented in the above-described manner, alters the LU logical position name table <b>191</b> in the local disk <b>190</b>, and copies data stored by the disk arrays <b>200</b> in the LU, for example from disk array <b>200</b>-<b>1</b> to <b>200</b>-<b>2</b>. LU relocation between the plurality of disk arrays <b>200</b> is accomplished in this manner. By making possible such data relocation taking account of the usage of the disk arrays <b>200</b>, appropriate arrangement of data is also made possible.
P-0075[0075] The actions of the computer system in this first embodiment will be described in detail below.
P-0076[0076] First will be described the read/write processing by the disk arrays <b>200</b> in accordance with a read/write request from the host <b>100</b> and usage acquisition processing by which the disk arrays <b>200</b> acquire the usage of the disk units <b>210</b>.
P-0077[0077]FIG. 2 illustrates the sequence of this processing of an embodiment of the present invention.
P-0078[0078] First, in the host <b>100</b>, as the application <b>140</b> designates a file by a specific file logical position name and requests the OS <b>120</b> to read from or write into the file, the OS <b>120</b> requests the file system <b>110</b> to read from or write into the file. In response, the file system <b>110</b> accesses the local disk <b>190</b> via the FC interface <b>160</b> to obtain from the LU logical position name table <b>191</b> the LU number in which the designated file is stored, and obtains from the intra-LU address logical position name table <b>195</b>, among other items, the intra-LU address at which the designated file is stored. Then, it issues a read command or a write command of the SCSI (Small Computer System Interface) standard, accompanied with the LU number and the intra-LU address to the disk arrays <b>200</b> providing the LU of the LU number so obtained via the FC interface <b>160</b> (step <b>1000</b>).
P-0079[0079] Here in this system wherein the application <b>140</b> designates a file by a description of a path to the logical position of the file in terms of a logical drive name, a directory name and a file name, the description of the path such a logical position (in terms of the logical drive, the directory and the file) serves as the logical position name of the file. Normally, the logical position name is an item of information on a logical position used by the application for designating the object of access.
P-0080[0080] The file system <b>110</b>, in order to manage such logical positions, not only manages hierarchical logic structures between logical positions such as a directory structure, but also describes matching or mapping between the logical position name and LU number of each logical position name in the LU logical position name table <b>191</b> and manages it. It further describes matching between the logical position name and the intra-LU address of each logical position in the logical position name table <b>195</b> and manages it. In this embodiment one disk array may have one or more LU's. FIG. 3 shows the logical/physical matching information in a disk array, for example, disk array <b>200</b>-<b>1</b>. Disk array <b>220</b>-<b>2</b> will have its own logical/physical matching table. The LU No. 2 in row <b>5030</b> of FIG. 3 has a special logical volume 4, i.e., COMMAND VOLUME, that is different from the logical volumes in LU. No. 0 or 1. In an alternate embodiment, an LU number also denotes the disk array <b>200</b> providing the LU of that LU number.
P-0081[0081] Next, as an disk array, for example, <b>200</b>-<b>1</b> receives a read/write command from the host <b>100</b> via the FC interface <b>260</b>-<b>1</b>, the control section <b>300</b>-<b>1</b>, using the logical/physical matching information <b>321</b>-<b>1</b> in the memory <b>320</b>-<b>1</b>, specifies a logical volume number associated with the LU number and a logical volume address associated with the intra-LU address, and carries out address conversion into the physical address by determining an area in the disk unit <b>210</b>-<b>1</b> matching the logical volume and logical volume address (step <b>1010</b>). Then, the control section <b>300</b>, where read is requested, reads out data from the physical address obtained by the address conversion of the disk unit <b>210</b>-<b>1</b> and transfers it to the host <b>100</b> or, where write is requested, stores the data transferred from the host <b>100</b> and a parity generated in that connection at the physical address obtained by the address conversion of the disk unit <b>210</b>-<b>1</b> (step <b>1020</b>).
P-0082[0082] The logical/physical matching information <b>321</b>-<b>1</b> here used for the address conversion at step <b>1010</b> has the contents listed in FIG. 3 for example.
P-0083[0083] In FIG. 3, the LU number <b>5001</b> and the intra-LU address <b>5002</b> respectively indicate an LU number and an intra-LU address that the file system <b>110</b> of the host <b>100</b> designates by read/write processing command. The logical volume number <b>5003</b> is a logical volume number matching the LU specified by the LU number <b>5001</b>, and the logical volume address <b>5004</b> is an address in the logical volume matching the intra-LU address <b>5002</b>.
P-0084[0084] Further, the physical address <b>5020</b> is an address indicating an area on a disk unit in which data <b>5022</b> and parities <b>5024</b> are to be stored, and has the parity group number <b>5005</b>, the disk unit numbers <b>5006</b> and the intra-disk unit addresses <b>5007</b>, one each for the data <b>5022</b> and the parities <b>5024</b>. The parity group number <b>5005</b> indicates an individual parity group <b>220</b>. The disk unit number <b>5006</b> indicates an individual disk unit <b>210</b>, and the intra-disk unit address <b>5007</b> is an address indicating an area in a disk unit.
P-0085[0085] Referring back to FIG. 2, the description will be continued for the example of disk array <b>200</b>-<b>1</b>.
P-0086[0086] The control section <b>300</b>-<b>1</b>, upon completion of the read/write processing described above, executes usage acquisition processing, in which read or write and sequential or random access in the read/write processing are distinguished, and updates the logical volume usage <b>322</b>-<b>1</b> in the memory <b>320</b>-<b>1</b> which has been the object of read/write (step <b>1030</b>).
P-0087[0087] Here the logical volume usage <b>322</b>-<b>1</b> has the contents listed in FIG. 4 for instance.
P-0088[0088] As shown, a logical volume number <b>5101</b>, and a disk use duration (in microseconds) <b>5102</b> for read or write and sequential or random access are described for each logical volume. Therefore, at step <b>1030</b>, the length of time taken to read or write is added to the disk use duration <b>5102</b> of the determined access type matching the logical volume number <b>5101</b> of the logical volume which is the object of reading or writing.
P-0089[0089] Next will be described usage collection processing in which the host <b>100</b> collects the usage of disk units <b>210</b> from each disk array <b>200</b>.
P-0090[0090]FIG. 5 shows the sequence of this processing of an embodiment of the present invention.
P-0091[0091] First, a disk array <b>200</b> provides the host <b>100</b> with an LU (command volume) for information transfer. This LU (command volume) is a logical volume having no matching disk unit <b>210</b> as shown, for example, by row <b>5030</b> (LU No. 2, Logical Volume No. 4 with the words “Command Volume” as the parity group <b>5005</b>) in the logical/physical matching table of FIG. 3.
P-0092[0092] In the host <b>100</b>, the manager <b>130</b> issues a write command of the SCSI standard to the LU (command volume), for example, logical volume 4 of row <b>5003</b>, of a disk array, for example <b>200</b>-<b>1</b>, via the FC interface <b>160</b>, and writes parameters for information collection as data (step <b>1100</b>).
P-0093[0093] When the disk array <b>200</b>-<b>2</b> receives the write command from the host <b>100</b> via the FC interface <b>260</b>-<b>2</b>, the control section <b>300</b>-<b>2</b> perceives that the write command concerns the LU (command volume), and checks an operation code contained in the parameters for information collection transferred from the host <b>100</b> to distinguish the requested information. It then readies the requested information on the memory <b>320</b>-<b>2</b> (step <b>1110</b>). After that, the control section <b>300</b>-<b>2</b> reports the completion of write to the host <b>100</b> via the FC interface <b>260</b>-<b>2</b> (step <b>1120</b>).
P-0094[0094] Next in the host <b>100</b>, the manager <b>130</b> issues a read command of the SCSI standard to the LU (command volume) of the disk array <b>200</b>-<b>2</b> via the FC interface <b>160</b> (step <b>1130</b>).
P-0095[0095] When the disk array <b>200</b>-<b>2</b> receives the read command from the host <b>100</b> via the FC interface <b>260</b>-<b>2</b>, the control section <b>300</b>-<b>2</b> perceives that the read command concerns the LU (command volume), and transfers the information readied on the memory <b>320</b>-<b>2</b> to the host <b>100</b> via the FC interface <b>260</b>-<b>2</b> (step <b>1140</b>). After that, the control section <b>300</b>-<b>2</b> reports the completion of read to the host <b>100</b> via the FC interface <b>260</b>-<b>2</b> (step <b>1150</b>).
P-0096[0096] Here, the parameters for collecting the information to be written at step <b>1100</b> and the information to be readied at step <b>1110</b> in this connection include, a table of three kinds of information including logical volume information, parity group information and usage information.
P-0097[0097] For example, where the parameters for collecting the information to be written at step <b>1100</b> are parameters for logical volume information as shown in FIG. 6, for a logical volume identified by the logical volume number <b>5201</b> designated by the 0th through 1 st bytes, logical volume information (information indicating the configuration of that logical volume in the disk array <b>200</b>) shown in FIG. 7 is readied. Incidentally, in the logical volume information shown in FIG. 7, various items of information <b>5202</b> of the logical volume identified by the logical volume number <b>5201</b> designated by the 0th through 1 st bytes are described in the 8th through 47th bytes, and items of information <b>5203</b> of logical volumes constituting the LU to which this logical volume belongs are described in the 49th through 121st bytes.
P-0098[0098] In another example, where the parameters for collecting the information to be written at step <b>1100</b> are parameters for parity group information as shown in FIG. 8, for a parity group <b>220</b> identified by the parity group number <b>5204</b> designated by the 2nd through 3rd bytes, to which the logical volume identified by the logical volume number <b>5201</b> designated by the 0th through 1st bytes belongs, parity group information (items of information indicating the configuration of the parity group <b>220</b> in the disk array <b>200</b> including the RAID configuration and the type denomination of the disk unit <b>210</b>) shown in FIG. 9 is readied. Incidentally, in the parity group information shown in FIG. 9, various items of information <b>5205</b> of a parity group identified by the parity group number <b>5204</b> described in the 2nd through 3rd bytes are described in the 8th through 29th bytes, and items of information <b>5206</b> of logical volumes allocated to that parity group are described in the 30th through 287th bytes.
P-0099[0099] For the formation of the logical volume information and parity group information mentioned above in the control section <b>300</b> here, part or the whole of the logical/physical matching information <b>321</b> is used. Incidentally, the manager <b>130</b> has information on the performance of disk units <b>210</b> of each type, and accordingly the performance features of disk units <b>210</b> constituting a parity group <b>220</b> can be known according to the model denomination of the disk units <b>210</b>.
P-0100[0100] In yet another example, where the parameters for collecting the information to be written at step <b>1100</b> are usage parameters as shown in FIG. 10, for a logical volume identified by the logical volume number <b>5201</b> designated by the 0th through 1st bytes, usage information (resource usage in the disk array <b>200</b>) shown in FIG. 11 is readied. Incidentally, in the usage information shown in FIG. 11, occupation duration's <b>5207</b> of the logical volume usage <b>322</b>, whose example is shown in FIG. 4, of the logical volume identified by the logical volume number <b>5201</b> designated by the 0th through 1st bytes are described in the 136th through 159th bytes. In the 48th through 79th and the 100th through 115th bytes, information items <b>5208</b> and <b>5209</b> on the number of receipts of various commands by that logical volume, the number of hits on the cache <b>330</b> and the like are described, and in the 160th through 991st bytes, information item <b>5210</b> on the occupation of the processor <b>310</b>, that of the internal bus and so forth are described.
P-0101[0101] Here, the control section <b>300</b> has acquired with respect to each logical volume the number of receipts of various commands by that logical volume, the number of hits on the cache <b>330</b>, the occupation of the processor <b>310</b>, that of the internal bus and so forth, and these information items, as shown in FIG. 11, are reflected in the usage information. In addition, the manager <b>130</b> can determine the rate of occupation duration per unit length of time by, for instance, dividing the average duration of a plurality of acquisitions by the interval of acquisition.
P-0102[0102] Incidentally, the manager <b>130</b> of the host <b>100</b>, by issuing an INQUIRY command of the SCSI standard to an LU and obtaining response data separately from the above-described processing charted in FIG. 5, can obtain from the response data the logical volume number to which the LU belongs.
P-0103[0103] Next will be described relocation object determination processing by which the host <b>100</b> determines the data to be relocated in the first embodiment.
P-0104[0104]FIG. 12 shows the sequence of this processing of an embodiment of the present invention.
P-0105[0105] In the host <b>100</b>, the manager <b>130</b> distinguishes LUs used by the OS <b>120</b> from unused LUs (unoccupied LUs) by referencing, for instance, the LU logical position name table <b>191</b> stored in the local disk <b>190</b>. With respect to each of the LUs being used by the OS <b>120</b>, the manager <b>130</b> computes the usage of each logical volume in each disk array <b>200</b>, for example disk arrays <b>200</b>-<b>1</b> and <b>200</b>-<b>2</b>, the usage of the logical volume matched by each LU, and so forth from the logical volume number belonging to that LU obtained by issuing the INQUIRY command, the logical volume information in each disk array <b>200</b> obtained by the aforementioned usage collection processing, the usage of parity group information and logical volume, and the like (step <b>1200</b>).
P-0106[0106] The results of these calculations, together with the attributes of the parity group <b>220</b> to which the logical volumes belong (RAID configuration, model denominations of disk units <b>210</b>, performance features of the disk units <b>210</b> and so forth) are presented to the user (step <b>1210</b>).
P-0107[0107] The manager <b>130</b> also presents unoccupied LUs to the user. For each LU, it calculates the usage of the logical volume matched by each logical volume from the logical volume number belonging to that LU obtained by issuing the INQUIRY command, the logical volume information in each disk array <b>200</b> obtained by the aforementioned usage collection processing, the usage of parity group information and logical volume, and the like (step <b>1220</b>), and presents to the user the results of these calculations in the parity group <b>220</b> and the like related to each unoccupied LU (step <b>1230</b>).
P-0108[0108] Here, the aforementioned items of information including usage can be displayed by the host <b>100</b> or another computer connected via a network to the host <b>100</b>.
P-0109[0109] The user references the above-stated items of information with respect to each LU in each disk array <b>200</b> to determine the LU whose data are to be relocated (relocation origin LU) and the relocation destination LU for the data, though the manager <b>130</b> may as well determine the data relocation origin and the relocation destination from the above-stated items of information instead of having the user do it (step <b>1240</b>). The determination of these particulars of relocation is so accomplished that, for instance, load distribution among the disk arrays <b>200</b>, load distribution among the parity groups <b>220</b>, and allocation of LUs in which files requiring high performance high performance parity groups <b>220</b> can be realized after the relocation. The size of the relocation destination LU here should not be smaller than that of the relocation origin LU. Incidentally, the size of each LU can be acquired with a READ CAPACITY command of the SCSI standard.
P-0110[0110] Next will be described data relocation processing which the host <b>100</b> performs following the relocation object determination processing described above. For purposes of illustration only, let disk array <b>200</b>-<b>1</b> be the relocation origin LU and disk array <b>200</b>-<b>2</b> be the relocation destination LU.
P-0111[0111]FIG. 13 shows the sequence of this processing of an embodiment of the present invention.
P-0112[0112] In the host <b>100</b>, the manager <b>130</b> first instructs the file system <b>110</b> to lock the relocation origin LU (step <b>1300</b>). In response, the file system <b>110</b> suspends acceptance of any request to read/write from or into the relocation origin LU (step <b>1310</b>). Then the manager <b>130</b> instructs the file system <b>110</b> to flush the cache with respect to the relocation origin LU (step <b>1320</b>).
P-0113[0113] Following this, the file system <b>110</b> writes into the relocation origin LU of the disk array <b>200</b>-<b>1</b> any data to be stored in the relocation origin LU but cached in the memory on the host <b>100</b> but not written into the disk array <b>200</b>-<b>1</b> (step <b>1330</b>).
P-0114[0114] Then the manager <b>130</b> instructs the file system <b>110</b> to invalidate the cache with respect to the relocation origin LU (step <b>1340</b>). Following this, the file system <b>110</b> invalidates any data to be stored in the relocation origin LU but cache in the memory on the host <b>100</b> (step <b>1350</b>).
P-0115[0115] The above-described LU locking and flushing and invalidation of the cache may regarded as equivalent to so-called unmounting of the LU.
P-0116[0116] Next the manager <b>130</b> instructs the disk array <b>200</b>-<b>2</b> in which the relocation destination LU is present to copy data from the relocation origin LU, disk array <b>200</b>-<b>1</b>, to the relocation destination LU, disk array <b>200</b>-<b>2</b> (step <b>1360</b>). This instruction, as in the usage collection processing described above, is accomplished by writing into the command volume of the aforementioned disk array <b>200</b>-<b>2</b> copy instructing parameters including a copy instructing operation code, the relocation origin LU and the relocation destination LU. Having received this instruction, the disk array <b>200</b>-<b>2</b> starts copy processing to be described later, and notifies the manager <b>130</b> of the receipt of the copy instruction (step <b>1370</b>).
P-0117[0117] Now, notified of the receipt of the copy instruction,the manager <b>130</b> rewrites the LU logical position name table <b>191</b> stored in the local disk <b>190</b>, to be used by the file system <b>1</b><b>10</b>, and replaces the logical position names (mapping) of the relocation origin LU and of the relocation destination LU with each other (step <b>1380</b>). Further, the manager <b>130</b> instructs the file system <b>110</b> to update the LU logical position name table <b>191</b> (re-reading) and to release the lock which was instructed at step <b>1300</b> (step <b>1390</b>).
P-0118[0118] In response, the file system <b>110</b> re-reads the LU logical position name table <b>191</b> to update information (step <b>1400</b>), and release the aforementioned lock to accept a read/write request step <b>1410</b>).
P-0119[0119] The above-stated updating and lock release may be regarded as so-called mounting of the LU.
P-0120[0120] As a result, when the file system <b>110</b> reads from or writes into an LU, including cases in which the application <b>140</b> reads from or writes into an LU via the OS <b>120</b> and the file system, if the object LU of read/write is an LU to relocated, read/write by the file system <b>110</b> will be from or into the relocation destination LU.
P-0121[0121] Hereupon, examples of the LU logical position name table <b>191</b> rewritten at step <b>1380</b> described above are shown in FIG. 14 and FIG. 15.
P-0122[0122] The tables of FIGS. 14 and 15 show another embodiment of the LU NO. of the present invention. Both tables show, the disk array number, ID and LUN that indicate an LU number <b>6001</b>. FIG. 14 shows a logical position name <b>6002</b> in a directory form, while FIG. 15 shows a logical position name <b>6002</b> in a drive form. Both show the logical position of the LU as a storage area to be used by the application <b>140</b>.
P-0123[0123] Next will be described copy processing that takes place when, in the above-described relocation processing, a disk array, for example, <b>200</b>-<b>2</b> receives a copying instruction from the host <b>100</b>.
P-0124[0124]FIG. 16 shows the sequence of this processing of an embodiment of the present invention.
P-0125[0125] In a disk array <b>200</b>-<b>2</b> in which the relocation destination LU is present, when a copying instruction is received from the host <b>100</b> via the FC interface <b>260</b>-<b>2</b>, the control section <b>300</b>-<b>2</b> readies a copy area management table <b>323</b>-<b>2</b> regarding the relocation destination LU designated by the copying instruction on the memory <b>320</b>-<b>2</b> and makes setting (step <b>1500</b>). Particulars of the copy area management table <b>323</b>-<b>2</b> are shown in FIG. 17.
P-0126[0126] In FIG. 17, the copying destination LU number <b>6101</b> and the copying origin LU number <b>6102</b> are numbers indicating the relocation destination LU, e.g. disk array <b>200</b>-<b>2</b>, and the relocation origin LU, disk array <b>200</b>-<b>1</b>, respectively, on the FC <b>600</b>. More specifically, in this embodiment, the LU No. are represented uniquely by either eight-byte numbers (WORLD WIDE NAME) or three-byte numbers (N_PORT ID) designated by the host <b>100</b>. In another embodiment the LU No. can be represented by disk array No., ID and LUN like in FIG. 14 and <b>15</b>. The number of blocks to be copied <b>6103</b>, i.e. the number of blocks (smallest read/write units) in the area to be copied, represents the magnitude of the area to be copied, and in the bit map <b>6104</b>, a bit is allocated to each block in the area to be copied in the LU, wherein “1” denotes uncopied and “0”, copied. At the time of initialization, every bit in the area to be copied is set to “1”.
P-0127[0127] Now referring back to FIG. 16, the control section <b>300</b>-<b>2</b> notifies the host <b>100</b> of the receipt of the copying instruction (step <b>1510</b>). This notification is given at the time of actual copying after the setting of the aforementioned information following the actual receipt of the copying instruction. Therefore the lapse of time from the receipt of the copying instruction until its receipt is short.
P-0128[0128] Next, the control section <b>300</b>-<b>2</b> performs copying to read the contents stored in the relocation origin LU, disk array <b>200</b>-<b>1</b>, via the FC interface <b>260</b>-<b>2</b> and to stored them into the relocation destination LU, disk array <b>200</b>-<b>2</b> (step <b>1520</b>).
P-0129[0129] Then, with respect to the area to be copied in the LU, the control section <b>300</b>-<b>2</b> alters the bits of the blocks corresponding to the copied areas shown in FIG. 17 successively to “0” (step <b>1530</b>) and, upon completion of the blocks <b>6103</b> to be copied, the copy processing is ended (step <b>1540</b>).
P-0130[0130] In addition, if the disk array <b>200</b>-<b>1</b> in which the relocation origin LU is present and the disk array <b>200</b>-<b>2</b> in which the relocation destination LU is present, are the same, LU copying may be accomplished within the disk array <b>200</b>-<b>1</b> (or <b>200</b>-<b>2</b>).
P-0131[0131] Incidentally, any read/write action by the host <b>100</b> from or into the LU to be relocated is accomplished, even during a copying process, upon the relocation destination LU, i.e. the disk array <b>200</b>-<b>2</b> in which the relocation destination LU is present.
P-0132[0132] Next will be described the processing in the case wherein a disk array <b>200</b>-<b>2</b> has received a read/write command regarding the LU to be relocated during the aforementioned copy processing.
P-0133[0133]FIG. 18 shows the sequence of this processing of an embodiment of the present invention.
P-0134[0134] In the disk array <b>200</b>-<b>2</b>, when the control section <b>300</b>-<b>2</b> receives a read command via the FC interface <b>260</b>-<b>2</b>, it compares the range to be read and the bit map <b>6104</b> shown in FIG. 17 (step <b>1610</b>); if any uncopied part remains in the area to be read (step <b>1620</b>, Y), the control section <b>300</b>-<b>2</b> reads out and copies data in the area to be read with priority (step <b>1630</b>), updates bits in the area to be read in the bit map <b>6104</b> to “copied” (step <b>1640</b>), and transfers the copied data in its own disk array <b>200</b>-<b>2</b> to the host <b>100</b> (step <b>1650</b>). On the other hand, if the area to be read has been wholly copied (step <b>1620</b>, N), the control section <b>300</b>-<b>2</b> immediately transfers the copied data in its own disk array <b>200</b>-<b>2</b> to the host <b>100</b> (step <b>1650</b>).
P-0135[0135] Or the control section <b>300</b>-<b>2</b>, when it receives a write command via the FC interface <b>260</b>-<b>2</b> (step <b>1600</b>, N), writes the data transferred from the host <b>100</b> into the area to be written (step <b>1670</b>), updates bits in the area to be written in the bit map <b>6104</b> shown in FIG. 17 to “copied” (step <b>1680</b>), and continues copying from the remaining uncopied parts (step <b>1660</b>).
P-0136[0136] The processing so far described enables a disk array in which any relocation destination LU is present to process a read/write command from the host <b>100</b> even during a copying process.
P-0137[0137] Also, in this read/write processing, the control section <b>300</b> performs the earlier described usage acquisition processing at the same time.
P-0138[0138] Incidentally, the manager <b>130</b> of the host <b>100</b>, while copying during the aforementioned copying process, can inquire of the disk array <b>200</b>-<b>2</b> about copying progress information by writing into the command volume of the disk array <b>200</b>-<b>2</b> parameters for acquiring the state of copying progress and reading the data.
P-0139[0139] In this case, the control section <b>300</b>-<b>2</b> of the disk array <b>200</b>-<b>2</b> having accepted a write command regarding a command volume checks the parameters written into the command volume in accordance with the command, references the copy area management table <b>323</b>-<b>2</b> to make ready on the memory <b>320</b>-<b>2</b> items of information including the rate of copying progress, and notifies the host <b>100</b> of the completion of writing. In response, the manager <b>130</b> of the host <b>100</b> reads out of the aforementioned command volume, and the control section <b>300</b>-<b>2</b> answers the inquiry about copying progress and other factors by transferring data readied on the memory <b>320</b>-<b>2</b> in response to the reading.
P-0140[0140] In this first embodiment, appropriate arrangement of data among a plurality of disk arrays <b>200</b> by relocation of LUs can be realized so that logical equivalence can be ensured for the application <b>140</b> between before and after the relocation, i.e. the logical position name to be used by the application for accessing the object of access remains the same.
P-0141[0141] Although, relocation of data among a plurality of disk arrays <b>200</b> was described with respect to this embodiment, this description is not intended to limit the scope of the invention. Storage subsystems for data to be relocated need not be disk array subsystems. They may be some other kind of storage subsystems using magnetic disk units, floppy disks, zip drives, jazz drives, photomagnetic disk units, magnetic tape devices, semiconductor disk units or the like.
P-0142[0142] Further, in this embodiment, the manager <b>130</b> of the host <b>100</b> is supposed to collect information or give instructions using a command of the SCSI standard via the FC <b>600</b>. However, some other kind of command may be used as well. Also, the manager <b>130</b> may collect information or give instructions using a protocol prescribed under the SNMP (Simple Network Management Protocol), for instance, via the network <b>700</b>, instead of the FC <b>600</b>.
P-0143[0143] Further, with respect to this embodiment, the logical volume usage <b>322</b>-<b>2</b> acquired by the control section <b>300</b>-<b>2</b> of a disk array <b>200</b>-<b>2</b> is supposed to the cumulative total of the durations of use. However, the control section <b>300</b>-<b>2</b> may as well accumulate in the memory <b>320</b>-<b>2</b> durations of use in unit lengths of time lapse in the form of the rate of use, this may be collected by the manager <b>130</b> of the host <b>100</b> as the logical volume usage <b>322</b>-<b>2</b>.
P-0144[0144] Next will be described a second embodiment of the invention.
P-0145[0145]FIG. 19 illustrates the configuration of a computer system of a second embodiment of the invention.
P-0146[0146] As illustrated, the computer system in this embodiment has a similar configuration to the computer system in the first embodiment illustrated in FIG. 1. However, this embodiment has in local disk <b>190</b> an LU area range table <b>192</b> and in its switch <b>500</b> a copy control section <b>510</b>.
P-0147[0147] In such a configuration in this embodiment, the disk arrays <b>200</b>-<b>1</b>/<b>2</b> acquire the usage of disk units <b>210</b>, and the host <b>100</b> collects usage from the plurality of disk arrays <b>200</b>, and presents it to the user including analyses based on files of this computer system. Further the host <b>100</b> alters data for file management (metadata). The switch <b>500</b>, as instructed by the host <b>100</b>, copies data stored in the disk arrays <b>200</b>. This enables files to be relocated among the plurality of disk arrays <b>200</b> to efficiently arrange the data.
P-0148[0148] Now, in the above-described first embodiment, the file system <b>110</b> of the host <b>100</b> manages LUs as differentiated between those in use and those not in use. By contrast in this embodiment, the file system <b>110</b> uses all the LUs, and manages the set of the areas of all the LUs as a single area (hereinafter to be referred to as an “integrated area” ). Files in the integrated area are managed with metadata. The metadata are stored in a predetermined position in the integrated area.
P-0149[0149] The computer system in this second embodiment will be described in detail below.
P-0150[0150] First, one example of LU area range table <b>192</b> used by the file system <b>110</b> to manage the integrated area in this embodiment is illustrated in FIG. 20.
P-0151[0151] In the figure, an intra-area address <b>6301</b> is an address in the integrated area. A disk array number, an ID and a LUN constitute an LU number <b>6302</b> indicating an LU to be stored in the matching intra-area address <b>6301</b>. The ID and LUN have a format given in the SCSI standard. An intra-LU address <b>6303</b> is an address in an LU identified by the matching LU number <b>6302</b>. Thus the LU area range table <b>192</b> shows matching or mapping between the range of the integrated area <b>6301</b> with that of each LU No. <b>6302</b> and intra-LU area <b>6303</b>.
P-0152[0152] Next will be described processing by the host <b>100</b> to perform a read/write.
P-0153[0153]FIG. 21 shows the sequence of this processing for this embodiment.
P-0154[0154] It is presupposed that the application <b>140</b> of the host <b>100</b> designates the logical positions of the files managed by the file system <b>110</b>, and reads from or writes into data stored in the disk arrays <b>200</b>-<b>1</b> and <b>200</b>-<b>2</b>. The file system <b>110</b>, in order to manage data as files, also stores in the disk arrays <b>200</b> data for file management (metadata) in addition to the data to be managed as such.
P-0155[0155] An example of the metadata is illustrated in FIG. 22.
P-0156[0156] As shown, the metadata, for example, may include a date of preparation <b>6401</b>, a date of updating <b>6402</b>, a date of access <b>6403</b>, an attribute <b>6404</b>, a logical position name <b>6405</b>, security information <b>6406</b> and a file position (intra-area address) <b>6407</b> for each file. Also the range corresponding to each file in the integrated area is represented by the file position <b>6407</b>. For example, file position <b>6410</b> has intra-area address starting at 100 and ending at 150 and its range is (150-100)=50.
P-0157[0157] Now, as the application <b>140</b> in the host <b>100</b> designates a file by a specific file logical position name and requests the OS <b>120</b> to read from or write into the file (step <b>1700</b>), the OS <b>120</b> requests the file system IIO to read from or write to the file (step <b>1710</b>). In response, the file system <b>110</b> first references the metadata, for example, FIG. 22, and obtains the position of the designated file <b>6407</b> (intra-area address) and next uses the LU area range table <b>192</b>, for example FIG. 20, to get the LU No. <b>6302</b> and intra-LU address <b>6303</b> (step <b>1720</b>). If the request is to write, the file system <b>110</b> also updates the metadata (step <b>1740</b>). Then the file system <b>110</b> reads from or writes to a disk array <b>200</b> with respect to the LU and intra-LU address indicated by the intra-area address ( obtained at step <b>1720</b> (step <b>1750</b>), and finally updates the metadata (step <b>1760</b>).
P-0158[0158] In updating the metadata here at steps <b>1740</b> and <b>1760</b>, the date of preparation <b>6401</b> (FIG. 22), date of updating <b>6402</b>, date of access <b>6403</b>, attribute <b>6404</b>, logical position name <b>6405</b>, security information <b>6406</b>, file position <b>6407</b> and so forth of the accessed file are updated according to the particulars of the access. For instance, where writing results in a change in file size, the intra-area range indicated by the file position <b>6407</b> of the metadata is expanded or compressed accordingly. If a new file is to be prepared, an entry is added to the metadata, or if an existing file is deleted, the matching entry is deleted.
P-0159[0159] In addition, although the metadata are stored in the disk arrays <b>200</b> here, in an alternative embodiment, they may be cached in the memory on the host <b>100</b> under the management of the file system <b>110</b>.
P-0160[0160] Now in this embodiment, the control sections <b>300</b>, e.g., <b>300</b>-<b>1</b> and <b>300</b>-<b>2</b>, of the disk arrays <b>200</b> perform usage acquisition processing as in the above-described first embodiment. Also, the manager <b>130</b> of the host <b>100</b> performs usage collection processing as in the above-described first embodiment.
P-0161[0161] Next will be described file-by-file relocation determination processing performed by the host <b>100</b>.
P-0162[0162] The sequence of this processing of an embodiment of the present invention is shown in FIG. 23.
P-0163[0163] The manager <b>130</b> in the host <b>100</b> inquires of the file system <b>110</b> about file-LU matching with respect to each file present in the integrated area (step <b>1800</b>). In response the file system <b>110</b>, using the metadata and the LU area range table <b>192</b>, answers this inquiry (step <b>1810</b>).
P-0164[0164] Next, the manager <b>130</b> computes the usage of each logical volume in each disk array <b>200</b>, that of each logical volume in each LU, that of each logical volume in each file and so forth from the logical volume number belonging to each LU in each disk array <b>200</b> obtained by issuing an INQUIRY command (step <b>1820</b>). Then the manager <b>130</b> presents the computation results to the user together with the attribute of the parity group <b>220</b> to which each logical volume belongs (step <b>1830</b>). Thus the host <b>100</b> presents, for example, displays, to the user information concerning usage from the viewpoints of the disk arrays <b>200</b>, logical volumes, LUs, and files.
P-0165[0165] Also the manager <b>130</b> presents to the user available unoccupied areas with respect to LUs and logical volume offered by each disk array <b>200</b>. Thus the manager <b>130</b> inquires of the file system <b>110</b> about available unoccupied areas with respect to LUs and logical volumes offered by each disk array <b>200</b> (step <b>1850</b>). In response the file system <b>110</b>, referencing the metadata and the LU area range table <b>192</b>, specifies unoccupied areas where no file is present, and replies to the manager <b>130</b> (step <b>1860</b>). Also the manager <b>130</b> presents to the user in a classified form, according to various aspects of usage obtained by the usage collection processing, the usage of logical volumes in unoccupied areas together with the attributes of the logical volumes and the parity groups <b>220</b> (step <b>1870</b>).
P-0166[0166] The host <b>100</b> or another computer network-connected to the host <b>100</b> can display information on usage and unoccupied areas. The user determines, on the basis of these items of information, the file to be relocated and the unoccupied area for which the relocation is to be destined. Or the manager <b>130</b>, on the basis of these items of information, automatically determines a similar object of relocation or unoccupied area (step <b>1880</b>).
P-0167[0167] Optionally, the file system <b>110</b> of the host <b>100</b> may monitor the read/write request frequencies (access frequencies)from the OS <b>120</b> and the application <b>140</b> to each file to generate statistical information, and present it to the user together with other items of information.
P-0168[0168] This enables the user to take into account the frequency of accesses to each file in the host <b>100</b> in determining the file to be relocated.
P-0169[0169] Next will be described relocation processing performed by the host <b>100</b> in response to determination by the above-described relocation object determination processing.
P-0170[0170]FIG. 24 shows the sequence of this processing of an embodiment of the present invention.
P-0171[0171] The manager <b>130</b> in the host <b>100</b> instructs the file system <b>110</b> to lock the file to be relocated (step <b>1900</b>). In response, the file system <b>110</b> suspends acceptance of a request to read from write into the file (step <b>1910</b>). Then the manager <b>130</b> instructs the file system <b>110</b> to flush the cache with respect to the file to be relocated (step <b>1920</b>). Responding to this, the file system <b>110</b>, with respect to that file, writes into a disk array <b>200</b> data cached in the memory on the host <b>100</b> but not yet written into any disk array <b>200</b> (step <b>1930</b>).
P-0172[0172] Next, the manager <b>130</b> instructs the file system <b>110</b> to reserve an unoccupied area at the relocation destination (step <b>1940</b>). In response, the file system <b>110</b> updates the metadata to secure the area at the designated relocation destination (step <b>1950</b>). For example, in the table of FIG. 22, a date of reservation <b>6420</b>, and a file position <b>6407</b> are two possible example entrees. Further the manager <b>130</b> instructs the file system <b>110</b> to flush the cache of the metadata (step <b>1960</b>). Responding to this, the file system <b>110</b> writes into a disk array <b>200</b> the metadata cached in the memory on the host <b>100</b> (step <b>1970</b>)
P-0173[0173] Then the manager <b>130</b> locates the LU and intra-LU area (relocation origin area) and the relocation destinationLUandintr a-LU area in which the data of the designated file are currently stored, and instructs the copy control section <b>510</b> of the switch <b>500</b> to copy data from the relocation origin area to the relocation destination area (step <b>1980</b>). This instruction is given by using an EXTENDED COPY command of the SCSI standard.
P-0174[0174] Having received the copying instruction, the switch <b>500</b> reads the data in the relocation origin area from the disk array <b>200</b> in which the relocation origin area is present, and copies the designated data by writing the data into the relocation destination area of the disk array <b>200</b> in which the relocation destination area is present (step <b>1990</b>). Then, after the copying is finished, it notifies the manager <b>130</b> of the completion of copying (step <b>2000</b>).
P-0175[0175] In response the manager <b>130</b> rewrites the metadata, and alters the position of the designated file from the relocation origin area to the relocation destination area. This makes the relocation origin area an unoccupied area (step <b>2010</b>). Next the manager <b>130</b> instructs the file system <b>110</b> to invalidate the cache of the metadata (step <b>2020</b>). In response, the file system <b>110</b> invalidates the metadata cache in the memory on the host <b>100</b> (step <b>2030</b>).
P-0176[0176] Further, the manager <b>130</b> instructs the file system <b>110</b> to release the lock instructed at step <b>1900</b> (step <b>2040</b>). Responding to this, the file system <b>110</b> releases the lock, and accepts a request to read from or write into a designated file (step <b>2050</b>).
P-0177[0177] Thereafter, when the file system <b>110</b> reads from or writes to a file, including cases in which the application <b>140</b> reads from or writes into that file via the OS <b>120</b> and the file system <b>110</b>, data copied into the relocation destination area can be read from or written into in a normal way.
P-0178[0178] The second embodiment of implementing the present invention has been described so far.
P-0179[0179] In this second embodiment, appropriate arrangement of files among a plurality of disk arrays <b>200</b> can be accomplished so that logical equivalence can be ensured for the application <b>140</b> between before and after the relocation.
P-0180[0180] In this embodiment, the manager <b>130</b> of the host <b>100</b> is supposed to give a copying instruction to the copy control section <b>510</b> of the switch <b>500</b> using an EXTENDED COPY command of the SCSI standard. However, in other embodiments, some other kind of command may be used as well. Also, as shown in FIG. 19, the disk arrays <b>200</b> may have copy control sections <b>510</b>, and the manager <b>130</b> may give a copying instruction to the copy control section <b>510</b> of a disk array <b>200</b> to cause the disk array <b>200</b> to perform copy processing.
P-0181[0181] Next will be described a third embodiment of the invention.
P-0182[0182]FIG. 25 illustrates the configuration of a computer system of the third embodiment of the invention.
P-0183[0183] As illustrated, the computer system in this third embodiment has a configuration in which the computer system in the second embodiment, illustrated in FIG. 19, has in each of its clients <b>800</b>, FC interface <b>860</b> and network interface <b>870</b>; and the client <b>800</b> is connected by the FC interface <b>860</b> to the host <b>100</b>, the disk arrays <b>200</b> and the switch <b>500</b> via the FC <b>600</b> and by the network interface <b>870</b> to the host <b>100</b> and the disk arrays <b>200</b> via the network <b>700</b>. In such a configuration in this embodiment, the plurality of clients <b>800</b> and the host <b>100</b> share the files on the disk arrays <b>200</b>. In addition, on each client <b>800</b>, an OS <b>820</b> and an application <b>840</b> are present. Further, as the hardware configuration of the clients <b>800</b>, like the above-described host <b>100</b>, a usual configuration of electronic computers can be used.
P-0184[0184] Now, in this third embodiment too, as in the above-described second embodiment, the file system <b>110</b> uses all the LUs, and manages the set of the areas of all the LUs as a single integrated area. It manages the files in the integrated area with metadata described with reference to the second embodiment.
P-0185[0185] The computer system in this third embodiment will be described in detail below.
P-0186[0186] First will be described processing of access by a client <b>800</b> to a filed stored in a disk array <b>200</b>.
P-0187[0187]FIG. 26 shows the sequence of read processing of an embodiment of the present invention.
P-0188[0188] When in a client, for example, client <b>800</b>-<b>1</b>, the application <b>840</b>-<b>1</b> requests the OS <b>820</b>-<b>1</b> to read from a file (step <b>2100</b>), the OS <b>820</b>-<b>1</b> notifies, via the network interface <b>870</b>-<b>1</b> or the FC interface <b>860</b>-<b>1</b>, the file system <b>110</b> of the host <b>100</b> of reading from the file (step <b>2110</b>).
P-0189[0189] In the host <b>100</b>, the file system <b>110</b> notified of reading from the file finds the LU (for example, LU NO. <b>1</b>, on disk array <b>200</b>-<b>1</b>) and the intra-LU address at which the file is stored by referencing the metadata and the LU area range table <b>192</b> (step <b>2120</b>), and locks the intra-LU address of the LU in which the file is stored against other writing access (step <b>2130</b>). Then, after flushing the cache of the metadata on the host <b>100</b> (step <b>2140</b>), the file system <b>110</b> gives a reply to the OS <b>820</b>-<b>1</b> of the client <b>800</b>-<b>1</b> on the LU No. and the intra-LU address at which the file is stored and the LU No. and the intra-LU address at which the metadata are stored (step <b>2150</b>).
P-0190[0190] In the client <b>800</b>-<b>1</b>, the OS <b>820</b>-<b>1</b> having received the reply, with respect to the disk array <b>200</b>-<b>1</b> in which the LU the file to be read from is stored, reads via the FC interface <b>860</b>-<b>1</b> from the intra-LU address at which the file is stored, and processes the request from the application <b>840</b>-<b>1</b> (step <b>2160</b>).
P-0191[0191] Upon completion of the above-described read processing, the OS <b>820</b>-<b>1</b> updates the data of accessing the file on the metadata stored at the LU and the intra-LU address notified from the file system <b>110</b> of the host <b>100</b> (step <b>2170</b>). After that, the file system <b>110</b> is notified via the network interface <b>870</b>-<b>1</b> or the FC interface <b>860</b>-<b>1</b> of the completion of processing (step <b>2180</b>).
P-0192[0192] In the host <b>100</b>, the file system <b>110</b> notified of the completion of processing invalidates the cache of the metadata on the host <b>100</b> (step <b>2190</b>), and then releases the lock effected at step <b>2130</b> (step <b>2200</b>).
P-0193[0193]FIG. 27 shows the sequence of write processing of an embodiment of the present invention.
P-0194[0194] In a client <b>800</b>, when the application <b>840</b> requests the OS <b>820</b> to write into a file (step <b>2300</b>), the OS <b>820</b> notifies via the network interface <b>870</b> or the FC interface <b>860</b> the file system <b>110</b> of the host <b>100</b> of writing into the file (step <b>2310</b>).
P-0195[0195] In the host <b>100</b>, the file system <b>110</b> notified of writing into the file finds the LU and the intra-LU address at which the file is stored by referencing the LU area range table <b>192</b> and the metadata (step <b>2320</b>), locks the LU and the intra-LU address in which the file is stored (step <b>2330</b>), describes in the metadata a reservation for an area to be used by a file which may be expanded by writing (step <b>2340</b>), and then flushes the cache of the metadata on the host <b>100</b> (step <b>2350</b>). Next, the file system <b>110</b> gives the OS <b>820</b> of the client <b>800</b> a reply on the LU and the intra-LU address at which the pertinent file is stored (including the area reserved for use by the file which may be expanded by writing) and the LU and the intra-LU address at which the metadata is stored (step <b>2360</b>). The quantity of the expansion of the file which may be expanded by writing is supposed to be included in the notification of writing from the OS <b>820</b> of the client <b>800</b>.
P-0196[0196] In the client <b>800</b>, the OS <b>820</b> having received the reply, with respect to the disk array <b>200</b> in which the LU the file to be written into is stored, writes via the FC interface <b>860</b> into the intra-LU address at which the file is stored, and processes the request from the application <b>840</b> (step <b>2370</b>).
P-0197[0197] Upon completion of the above-described write processing, the OS <b>820</b> updates the area used by the file, the date of updating, and that of access on the metadata stated at the LU and the intra-LU address notified from the file system <b>110</b> of the host <b>100</b> (step <b>2380</b>).
P-0198[0198] Then it notifies the file system <b>110</b> of the completion of processing via the network interface <b>870</b> of the FC interface <b>860</b> (step <b>2390</b>).
P-0199[0199] In the host <b>100</b>, the file system <b>110</b> notified of the completion of processing invalidates the cache of the metadata on the host <b>100</b> (step <b>2400</b>), and then releases the lock effected at step <b>2330</b> (step <b>2410</b>).
P-0200[0200] By processing access by the clients <b>800</b> in the manner described above, the clients <b>800</b> and the host <b>100</b> can share files stored in the disk arrays <b>200</b> without conflict. To add, file access by the host <b>100</b> itself is processed in a similar way to the above-described file access by the clients <b>800</b>.
P-0201[0201] Next will be described the relocation of a file in this embodiment.
P-0202[0202] The steps of processing the relocation of a file in this embodiment (usage acquisition processing, usage collection processing, relocation object determination processing and relocation processing) are similar to those in the above-described second embodiment. However, while the file being locked during the above-described read/write processing, relocation processing is not executed. Further, the cache flushing for the file at steps <b>1920</b> and <b>1930</b> of relocation processing shown in FIG. 24 and writing back to the disk array <b>200</b> is instructed by the file system <b>110</b> to the client <b>800</b> caching the file, and the client <b>800</b> executes them.
P-0203[0203] The third mode of implementing the present invention has been described so far.
P-0204[0204] In this embodiment, in an environment wherein data stored in the disk arrays <b>200</b> are shared for use, physical relocation of files among the plurality of disk arrays <b>200</b> can be accomplished so that logical equivalence can be ensured for the applications <b>140</b> and <b>840</b> before, during, and after the relocation.
P-0205[0205] Optionally, in this embodiment as well, the file system <b>110</b> of the host <b>100</b> may monitor the read/write request frequencies from the OSs <b>120</b> and <b>820</b> and the applications <b>140</b> and <b>840</b> to each file to generate statistical information, and present it to the user in relocation object determination processing.
P-0206[0206] In another embodiment, a manager <b>130</b> may be provided on each client <b>800</b>, in order to process the collection of information on usage and to give instructions to the file system <b>10</b> of the host <b>100</b> and the disk arrays <b>200</b> by using the FC interface <b>860</b> or the network interface <b>870</b>.
P-0207[0207] The present invention is not limited to the above-described embodiments, but a number of variations are conceivable within the scope of the invention.
P-0208[0208] For example, as illustrated in FIG. 1, FIG. 19 and FIG. 25, the manager <b>130</b> can as well be disposed outside the host <b>100</b> as a program on a remote computer <b>400</b> having a network interface <b>470</b> and an FC interface <b>460</b>. Then, if the aforementioned information is collected and instructions are given via the FC <b>600</b> or the network <b>700</b> to perform similar sequences of processing as described above, appropriate arrangement of data by the relocation of LUs among a plurality of disk arrays <b>200</b> can also be accomplished equivalently for the application <b>140</b>.
P-0209[0209] In another example, the files may be shared in the earlier described first embodiment as in the third embodiment. In this case, too, as in the first embodiment, physical relocation of data among a plurality of disk arrays <b>200</b> can be accomplished so that logical equivalence can be ensured for the applications <b>140</b> and <b>840</b> before, during, and after the relocation.
P-0210[0210] Next will be described a fourth mode embodiment of the present invention.
P-0211[0211]FIG. 28 illustrates the configuration of a computer system to which the fourth embodiment of the invention is applied.
P-0212[0212] As illustrated in FIG. 28, the computer system in this embodiment has a configuration in which the computer system in the first embodiment illustrated in FIG. 1 has in its host <b>100</b> an LU pool manager <b>900</b> and an LU management table <b>910</b>.
P-0213[0213] This configuration facilitates the selection of the relocation destination for LUs. This embodiment will be described below.
P-0214[0214] An example of the LU management table <b>910</b> is illustrated in FIG. 29.
P-0215[0215] Herein, the LU number <b>3310</b> is a reference number allocated uniquely to each LU for use by the LU pool manager <b>900</b> for the management of LUs. The size <b>3320</b> represents the capacity of the pertinent LU. The configuration <b>3330</b> indicates the type of RAID configuration or, where the LU is composed of a cache <b>330</b> or an independent disk, indicates that configuration.
P-0216[0216] The state <b>3340</b> indicates the state of the LU, which can be one of the following types: “on line”, “off line”, “unmounted” and “in trouble off line”. “On line” is a state in which the LU is normal, indicating accessibility from the host <b>100</b>. “Off line” refers to an unoccupied LU, i.e. a state in which an LU is normally present but inaccessible from the host <b>100</b>. “Unmounted” means that this LU is undefined and therefore inaccessible from the host <b>100</b>. “In trouble off line” means that the LU is in trouble and inaccessible from the host <b>100</b>.
P-0217[0217] The disk array number <b>3350</b> indicates a disk array <b>200</b> in which LU(s) is/are present.
P-0218[0218] The path <b>3360</b> is a reference number indicating which of the FCs <b>600</b>, a plurality of which is connected to each disk array <b>200</b>, the pertinent LU is allocated. The ID <b>3370</b> and the LUN <b>3380</b> are reference numbers identifying an LU.
P-0219[0219] The disk performance <b>3390</b> is an indicator of the performance of the disk unit <b>210</b> in which the pertinent LU is currently arranged. In FIG. 29, it is classified into high, medium and low by the average seek time and the average rotation awaiting time of the disk unit <b>210</b> and according to the configuration, and LUs on a cache are further classified to be of ultra-high performance.
P-0220[0220] The emulation type <b>3400</b> is the form of each LU provided by a disk array <b>200</b> to the host <b>100</b> as a disk unit.
P-0221[0221] The relocatability flag <b>3410</b> is a flag with which it is possible to designate, in relocating an LU, whether the relocation destination of the LU is usable as such. The user can use this flag <b>3410</b> to distinguish LUs to be relocated from other LUs, and also can change the on/off state of this flag <b>3410</b>.
P-0222[0222] Whereas FIG. 29 includes an LU management table <b>910</b> for the disk array number 0, the manager <b>130</b> holds an LU management table <b>910</b> for every disk array <b>200</b>.
P-0223[0223] Relocation is determined in this embodiment in the following manner.
P-0224[0224] First the user designates to the manager <b>130</b> the relocation origin LU and specifies requirements of a relocation destination LU. Specific requirements include performance conditions and the level of reliability.
P-0225[0225] For instance, where the relocation origin LU is excessively bearing a load beyond its hardware capacity, if a relocation destination of a higher performance is designated, the relocation will increase the processing capacity of the pertinent LU, and the performance of the computer system can be expected to improve.
P-0226[0226] Or, where an LU storing important data is present on an independent disk or a non-redundant RAID (RAID0), if a RAID5 or a RAID1 is designated as the relocation destination, resistance to trouble provided by redundancy can be secured.
P-0227[0227] After that, the manager <b>130</b>, using information registered in the LU management table <b>910</b>, determines the relocation destination for the LU and, after notifying the user, relocates the LU.
P-0228[0228] Specific steps of relocation object determination processing in this embodiment will be described with reference to FIG. 30. First, the user designates to the manager <b>130</b> the disk array number <b>3350</b> of the relocation origin LU, path <b>3360</b>, ID <b>3370</b> and LUN <b>3380</b> (<b>2500</b>). In this case the disk array number <b>3350</b> and LU number <b>3310</b> may as well be designated in place of the bus <b>3360</b>, ID <b>3370</b> and so forth.
P-0229[0229] Next, the user designates, for example, inputting information via Graphical User Interface (GUI), to the manager <b>130</b> performance conditions and the level of reliability as requirements of the relocation destination (<b>2510</b>).
P-0230[0230] The manager <b>130</b> notifies the LU pool manager <b>900</b> of the aforementioned requirements concerning the relocation origin LU and the relocation destination (<b>2520</b>). The LU pool manager <b>900</b>, searching the LU management table <b>910</b>, checks if there is an LU satisfying the requirements (<b>2530</b>).
P-0231[0231] In this case, the conditions of search should be “the state is off line”, “the size is not smaller than the relocation origin LU”, “the emulation type is the same as the relocation origin LU”, “the relocatability flag is on (yes), i.e. relocation is possible”, “the performance conditions meet the requirements” and “the level of reliability satisfies the requirement”.
P-0232[0232] If an LU satisfying the above-stated requirements is found at step <b>2540</b>, the LU pool manager notifies the manager <b>130</b> of the LU (<b>2550</b>), and the manager <b>130</b> determines this LU to be the relocation destination LU (<b>2560</b>).
P-0233[0233] If no LU satisfying the requirements is found at step <b>2540</b>, the LU pool manager <b>900</b> searches the LU management table to find an LU number <b>3310</b> whose “state is unmounted” (<b>2570</b>).
P-0234[0234] If no unmounted LU number <b>3310</b> is found to exist, the LU pool manager <b>900</b> notifies the manager <b>130</b> of the unavailability of an LU satisfying the requirements (<b>2580</b>), the manager <b>130</b> so notified, notifies the user that there is no relocation destination LU available(<b>2590</b>).
P-0235[0235] If an unmounted LU is found at step <b>2570</b>, the LU pool manager <b>900</b> designates requirements for the unmounted LU number <b>3310</b> and the relocation destination LU, and instructs a satisfying disk array <b>200</b> to build up a relocation destination LU (<b>2600</b>).
P-0236[0236] The requirements for the relocation destination LU in this case include: “the size is not smaller than the relocation origin LU”, “the emulation type is the same as the relocation origin LU”, “the performance conditions meet the requirements” and “the level of reliability satisfies the requirement”.
P-0237[0237] The disk array <b>200</b> so instructed performs LU build-up processing (<b>2610</b>) and, if it succeeds in building up one, notifies the LU pool manager <b>900</b> of the above-cited items of information including the disk array number <b>3350</b>, bus <b>3360</b>, ID <b>3370</b> and LUN <b>3380</b> regarding the built-up LU (<b>2620</b>) or, if it fails to build up one, notifies the LU pool manager <b>900</b> of the impossibility to build up the required LU (<b>2610</b>).
P-0238[0238] The LU pool manager <b>900</b> registers into the LU management table <b>910</b> information on the LU which was notified of (<b>2630</b>), and notifies the manager <b>130</b> (<b>2550</b>). The manager <b>130</b> determines this LU to be the relocation destination LU (<b>2560</b>).
P-0239[0239] The LU pool manager <b>900</b> notified of the inability to build up the required LU notifies the manager <b>130</b> of the unavailability of an LU satisfying the requirements (<b>2580</b>), the manager <b>130</b> so notified notifies the user of the unavailability of any relocation destination LU (<b>2590</b>).
P-0240[0240] Here will be described LU build-up processing performed by a disk array <b>200</b> with reference to FIG. 31 of an embodiment of the present invention.
P-0241[0241] The disk array <b>200</b>, instructed as described above, receives requirements concerning the unmounted LU number <b>3310</b> and the relocation destination LU (<b>2700</b>).
P-0242[0242] Next, the disk array <b>200</b> judges from the state of allocation of internal resources and other factors in the disk unit <b>210</b> and the cache <b>330</b> whether or not an LU satisfying the above-stated requirements can be built up (<b>2710</b>). If one can be, the disk array <b>200</b> builds up the LU by allocating internal resources and carrying out formatting/initialization processing, and allocates a designated unmounted LU number <b>3310</b> (<b>2720</b>).
P-0243[0243] Further the disk array <b>200</b> sets the FC interface <b>260</b> to allocate the path <b>3360</b>, ID <b>3370</b> and LUN <b>3380</b> to the aforementioned LU (<b>2730</b>). Next the disk array <b>200</b> notifies the LU pool manager <b>900</b> the aforementioned items of information regarding the built-up LU including the disk array number <b>3350</b>, path <b>3360</b>, ID <b>3370</b> and LUN <b>3380</b> (<b>2740</b>).
P-0244[0244] If a build up of an LU cannot be done at step <b>2710</b>, the LU pool manager <b>900</b> is notified of the inability to build up an LU (<b>2750</b>).
P-0245[0245] The manager <b>130</b> having determined the relocation destination LU as described above, performs relocation processing with respect to the relocation origin LU and the relocation destination LU as in the first embodiment.
P-0246[0246] Next will be described processing to place the relocation origin LU off line with reference to FIG. 32 of an embodiment of the present invention.
P-0247[0247] The manager <b>130</b> acquires the state of copying progress by the method described with reference to the first embodiment and, if the copying is completed, instructs the LU pool manager <b>900</b> to place the relocation origin LU off line (<b>2800</b>).
P-0248[0248] The LU pool manager <b>900</b> so instructed instructs the disk array <b>200</b> of the relocation origin LU to place the relocation origin LU off line (<b>2810</b>). The disk array <b>200</b> so instructs sets the FC interface <b>260</b> and places the aforementioned LU off line (<b>2820</b>). The disk array <b>200</b> notifies the LU pool manager <b>900</b> of the placement of the LU off line (<b>2830</b>).
P-0249[0249] The LU pool manager so notified updates the state <b>3340</b> of the pertinent in the LU management table to an off-line state (<b>2840</b>).
P-0250[0250] Although a case in which the manager <b>130</b> acquires information on the progress of copying here, it is also possible for the disk array <b>200</b> to notify the manager <b>130</b> of the completion of copying.
P-0251[0251] Or, instead of having the manager <b>130</b> instruct the placement of the LU off line, the disk array <b>200</b> may place the relocation origin LU off line upon completion of copying, and notify the LU pool manager <b>900</b> of the placement of the LU off line.
P-0252[0252] Other embodiments accomplish instructions and notifications between the LU pool manager <b>900</b> and the disk arrays <b>200</b> described with respect to this embodiment according to the SCSI, SNMP or some other protocol or command system via the FC <b>600</b> or the network <b>700</b>.
P-0253[0253] Also, although information on the requirements for a relocation destination LU and the like are supposed to be designated by the user in this embodiment, in an alternative embodiment the manager <b>130</b> may automatically make the judgment and assign the designation accordingly.
P-0254[0254] Further, while the LU pool manager <b>900</b> and the manager <b>130</b> are supposed to be present on the host <b>100</b> in this embodiment, in another embodiment the LU pool manager may as well be present on some other computer then the manager <b>130</b>, such as a remote computer <b>400</b>.
P-0255[0255] In this case, the LU pool manager <b>900</b> and the manager <b>130</b> accomplish instructions and notifications according to the SCSI, SNMP or some other protocol or command system via the FC <b>600</b> or the network <b>700</b>.
P-0256[0256] Also, although it is supposed in this embodiment for LU management to be carried out by the LU pool manager <b>900</b> and processing regarding relocation to be accomplished by the manager <b>130</b>, in another embodiment the manager <b>130</b> may perform both types of processing.
P-0257[0257] This embodiment, in processing LU relocation, can facilitate the management and selection of the relocation destination LU to reduce the burden on the user, and thereby make the management of the computer system easier.
P-0258[0258] Next will be described a fifth embodiment of the present invention.
P-0259[0259] As illustrated in FIG. 33, the computer system in this fifth embodiment has a configuration in which the computer system in the third embodiment illustrated in FIG. 25 uses, to include, a LU area range table <b>195</b>, which is created by adding new items to the LU area range table <b>192</b>, to let the host <b>100</b> read from or write into a file in compliance with a read/write request from a client <b>800</b>, and process data transfers to or from the client <b>800</b> via the network <b>700</b>.
P-0260[0260] As such file sharing protocols via the network <b>700</b>, the NFS (Network File System) and CIFS (Common Internet File System) are extensively used, and the use of any such protocol and the extensively disseminated network <b>700</b> makes possible ready realization of a file sharing environment. In this embodiment as well, the use of the NFS or the CIFS is implemented. In an embodiment files on the host or server or disk arrays, appear as if they are on the client. Hence the client has transparent access to storage on a network <b>700</b> using, for example, a TCP/IP or an IPX, network protocols.
P-0261[0261] The LU area range table <b>193</b> in this embodiment is illustrated in FIG. 34.
P-0262[0262] In FIG. 34, the type of use <b>3510</b> refers to distinction between an area read/write from or to which is accomplished via the FC <b>600</b> as in the third embodiment and an area it is carried out via the network <b>700</b> as will be described with reference to this embodiment.
P-0263[0263] This type of use <b>3510</b> can also include distinction between an area for use in the configuration and method for LU relocation as in the first embodiment (a read/write request in this case is made via the FC <b>600</b>) and an area read/write from or to which is accomplished via the network <b>700</b>. It may further include information on unused areas.
P-0264[0264] Description of the other items including the intra-area address, disk array number, ID, LUN and intra-LU address is dispensed with, because they are the same as their respective counterparts in the description of the third embodiment.
P-0265[0265] Collective management of LUs using this LU area range table <b>193</b> enables the file system <b>110</b> to manage the LUs as a plurality of areas differentiated at least by the type of use.
P-0266[0266] By setting the LU area range table <b>193</b> as described above, the host <b>100</b> can distinguish a request from a client <b>800</b> whether it is for access by the method described with reference to the third embodiment or for access by the above-described method, i.e. access via the network <b>700</b>, according to the protocol or the like the request uses, and handles areas consisting of LUs according to this type of use <b>3510</b>.
P-0267[0267] Thus, as the host <b>100</b> processes the third embodiment and files and areas accessed in this embodiment as distinguished between each other, there is no conflict which would otherwise result from the coexistence of different methods of access to the same file or area.
P-0268[0268] Further, as the host <b>100</b> applies similar distinction in search for an accessible file, if for instance there is a request for access from a client <b>800</b> to files in the same disk array, e.g., <b>200</b>-<b>1</b>, the host <b>100</b> distinguishes the type of use <b>3510</b> of the file requested by the client <b>800</b> and does not notify the client <b>800</b> of files of any other type of use <b>3510</b> than the type of use <b>3510</b> of the file requested by the client <b>800</b>, which accordingly is notified only of a file accessible by its own method of access. Therefore in this system, shared files can be readily managed.
P-0269[0269] Moreover, by also distinguishing areas used in the configuration and method of LU relocation as in the first embodiment (read/write is via the FC <b>600</b>) and areas read/write from or to, whose LUs is accomplished via the host <b>100</b> and the network <b>700</b>, the above-described advantages can be achieved for all these types of use. Also, the user can freely set the LU area range table <b>193</b> from the host <b>100</b> or the remote computer <b>400</b>.
P-0270[0270] In addition, although in the foregoing description a file-sharing protocol such as the NFS or the CIFS is supposed to be used via the network <b>700</b> and data are to be transferred between the host <b>100</b> and the clients <b>800</b> via the network <b>700</b>, in an alternative embodiment, the process may be done via the FC <b>800</b> instead of the network <b>700</b>.
P-0271[0271] In yet another embodiment, the read/write requests the clients <b>800</b> make may be processed for LUs via the host <b>100</b> and the network <b>700</b> as in the above-described processing.
P-0272[0272] In this processing, the host <b>100</b>, as in the searching of file storage areas in the above-described processing, the area from or to which read/write is requested by a client <b>800</b> is located by using the LU area range table <b>192</b>, data is read and transferred to the client <b>800</b> via the network <b>700</b>, or data is received from the client <b>800</b> via the network <b>700</b> and written.
P-0273[0273] Processing by the host <b>100</b> when the application <b>840</b> of a client <b>800</b> reads from a file stored in a disk array <b>200</b> will now be described with reference to FIG. 35 of an embodiment of the present invention.
P-0274[0274] As in the third embodiment, the file system <b>110</b> of the host <b>100</b> having received a read notification locates the LU and the intra-LU area in which the pertinent file is stored by referencing the LU area range table <b>193</b> and the metadata (<b>2900</b>), locks the file against other write requests (<b>2910</b>), reads data in the file (<b>2920</b>), transfers the read contents to the client <b>800</b> via the network <b>700</b> (<b>2930</b>), updates the date of accessing the file on the metadata (<b>2940</b>), releases the lock (<b>2950</b>), and notifies the client <b>800</b> of the completion of read (<b>2960</b>).
P-0275[0275] Processing by the host <b>100</b> when the application <b>840</b> writes into a file stored in a disk array <b>200</b> will now be described with reference to FIG. 36 of an embodiment of the present invention.
P-0276[0276] The host <b>100</b> having received a write notification receives data to be written from the client <b>800</b> via the network <b>700</b> (<b>3000</b>), locates the LU and the intra-LU area in which the pertinent file is stored by referencing the LU area range table <b>192</b> and the metadata (<b>3010</b>), locks the file (<b>3020</b>), and writes the aforementioned data into the file (<b>3030</b>). On this occasion, if necessary, it updates the metadata to add areas in which files are used.
P-0277[0277] After that, the host updates the data of file updating and that of access on the metadata (<b>3040</b>), releases the aforementioned lock (<b>3050</b>), and notifies the client <b>800</b> of the completion of write (<b>3060</b>).
P-0278[0278] Next will be described, with reference to FIG. 37, processing that takes when the application <b>840</b> or the OS <b>820</b> of a client <b>800</b> makes an inquiry about the presence of any accessible file of an embodiment of the present invention.
P-0279[0279] At the request of the application <b>840</b> or the OS <b>820</b>, the OS <b>820</b> inquires of the host <b>100</b> via the network <b>700</b> about the presence of any accessible file (<b>3100</b>).
P-0280[0280] The file system <b>110</b> of the host <b>100</b> so notified locates any such file by referencing the LU area range table <b>193</b> and the metadata (<b>3110</b>), and notifies the client <b>800</b> of the file name and other information on each such file (<b>3120</b>).
P-0281[0281] Processing as described above enables the clients <b>800</b> and the host <b>100</b> to use files stored in the disk arrays <b>200</b> in a shared manner via the host <b>100</b>. The method of relocation and other manners of processing are the same as their respective counterparts in the third embodiment.
P-0282[0282] Relocation processing is supposed to be accomplished in an area of each type of use. This makes possible, even in an environment in which data stored in the disk arrays <b>200</b> are used in a shared manner, that the physical relocation of files can be accomplished among a plurality of disk arrays <b>200</b> without intervention by the host application <b>140</b> or the client application <b>840</b>.
P-0283[0283] Although the above functionality has generally been described in terms of specific hardware and software, it would be recognized that the invention has a much broader range of applicability. For example, the software functionality can be further combined or even separated. Similarly, the hardware functionality can be further combined, or even separated. The software functionality can be implemented in terms of hardware or a combination of hardware and software. Similarly, the hardware functionality can be implemented in software or a combination of hardware and software. Any number of different combinations can occur depending upon the application.
P-0284[0284] Many modifications and variations of the present invention are possible in light of the above teachings. Therefore, it is to be understood that within the scope of the appended claims, the invention may be practiced otherwise than as specifically described.
Contents5
33 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17 Sheet 18 Sheet 19 Sheet 20 Sheet 21 Sheet 22 Sheet 23 Sheet 24 Sheet 25 Sheet 26 Sheet 27 Sheet 28 Sheet 29 Sheet 30 Sheet 31 Sheet 32 Sheet 33
Every citation, both ways
| Document | Relation | Office | Cited during |
|---|---|---|---|
| US7096336B2 | Cited by | United States of America | Applicant |
| US8010558B2 | Cited by | United States of America | Search report |
| US8806657B2 | Cited by | United States of America | Applicant |
| US7634588B2 | Cited by | United States of America | Applicant |
| US7231466B2 | Cited by | United States of America | Applicant |
| US2005015384A1 | Cited by | United States of America | Pre-grant |
| US7412543B2 | Cited by | United States of America | Applicant |
| US2005193167A1 | Cited by | United States of America | Pre-grant |
| US7502904B2 | Cited by | United States of America | Applicant |
| US7290103B2 | Cited by | United States of America | Applicant |
| US9430168B2 | Cited by | United States of America | Search report |
| US7493466B2 | Cited by | United States of America | Applicant |
| US7441095B2 | Cited by | United States of America | Applicant |
| US8122214B2 | Cited by | United States of America | Applicant |
| US7552294B1 | Cited by | United States of America | Applicant |
| US7051121B2 | Cited by | United States of America | Applicant |
| US7660866B2 | Cited by | United States of America | Applicant |
| US8693764B2 | Cited by | United States of America | Applicant |
| US7937513B2 | Cited by | United States of America | Applicant |
| US8156561B2 | Cited by | United States of America | Applicant |
| US7418562B2 | Cited by | United States of America | Applicant |
| US2004103261A1 | Cited by | United States of America | Pre-grant |
| US7617256B2 | Cited by | United States of America | Search report |
| US2005060505A1 | Cited by | United States of America | Pre-grant |
| US2011161460A1 | Cited by | United States of America | Pre-grant |
| US9606874B2 | Cited by | United States of America | Applicant |
| US8572352B2 | Cited by | United States of America | Applicant |
| US2007036444A1 | Cited by | United States of America | Pre-grant |
| US10289338B2 | Cited by | United States of America | Applicant |
| US2005138308A1 | Cited by | United States of America | Pre-grant |
| US8607010B2 | Cited by | United States of America | Applicant |
| US2006026165A1 | Cited by | United States of America | Pre-grant |
| US7111138B2 | Cited by | United States of America | Applicant |
| US10534681B2 | Cited by | United States of America | Applicant |
| US2005138313A1 | Cited by | United States of America | Pre-grant |
| US2006090049A1 | Cited by | United States of America | Pre-grant |
| US2005160222A1 | Cited by | United States of America | Pre-grant |
| US2005102479A1 | Cited by | United States of America | Pre-grant |
| US8443138B2 | Cited by | United States of America | Applicant |
| US7200727B2 | Cited by | United States of America | Search report |
| US2005246491A1 | Cited by | United States of America | Pre-grant |
| US7430648B2 | Cited by | United States of America | Applicant |
| US2010161560A1 | Cited by | United States of America | Pre-grant |
| US7203806B2 | Cited by | United States of America | Applicant |
| US2008092053A1 | Cited by | United States of America | Pre-grant |
| US7840767B2 | Cited by | United States of America | Applicant |
| US12032849B2 | Cited by | United States of America | Search report |
| US7975116B2 | Cited by | United States of America | Applicant |
| US7661019B2 | Cited by | United States of America | Applicant |
| US7865768B2 | Cited by | United States of America | Applicant |
| US2006041777A1 | Cited by | United States of America | Pre-grant |
| US2014297944A1 | Cited by | United States of America | Pre-grant |
| US7827369B2 | Cited by | United States of America | Applicant |
| US7155587B2 | Cited by | United States of America | Applicant |
| US2005138315A1 | Cited by | United States of America | Pre-grant |
| US2004181707A1 | Cited by | United States of America | Pre-grant |
| US7707377B2 | Cited by | United States of America | Applicant |
| US2006010502A1 | Cited by | United States of America | Pre-grant |
| US7130941B2 | Cited by | United States of America | Applicant |
| US7694104B2 | Cited by | United States of America | Applicant |
| US7177991B2 | Cited by | United States of America | Applicant |
| US2010146045A1 | Cited by | United States of America | Pre-grant |
| US9213497B2 | Cited by | United States of America | Search report |
| US7162603B2 | Cited by | United States of America | Applicant |
| JP2012198627A | Cited by | Japan | Search report |
| US8688838B2 | Cited by | United States of America | Applicant |
| US2005193168A1 | Cited by | United States of America | Pre-grant |
| US2006123213A1 | Cited by | United States of America | Pre-grant |
| US7133988B2 | Cited by | United States of America | Applicant |
| US7124267B2 | Cited by | United States of America | Applicant |
| US6925541B2 | Cited by | United States of America | Applicant |
| US8155431B2 | Cited by | United States of America | Applicant |
| US7624241B2 | Cited by | United States of America | Applicant |
| US2007150680A1 | Cited by | United States of America | Pre-grant |
| US2007192558A1 | Cited by | United States of America | Pre-grant |
| US2005166023A1 | Cited by | United States of America | Pre-grant |
| US7716434B2 | Cited by | United States of America | Applicant |
| US2006059301A1 | Cited by | United States of America | Pre-grant |
| US2006253549A1 | Cited by | United States of America | Pre-grant |
| US7457929B2 | Cited by | United States of America | Applicant |
| US7457899B2 | Cited by | United States of America | Applicant |
| US8489835B2 | Cited by | United States of America | Applicant |
| US2005114599A1 | Cited by | United States of America | Pre-grant |
| US2006277378A1 | Cited by | United States of America | Pre-grant |
| US7209986B2 | Cited by | United States of America | Applicant |
| US2008016303A1 | Cited by | United States of America | Pre-grant |
| US7373670B2 | Cited by | United States of America | Applicant |
| US7216209B2 | Cited by | United States of America | Applicant |
| US6842793B2 | Cited by | United States of America | Search report |
| US7363461B2 | Cited by | United States of America | Applicant |
| US2004205310A1 | Cited by | United States of America | Pre-grant |
| US7165163B2 | Cited by | United States of America | Search report |
| US2003221077A1 | Cited by | United States of America | Pre-grant |
| US7337292B2 | Cited by | United States of America | Applicant |
| US2007021211A1 | Cited by | United States of America | Pre-grant |
| US2003188058A1 | Cited by | United States of America | Pre-grant |
| US2007192554A1 | Cited by | United States of America | Pre-grant |
| US2008250201A1 | Cited by | United States of America | Pre-grant |
| US2012059854A1 | Cited by | United States of America | Pre-grant |
| US8412977B2 | Cited by | United States of America | Applicant |
14 members in 3 offices
Priority claims1
| Document | Office | Kind | Date |
|---|---|---|---|
| 2000205510 | Japan | A |
Members14
| Document | Office | Kind | |
|---|---|---|---|
| EP1170657A2 | European Patent Office (EPO) | A2 | |
| US2002004857A1 | United States of America | A1 | |
| JP2002082775A | Japan | A | |
| US2002184463A1 | United States of America | A1 | |
| US6763442B2 | United States of America | B2 | |
| US6766430B2 | United States of America | B2 | |
| US2004236772A1 | United States of America | A1 | |
| EP1170657A3 | European Patent Office (EPO) | A3 | |
| US7260696B2 | United States of America | B2 | |
| US2007266216A1 | United States of America | A1 | |
| JP2008152807A | Japan | A | |
| JP4115093B2 | Japan | B2 | |
| US7953949B2 | United States of America | B2 | |
| JP4862006B2 | Japan | B2 |
41 transactions on the USPTO file
Allowed after 1 non-final rejection.
- Non-final rejections
- 1
- Final rejections
- 0
- RCEs
- 0
- Appeals
- 0
Over time
Point at a mark for the transactionTransactions
| Event | Code | |
|---|---|---|
| Expire PatentEXP. | EXP. | |
| Recordation of Patent Grant MailedPGM/ | PGM/ | |
| Patent Issue Date Used in PTA CalculationAllowedPTAC | PTAC | |
| Issue Notification MailedAllowedWPIR | WPIR | |
| Receipt into PubsR1021 | R1021 | |
| Dispatch to FDCD1935 | D1935 | |
| Application Is Considered Ready for IssuePILS | PILS | |
| Issue Fee Payment VerifiedN084 | N084 | |
| Issue Fee Payment ReceivedIFEE | IFEE | |
| Receipt into PubsR1021 | R1021 | |
| Receipt into PubsR1021 | R1021 | |
| Workflow - File Sent to ContractorSENT | SENT | |
| Receipt into PubsR1021 | R1021 | |
| Dispatch to PublicationsD1220 | D1220 | |
| Mail Notice of AllowanceAllowedMN/=. | MN/=. | |
| Mail Notification of Terminal Disclaimer - AcceptedMN574 | MN574 | |
| Notice of Allowance Data Verification CompletedAllowedN/=. | N/=. | |
| Notification of Terminal Disclaimer - AcceptedN574 | N574 | |
| Date Forwarded to ExaminerFWDX | FWDX | |
| Terminal Disclaimer FiledDIST | DIST | |
| Response after Non-Final ActionA... | A... | |
| Request for Extension of Time - GrantedXT/G | XT/G | |
| Mail Non-Final RejectionNon-final rejectionMCTNF | MCTNF | |
| Non-Final RejectionNon-final rejectionCTNF | CTNF | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Transfer Inquiry to GAUTI1050 | TI1050 | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Application Dispatched from OIPEOIPE | OIPE | |
| Application Is Now CompleteCOMP | COMP | |
| Notice Mailed--Application Incomplete--Filing Date AssignedINCD | INCD | |
| Correspondence Address ChangeC.AD | C.AD | |
| IFW Scan & PACR Auto Security Review | – | |
| Workflow - Drawings FinishedDRWF | DRWF | |
| Workflow - Drawings Matched with File at ContractorDRWM | DRWM | |
| Request for Foreign Priority (Priority Papers May Be Included)RQPR | RQPR | |
| Information Disclosure Statement (IDS) Filed | – | |
| Information Disclosure Statement (IDS) Filed | – | |
| Initial Exam Team nnIEXX | IEXX |
11 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Lapsed due to failure to pay maintenance feeLapsedFP | FP | |
| Information on status: patent discontinuationPATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362STCH | STCH | |
| Information on status: patent discontinuationPATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362STCH | STCH | |
| Lapse for failure to pay maintenance feesLapsedLAPS | LAPS | |
| Maintenance fee reminder mailedREMI | REMI | |
| Fee paymentFPAY | FPAY | |
| Fee payment procedurePAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITYFEPP | FEPP | |
| Fee payment procedurePAYER NUMBER DE-ASSIGNED (ORIGINAL EVENT CODE: RMPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITYFEPP | FEPP | |
| Fee paymentFPAY | FPAY | |
| Fee payment procedurePAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITYFEPP | FEPP | |
| AssignmentAS | AS |
Numbers
- Application
- 83507301
Titles
- English
- Computer system
Patent term adjustment
- A delay
- +478 daysthe office missed an examination deadline
- Applicant delay
- −26 days
- Net adjustment
- 452 days
Classification
- CPC, 14
- G06F3/0613
- G06F3/0605
- G06F3/0608
- G06F3/0631
- G06F3/0635
- G06F3/0647
- G06F3/0659
- G06F3/067
- G06F11/1096
- G06F11/2087
- G06F12/0866
- H04L61/457
- Y10S707/99955
- Y10S707/99953
- IPC, 4
- G06F3 00
- G06F3 06
- G06F7 00
- G06F12 00