Transparent file migration using namespace replication

Cross-reference of related applications This application claims the following priority under Article 35U.SC119 (e). US Provisional Patent Application No. 60 / 465,579, filed April 24, 2003, by Thomas K. Wong et al., entitled "Methods and Devices for Transparent File Movement Using Namespace Replication Techniques". US Provisional Patent Application No. 60 / 465,578, filed April 24, 2003, by Thomas K. Wong et al., entitled "Methods and Devices for Transparent File Duplication Using Namespace Replication Techniques". US Provisional Patent Application No. (Agent Reference No. # 23313-07962), filed by Thomas K. Wong et al. (Same date as this application), entitled "Transparent File Reproduction Using Namespace Reproduction". The entire of each of these applications is incorporated herein by reference.

The present invention relates generally to storage networks, and more specifically to network devices that use namespace replication to move files between file servers within a distributed storage network.

In computer networks, NAS (Network Attached Storage) file servers that connect directly to the network provide a low-cost, easy-to-configure solution for storage networks. These NAS file servers are self-sufficient. This is because they include file systems that can interoperate with clients running any operating system and communicate using open protocols. For example, Unix-based clients use the NFS (Network File System) protocol from Sun Microsystems, Santa Clara, California, and Windows-based clients use Microsoft's CIFS (Common Internet Files) in Redmond, Washington. You can access the files on the NAS file server using System). However, the operating system does not affect the communication between the client and the file server. Therefore, the NAS file server will provide truly universal file access.

In contrast, more expensive and powerful SAN (Storage Area Network) file servers use fiber channels on the back end, that is, resources connected by a dedicated network. A SAN file system is an application that runs on a part of the operating system or on a client. However, a heterogeneous client operating system may request that an additional copy of each file be stored on the storage network to ensure SAN file server compatibility. In addition, since communication between file servers on the SAN uses a unique protocol, it is usually provided by a common vendor. As a result, NAS file servers are suitable when price and convenience are the main motives. However, the advantages of NAS storage networks over SAN storage networks also have drawbacks.

One of the drawbacks of NAS file servers is that they are not centrally managed. Therefore, each client must maintain a separate communication channel between each of the NFS file servers. When a NAS file server is added or removed from a storage network, each client must properly mount or unmount the directory of relevant storage resources. This is especially inefficient when there are changes in the hardware but no changes in the specific files available on the network, such as when replacing a failed NAS file server with a backup NAS file server with the same configuration. Is the target.

A related drawback is that the client must be reconfigured each time a file is relocated within the storage network, such as during a file move or file duplication. The client creates a NAS file handle that identifies the physical location of a directory or file object on the file server. To access the object, the client sends an object access request directly to the NAS file server. When relocating a file to another NAS file server, a request to access the following file requests a new collation, the file is placed, and a new NAS file handle is generated.

A further drawback is that the NAS file server cannot be accessed during mass data transfer operations such as file movement and duplication. Such data transfer is usually performed during non-business hours to reduce downtime as a result. However, if the storage capacity is further increased, the amount of time required for data transfer increases. In addition, many businesses and applications need data that is always available.

Therefore, what is needed is a network device that provides transparency to the client of a file server such as a NAS file server. Further, there is a need for a network device that can move files without the client reconfiguring. In addition, there is a need for network devices that provide access to data during file movement.

The present invention meets these needs by providing clients with file movement within a transparent distributed storage network. The NAS switch in the data path of the client and NAS file server ensures that the source file server's file movement to the destination file server is coordinated using namespace replication to track new file locations. In addition, NAS switches maintain data availability during time-consuming data transfers.

An embodiment of a system configured according to the present invention includes a NAS switch that communicates with both a client at the front end of a storage network and a source file server and a destination file server at the back end. The NAS switch associates the NAS file handle it receives from the source and destination file servers (for example, a CIFS file handle or an NFS file handle) with a location-independent switch file handle. The NAS switch then exports the switch file handle to the client. In response to subsequent object access requests from clients, the NAS switch replaces the switch file handle with the appropriate NAS file handle for submission to the appropriate NAS file server.

In another embodiment, the NAS switch further comprises a move module that coordinates the movement of the source file located on the source file server to the destination file located on the destination file server. The move module performs namespace replication and data replication separately. Namespace replication copies the namespace in the directory hierarchy on the source file server to the destination file server. Namespace replication can also use stored file handles as pointers from the source file server for files that are moved to the destination file server, or as pointers from the destination file server for files that are further moved from the source file server. Can include. The move module then moves the data and swaps the stored file handles.

In yet another embodiment, the move module updates the file move table when the move of the object is successful. Therefore, the move module inputs the location of the object on the source file server and the location of the object on the destination file server. Upon receiving the object access request, the NAS switch searches the roaming table by the switch file handle. If there is a match, the NAS switch sends an object access request to a location on the destination file server. Otherwise, the NAS switch will send an object access request to a location on the source file server. The move module has the advantage of providing move services to distributed file servers and file servers that do not originally support move.

In yet another embodiment, during the move, the move module replicates the namespace access request to both the namespace on the source file server and the replicated namespace on the destination file server.

The present invention provides clients with file movement within a transparent storage network. The NAS (Network Attached Storage) switch in the data path of the NAS file server on the client and storage network uses namespace replication to coordinate file movement between distributed servers while maintaining data availability to the client. Some embodiments of the system will be described with reference to FIGS. 1 to 2, and some embodiments of the method operating there will be described with reference to FIGS. 3 to 10.

Ancillary explanations are intended to provide a complete explanation with a variety of specific details. Of course, in the field of storage networks, it goes without saying that many different modifications can be made to the features of the invention disclosed and described. What is undoubtedly clear to those proficient in the art is that the present invention can be carried out without some specific detailed description described below and is self-evident, but many other of the present invention. It is true that modifications and embodiments can be carried out while satisfying their teachings and spirits. For example, the present invention describes a storage network that operates based on the NAS protocol, but can be similarly implemented in future protocols for distributed storage networks other than NAS, or mixed protocol networks. Therefore, it should be understood that the present invention is not limited to the particular practices described below, but only to the scope of the following claims.

The processing, features, or functions of the present invention can be performed by program instructions executed by an appropriate arithmetic unit. Examples of computing devices include corporate servers, application servers, workstations, personal computers, network computers, network home appliances, mobile information terminals, game consoles, televisions, set-top boxes, on-site automatic equipment, POS terminals, automobiles, and personal communications. Equipment is included. Program instructions can be distributed on computer-readable media, storage volumes, or the Internet. Program instructions can take any suitable form, such as source code, object code, or scripts.

FIG. 1 is a high-level block diagram showing a storage network system 100 according to an embodiment of the present invention. System 100 includes a NAS switch 110 connected to network 195 and a client 140. The NAS switch 110, the source file server 120, and the destination file server 130 are communicated with each other via the sub-network 196. Note that it is possible to configure a variety of systems 100, such as embodiments that include additional clients 140, additional source and / or destination file servers 120, 130, and additional NAS switches 110. The components of System 100 are, for example, operating systems and / or personal computers with x86 processors running application programs, workstations, optimized operating systems, and / or special NAS devices with application programs, modified server blades. Etc. are implemented. In one embodiment, the storage network 175 comprises a NAS that uses protocols such as NFS and CIFS. In another embodiment, the storage network 175 comprises a combination of NAS, SAN and other types of storage networks. In yet another embodiment, the storage network 175 comprises a distributed standard or proprietary non-NAS storage system.

The NAS switch 110 provides the client 140 with continuous transparency in terms of object management. In particular, NAS switches can offload tasks related to physical configuration, object management, object movement, object replication, efficient storage, and / or other services on the storage network 175. Preferably, the NAS switch 110 emulates the file server processing to the client 140 and the client processing to the file servers 120, 130. Therefore, the NAS switch 110 makes the intended NAS request to the source file server 120 in the proper location on the destination file server 130.<u style="single">(Ie storage position)</u>Client 140 is unaware of NAS Switch 110 because it can be redirected to. In this way, the client 140 submits an object request such as writing a file or reading a directory directly to the NAS switch 110. Similarly, since the NAS switch 110 can submit the request contained in the server file handle as if it originated from the client 140, the file servers 120 and 130 are unaware of the NAS switch 110. To that end, the NAS switch 110 uses mapping, code translation, bridging, packet sending, other network interface functions, and other management processes to perform file handle switching, so that the client 140 is the physical location of the file.<u style="single">(Storage position)</u>Frees you from the need to track changes.

In one embodiment, the NAS switch 110 comprises a client module 112 and a file server module 114 to facilitate communication and file handle switching. Client module 112 receives the file system directory exported from file servers 120, 130, including the NAS switch handles. To create compatibility between the client 140 and the NAS switch 110, the client module 112 maps the file system directory to the internal switch file system that sends to the client 140. To request an object, client 140 reviews the exported switch file system and selects a switch file handle to send to the NAS switch 110 along with the requested operation.

File server module 114 coordinates the move process. File server module 114 initiates tasks that the source and destination file servers 120, 130, which may not inherently have the ability to move, perform passively. The file server module 114 replicates the namespace containing the data to be moved from the source file server 120 to the destination file server 130, and then replicates the relevant data. In the meantime, and after that, the file server module 114 redirects the namespace and file object access requests from client 140 to the appropriate location. Therefore, the data transfer service remains available to client 140.

In one embodiment, the file server module 114 also tracks reconfigurations caused by move, duplicate, and other object relocation operations (eg, add or remove file server capacity), as well as a nested system of tables. Otherwise it will be done with the information linked to the switch file system. The switch file handle is static because it persists through the relocation process, but the NAS file handle involved can be dynamic because it is chosen based on the current location of the object. To keep track of various copies of an object, the file server module 114 maps the NAS file handles of the moved and duplicated objects to locations on the storage network 175. File handle move tables and file handle replicas for each file system. Keep the table. Further embodiments of the file server module 114 will be described with reference to FIG.

Client module 112 associates the NAS file handle with the switch file handle, as described below with respect to FIG. 4 (310). This allows the NAS switch 110 to act as an intermediary between the client 140 and the file servers 120, 130. The client 140 submits a NAS request using the switch file handle as if the NAS switch 110 were the file servers 120, 130, and then the file servers 120, 130 submit the NAS file handle from the NAS switch 110. Is processed as if it was submitted by the client 140.

In general, the NAS file handle uniquely identifies an object such as a directory file server on file servers 120, 130, such as a directory or file, as long as the object exists. The NAS file handle is unique to the file server and is valid only for the file servers 120 and 130 that issued the file handle. The process of acquiring a file handle from a file name is called collation. The NAS file handle may be formatted according to a protocol such as NFS or CIFS, with reference to, for example, Tables 1A and 1B, as described in more detail below. In contrast, the switch file handle identifies a directory or file object that is irrelevant to its location and persists through file duplication, movement, and other data transfers. The switch file handle can be a modified NAS file handle that references the internal system inside the NAS switch 110 instead of the source file server 120. This enables the NAS switch 110 when mapping the persistent file handle to the selection of the alternate NAS file handle. The original NAS file handle points to the first object location on the source file server 120. The stored NAS file handle points to a NAS file handle that points to an alternative file location stored as an object on file servers 120, 130.

Object access requests handled by NAS Switch 110 include, for example, reading, writing, creating, deleting, moving, and copying directories and / or files. Namespace access refers to operations that access or modify namespaces, such as collating, rewriting, deleting, or creating. File access refers to operations such as reading or writing to access or modify a file. Objects may refer to directory objects or file objects. Directory objects may also include subdirectories and file objects within the directory. As used herein, various terms that refer to the location of the object before the move are used as synonyms (eg, "primary," "source," "original," and "primary") to move. Use various terms that refer to the location of the same object later (eg, "duplicate", "destination", "replacement", and "second"). A further embodiment of the NAS switch 110 and a method of operating the NAS switch 110 will be described below.

The client 140 accesses the resources on the file servers 120 and 130 by submitting the intended switch file handle to the source file server 120 to the NAS switch 110. To find the switch handle, client 140 first mounts the exported switch file system that contains the switch file handle. Client 140 matches the object, gets its file handle, and submits the relevant request. From the perspective of client 140, transactions are executed by file servers 120, 130 that have unchanged object locations. In this way, the client 140 interacts with the NAS switch 110 before and after file replication in the same way. Users of Client 140 can submit operations via a command line interface, Windows environment, software application, or other means. In one embodiment, the NAS switch 110 further provides access to a storage network 175 other than the NAS storage network.

The source file server 120 is the default, or original network file server, for the client 140 before the file move. The source file server 120 further comprises a source object 125, which contains directories or files in the namespace such as corporate data, records, database information, applications, and the like. The source file server 120 can store a table of roaming directories maintained by the NAS switch 110 that correlates with the consequences of namespace roaming. Further, the source file server 120 can store a file handle movement table maintained by the NAS switch 110 and displayed for each movement directory and file object. The source file server 120 includes, for example, a personal computer using an x86 type processor having an operating system and / or an application, a workstation, a special NAS device having an optimized operating system and / or an application, a modified server blade, and the like.

The destination file server 130 becomes the primary network file server used by the NAS switch 110 after the file is moved. The destination file server 130 further comprises a destination object 135, which contains a replicated namespace directory and source files. The destination file server 130 may include the same hardware and / or software as described for the source file server 120. The source and destination file servers 120 and 130 are preferably NAS file servers, but can also be file servers using other distributed protocols that do not originally support file movement. Further embodiments of source and destination file servers 120, 130, and related methods are described below.

Network 195 facilitates data transfer between connected hosts (eg 110, 140). The connection to network 195 may be wired and / or wireless, packet and / or circuit switch, TCP / IP (Transmission Control Protocol / Internet Protocol), IEEE (Institute of Electrical and Electronics Engineers) 802.11, IEEE 802.3 (ie. , Ethernet), ATM (Asynchronous Transfer Mode) and other network protocols may be used. The network 195 includes, for example, a LAN (local area network), a WAN (wide area network), the Internet, and the like. In one embodiment, the NAS switch 110 acts as a gateway between the client 140 connected to the Internet, the directory file server 120 connected to the LAN, and the shadow file server 130. Subnetwork 196 is preferably a local area network that provides the NAS switch 110 with optimal response time. In one embodiment, the subnet 196 is integrated into the network 195.

FIG. 2 shows a file server module according to an embodiment of the present invention.<u style="single">114</u>It is a block diagram which shows. The file server module 114 includes a file server interface 210, a mobile module 220, and a redirection module 230. In general, the file server interface 210 manages pre-move operations, the move module 220 maintains data availability during the move, and the redirection module 230 provides the client 140 with post-move transparency. Note that modules are just an example of functional groups.

Prior to the move, the file server interface 210 receives the switch file handle at the request of the client 140 used to find the original NAS file handle. The file server interface 210 submits the original NAS file handle upon request to the source file server 120. If the object has not yet relocated the storage network 175, the file server interface 210 uses the original NAS file handle. The file server interface 210 can submit the switch file handle to the move module 220 to determine if the object is part of a data move. Also file server interface<u style="single">210</u>Can submit the switch file handle to the redirection module 230 to determine if the object has completed the data movement. In either case, the file server interface 210 returns the appropriate NAS file handle to use to send client requests to the appropriate file servers 120, 130.

During the move, the move module 220 in the NAS switch 110 coordinates the move from the source file server 120 to the destination file server 130 using namespace replication. Namespace replication copies the directory metadata of the source file server 120 separately from the data itself. Move module 220 moves directory<u style="single">Check it all over</u>When copying data, the move module updates the file handle move table, which shows if the object has moved and, if so, where the object has moved. Namespace replication is many times faster than data movement, so directory services remain available while data movement occurs. The file handle move table provides a current list of which objects have moved, so the NAS switch 110 can provide almost continuous data availability. In one embodiment, the NAS switch can reconfigure the file handle move table, for example, in the event of a device crash or data corruption.

After moving the file, the redirection module 230 matches the switch file handle received from the client 140 in the file handle move table. When the object is moved, the redirection module outputs the destination NAS file handle corresponding to the location on the destination file server 130.

FIG. 3 is a high-level flow diagram showing a method (300) of providing transparent file movement within a NAS storage network according to an embodiment of the present invention. Client module 112 associates the original NAS file handle with the switch file handle, as described below with respect to FIG. 4 (310). This allows the NAS switch 110 to act as an intermediary between the client 140 and the file servers 120, 130. Client 140 submits NAS requests using the switch file handle as if NAS switches 110 were file servers 120, 130, and then file servers 120, 130 submit them as if they were submitted by client 140. Handle the NAS file handle from the NAS switch 110 as if.

The move module 220 uses namespace duplication to perform file moves, as described below for FIGS. 5-9 (320). By separating directory replication from data replication, the move module 220 can maintain availability and data integration between file servers 120 and 130.

The redirection module 230 redirects NAS requests for moved files, as described below in Figure 10 (330). Since the NAS switch 110 adjusts and stores the elements related to movement, the client 140 continues to refer to the object stored in the alternative location by the same switch file handle used before the replication. However, many changes occur on the back end. For example, file movement is from the source file server 120 to the destination file server.<u style="single">130</u>Relocate the source object 125 to.

FIG. 4 is a flow chart showing a method (310) of associating a NAS file handle with a switch file handle according to an embodiment of the present invention. First, the NAS switch 110 mounts the file system directory exported from the primary server 120 (410). In general, the file system organizes objects on file servers 120, 130 into a directory hierarchy of NAS file handles. In one embodiment, the NAS switch 110 receives the exported directory from the associated source file server 120 and then sends the exported directory to the associated client 140.

Client module 112 generates a switch file handle that is independent of the object location in the primary file server 120 (420). Client module 112 systematizes the exported file system from file server 120 by replacing the file system or tree identifier with switch file system numbers as shown below in Tables 2A and 2B. Client module 112 exports the switch file system to client 140 for use in request operations (430). In the reverse process, the NAS switch 110 receives the NAS request and searches for the duplicated file handle and / or the duplicated namespace using the NAS file handle. Therefore, file server interface 210 checks for nested table entries maintained by synchronization module 230. The file server interface 210 generates a NAS file handle from the switch file handle based on the object location. Examples of the contents of NFS and CIFS file handles are shown in Tables 1A and 1B, while switch file handles or modified NFS and CIFS file handles are shown in Tables 2A and 2B.<tables num="1"><img file="JP4588024B2_D0001.tif" /></tables><tables num="2"><img file="JP4588024B2_D0002.tif" /></tables>

After moving the object, the NAS switch 110 can access the object in the new location using the updated NAS file handle, as described below.

FIG. 5 is a flow chart showing a method (220) of performing file movement using namespace duplication according to an embodiment of the present invention. The move module 220 replicates the directory hierarchy of the source file server 120, as described below with respect to FIGS. 6-7 (510). In one embodiment, when an object access request such as a directory operation is received from the client 140, it is sent to both the first and destination file servers 120, 130, between the namespace and the duplicated namespace. Maintain the consistency of. In this way, directories can be created or deleted during the file move process. On the other hand, in the conventional system, the directory is locked, so such an operation is impossible.

If a critical directory request is issued to the source file server during a file move (510), move module 220 resubmits the request to update the duplicated namespace (530). The duplicated namespace is preferably stored on the destination file server 130. As a result, if operations such as directory creation, file creation, directory deletion, and file deletion affect the source namespace, the same modifications are made to the duplicated namespace. Once directory replication is complete (540), critical directory operations can be submitted directly to the replicated namespace. In another process, the move module 220 copies data from the source file server 120 to the destination file server 130 (550).

FIG. 6 is a flow chart showing a method (510) of replicating a directory hierarchy from a source file server 120 to a destination file server 130 according to an embodiment of the present invention. The playback module 220 selects the current source directory from the directory hierarchy of the source file server 120 and selects the current destination directory from the destination file server 130 (610). The move module 220 adds a mapping entry to the replication table with switch file handles associated with the source and destination locations (620). The move module 220 selects the current object from the list of file and directory objects in the current source directory (630).

If the current object is a directory (530), move module 220 creates a directory in the destination file server 130 with the same name as the current directory in the primary file server 120 (650). On the other hand, if the current object is a file (640), playback module 220 creates a file from the file handle in the current destination directory with the file handle stored for that object (645). In one embodiment, the stored file handle is similar to a switch file handle. Preferably, since the stored file handle is of a predetermined size, the NAS switch 110 can determine whether the file contains the stored file handle simply by checking the file size. Table 3 shows the example stored file formats.<tables num="3"><img file="JP4588024B2_D0003.tif" /></tables>

However, it should be noted that there can be variations in the stored file format. The move module 220 adds a mapping entry to the duplicated file list with source and destination switch file handles (655).

If all objects are processed (660), no error occurs in the process (670), and there are no more directories to duplicate (680), playback module 220 performs namespace replication (690). However, if there are still objects to process (660), move module 220 continues processing from selecting the object (630). If there is an error in creating the directory or file (670), playback module 220 repeats the process from the point of deleting the destination directory (675) and adding a mapping entry (620). Also, if there is still a directory to process (680), the first file server 120 returns to the point of selecting the primary directory (510).

The mobile module 220 is involved in the namespace as shown in FIG. FIG. 7 shows a method of performing namespace duplication according to an embodiment of the present invention (<u style="single">690</u>) Is shown in the flow chart. The move module 220 adds an entry to the move directory table (710). As a result, subsequent object access requests will be sent to the source file server.<u style="single">120</u>Instead, it will be directed to the destination file server 130. The source file server 120 deletes the duplicate table because it is no longer needed (720).

FIG. 8 is a flow chart showing a data copying method (420) according to an embodiment of the present invention. NAS switch 110 selects the current entry in the moved file list (810). The source file server 120 copies the data in the source object 125 to the destination object 135 in the destination file server 130 (820).

If no error occurs during data transfer (830), the destination file server 130 moves the data as shown in Figure 9 (840). FIG. 9 is a flow chart showing a method (840) for moving data according to an embodiment of the present invention. The move module 220 locks the source file to prevent further access to the file (910). The move module 220 replaces the contents of the source file with a stored file handle that indicates a new location on the destination file server 130 (920). Once that is done, the move module 220 marks the current entry in the moved file list (930) and points to the source and destination file handles in the file handle move table, indicating the location on the source and destination file servers 120, 130. Fill in (<u style="single">940</u>). Finally, move module 220 resumes access to the source file (950).

In one embodiment, the mobile module 220 reconfigures the mobile module 220, for example, due to a device crash or data corruption. To that end, the move module 220 takes the namespace of the source file server 120.<u style="single">Check throughout</u>To Since the stored file handle has a consistent size, the move module 220 can immediately recognize the stored file handle and retrieve the pointer information. Add this association to the reconstructed file handle move table entry.

Referencing Figure 8 again, if an error occurs (830), repeat the data transfer. In one embodiment, the destination file server<u style="single">130</u>Detects an error by comparing the file size before the move with the file size after the move.

FIG. 10 is a flow chart showing a method (330) of redirecting a NAS request regarding a moved object according to an embodiment of the present invention. NAS switch 110 is a file system table from the switch file handle<u style="single">In</u>File system ID<u style="single">Find</u>(1010). The NAS switch 110 then moves the file handle move table from the file system ID.<u style="single">Find</u>(1020). If the file has not been moved (1030), NAS Switch 110 sends the client request using the source, the original NAS file handle. If the file is moved (1030), the NAS switch 110 looks up the destination NAS file handle from the file handle move table in the source file server 120.

<figref num="1">It is a high-level block diagram which shows the storage network system according to one Embodiment of this invention.</figref><figref num="2">It is a block diagram which shows the NAS switch which follows one Embodiment of this invention.</figref><figref num="3">FIG. 6 is a high-level flow diagram illustrating a method of providing transparent file movement within a NAS storage network according to an embodiment of the present invention.</figref><figref num="4">It is a flow diagram which shows the method of associating the original NAS file handle with a switch file handle according to one Embodiment of this invention.</figref><figref num="5">It is a flow diagram which shows the method of performing the file movement using namespace duplication according to one Embodiment of this invention.</figref><figref num="6">It is a flow diagram which shows the method of duplicating the directory hierarchy of a source file server according to one Embodiment of this invention.</figref><figref num="7">It is a flow diagram which shows the method of performing namespace duplication according to one Embodiment of this invention.</figref><figref num="8">It is a flow figure which shows the method of data copy according to one Embodiment of this invention.</figref><figref num="9">It is a flow figure which shows the method of performing the data movement according to one Embodiment of this invention.</figref><figref num="10">It is a flow diagram which shows the method of redirecting the NAS request about the moved object according to one Embodiment of this invention.</figref>