Shadowing storage gateway
15 claims: 2 independent, 13 dependent
- 1顧客ネットワーク上のストレージゲートウェイによって、リモートデータストアをサービスプロバイダの顧客に提供するサービスプロバイダに、前記ストレージゲートウェイを登録するプロセスを開始することと、 前記ストレージゲートウェイによって、前記ストレージゲートウェイが顧客ネットワーク上のシャドーイングゲートウェイとして機能するように指定する設定情報を受信することであって、前記シャドーイングゲートウェイが、前記顧客ネットワーク内のローカルデータストア上に格納されたデータを、前記リモートデータストアにシャドーイングし、前記ローカルデータストアが前記データに対する一次データストアとして機能する、設定情報を受信することと、 前記設定情報に応答して、前記ストレージゲートウェイを、前記顧客ネットワーク上の前記シャドーイングゲートウェイとして設定することと、 前記顧客ネットワーク上の1つまたは複数のプロセスから、前記ストレージゲートウェイによって、前記一次データストアに向けられた読取り要求および書込み要求を受信することであって、前記ストレージゲートウェイは、前記顧客ネットワーク上の前記1つまたは複数のプロセスから、ブロックストレージプロトコルを介して、前記読取り要求および書込み要求を受信することと、 前記読取り要求について、前記読取り要求を前記一次データストアに渡すことと、 前記書込み要求について、前記書込み要求を前記一次データストアに渡し、かつ、前記リモートデータストア上の前記一次データストアのスナップショットを書込みデータで更新するために、前記書込み要求によって示される書込みデータを前記サービスプロバイダに送信することとを含む、 方法。
- 2前記ストレージゲートウェイが、前記顧客ネットワーク上の前記一次データストアと一致する、前記一次データストアの前記スナップショットを作成するために、前記ローカルデータストア上に格納されたデータの少なくとも一部を前記サービスプロバイダにアップロードするブートストラッププロセスを開始することをさらに含む、請求項1に記載の方法。
- 3前記読取り要求および書込み要求を受信すること、前記読取り要求および前記書込み要求を前記一次データストアに渡すこと、および前記サービスプロバイダに前記書込み要求によって示される書込みデータを送信することが、前記ストレージゲートウェイによって、前記ブートストラッププロセスと同時に実行される、請求項2に記載の方法。
- 4前記ストレージゲートウェイが、前記書込み要求によって示される前記書込みデータのサービスプロバイダへの送信が、所定の期間中断されるパススルーモードに入ることと、 前記ストレージゲートウェイが、前記パススルーモードから出ることと、 前記ストレージゲートウェイが、前記パススルーモードを出た後、前記一次データストアと一致する、前記一次データストアの前記スナップショットを作成するために、前記ローカルデータストア上に格納されたデータの少なくとも一部を前記サービスプロバイダにアップロードすることとをさらに含む、請求項1に記載の方法。
- 5前記書込みデータを前記サービスプロバイダに送信することが、前記読取り要求および前記書込み要求を前記ローカルデータストアに渡すこと、ならびに前記読取り要求および前記書込み要求に対する前記ローカルデータストアからの応答を前記1つまたは複数のプロセスに返すことと非同期に実行される、請求項1に記載の方法。
- 6前記ローカルデータストアが、1つまたは複数の記憶装置を含み、前記1つまたは複数の記憶装置のうちの少なくとも1つ上の少なくとも一部のデータを、前記リモートデータストア上の前記一次データストアの前記スナップショットから復元することをさらに含む、請求項1に記載の方法。
- 7前記書込み要求によって示される書込みデータを前記サービスプロバイダに送信することが、前記書込みデータを書込みログにバッファリングすることと、前記ストレージゲートウェイのアップロード構成要素によって、前記バッファリングされた書込みデータを、前記サービスプロバイダの対応するアップロード構成要素にアップロードすることとを含む、請求項1に記載の方法。
- 8少なくとも1つのプロセッサとプログラム命令を含むメモリを備え、 前記プログラム命令は、 顧客ネットワーク上の1つまたは複数のプロセスから、前記顧客ネットワーク上のローカルデータストアに向けられた読取り要求および書込み要求を受信することであって、ゲートウェイプロセスは、前記顧客ネットワーク上の前記1つまたは複数のプロセスから、ブロックストレージプロトコルを介して、前記読取り要求および前記書込み要求を受信することと、 前記読取り要求および前記書込み要求を前記ローカルデータストアに渡すことと、 サービスプロバイダによりリモートデータストア上で維持される前記ローカルデータストアのスナップショットを書込みデータで更新するために、前記書込み要求によって示される書込みデータを前記サービスプロバイダ送信することと を行うように動作可能なゲートウェイプロセスを実現するように、前記少なくとも1つのプロセッサによって実行可能である、 装置。
- 9ゲートウェイプロセスが、前記顧客ネットワーク上の前記ローカルデータストアと一致する、前記ローカルデータストアの前記スナップショットを作成するために、前記ローカルデータストア上に格納されたデータの少なくとも一部を前記サービスプロバイダにアップロードするように動作可能である、請求項8に記載の装置。
- 10前記ゲートウェイプロセスが、前記書込みデータを前記サービスプロバイダに送信することを、前記読取り要求および前記書込み要求を前記ローカルデータストアに渡すこと、ならびに前記読取り要求および前記書込み要求に対する前記ローカルデータストアからの応答を前記1つまたは複数のプロセスに返すことと非同期に実行するように動作可能である、請求項8に記載の装置。
- 11前記ゲートウェイプロセスが、前記ゲートウェイプロセスを前記サービスプロバイダに登録するプロセスを開始するようにさらに動作可能である、請求項8に記載の装置。
- 12前記ゲートウェイプロセスが、前記サービスプロバイダから受信した設定情報に従って、シャドーイングゲートウェイとして設定されるようにさらに動作可能であり、前記シャドーイングゲートウェイが前記ローカルデータストア上に格納されたデータを前記リモートデータストアにシャドーイングする、請求項11に記載の装置。
- 13前記ゲートウェイプロセスが、前記ローカルデータストア内の少なくとも一部のデータを、前記ローカルデータストアの前記スナップショットから復元するようにさらに動作可能である、請求項8に記載の装置。
- 14前記書込み要求によって示される書込みデータを前記サービスプロバイダに送信するために、前記ゲートウェイプロセスが、前記書込みデータを書込みログにバッファリングすることと、前記バッファリングされた書込みデータを前記書込みログから前記サービスプロバイダにアップロードすることとを行うようにさらに動作可能である、請求項8に記載の装置。
- 15前記ローカルデータストアの前記スナップショットが、前記リモートデータストア上で前記サービスプロバイダによりブロックフォーマットで維持され、かつ、前記書込み要求によって示される書込みデータを前記サービスプロバイダに送信するために、前記ゲートウェイプロセスが、前記書込みデータによって修正されたデータのブロックを前記サービスプロバイダにアップロードするようにさらに動作可能である、請求項8に記載の装置。
Independent claims15
217 paragraphs, as filed
0001Many companies and other organizations have many computing systems, for example compiling. Place the tuting system in the same location (for example, as part of a local network) And, or instead, in multiple different geographic locations (eg, one or more plies). Place them (connected via a bate or public intermediate network) and support their behavior. Operate an interconnected computer network to port. For example, a single Private data centers operated by or on behalf of your organization Operated by an entity as a business that provides computing resources to customers Quite a few interconnected computing, such as public data centers Data centers that house systems have become commonplace. Some public data centers Tar's operating entity is online for hardware owned by various customers. Provides work access, power, and secure installation facilities, while other public data centers Tar's operating entity also includes hardware resources available to those customers. Provide "full service" facilities. However, the size and scope of regular data centers has expanded As it grows, it provisions, operates, and manages physical computing resources. The tasks to manage are becoming more complex.
0002The advent of virtualization technology for commodity hardware has led to a large number of diverse needs. Benefits from managing large-scale computing resources for customers, various Computing resources can now be effectively and securely shared by multiple customers. For example, virtualization technology is one hosted by a single physical computing machine. Or a single physical compute by providing multiple virtual machines to each user You can allow a virtual machine to be shared among multiple users, but each such virtual machine is To the user that they are the only operators and administrators of a given hardware resource It functions as a separate logical computing system that gives the illusion of Software that also provides application isolation and security between various virtual machines This is an air simulation. In addition, some virtualization technologies have multiple separate physical computers. For example, a single virtual machine with multiple virtual processors across a putting system, etc. Can provide virtual resources that span two or more physical resources.
0003As another example, virtualization technology can be distributed across multiple data storage devices. By providing data stores to each user, data storage hardware can be used by multiple users. Each such virtualized data store can be shared between users, they Gives the illusion that is the only operator and administrator of that data storage resource , Act as a separate logical data store.
0004[Web service] The traditional web model uses HTTP client programs such as web browsers. Via web resources (eg applications, services, and And data). A technology called web services is a web litho Developed to provide programmatic access to the booth. Web service is Web connection such as web server system via web service interface Technical platforms hosted on computers (eg applications and And services) and data (eg product catalogs and other databases) Can be used to provide programmatic access to web resources. Generally speaking For example, the web service interface requires that some service be performed. Standard cross-platform for communication between clients and service providers Set to provide a form API (application programming interface) Can be determined. In some embodiments, the web service interface is a service request. Supports the exchange of documents or messages that contain information that describes the response to the request. Can be set to Such documents, or messages, may be, for example, hypertexture. Can be exchanged using standard web protocols such as Hypertext Transfer Protocol (HTTP). Also, platform-independent devices such as Extended Markup Language (XML) It can be formatted in data format.
0005<figref num="1">FIG. 3 is a high-level block diagram of an example networking environment, including an example service provider and an example service customer, according to at least some embodiments.</figref><figref num="2">An architectural example and its components for a storage gateway are shown according to at least some embodiments.</figref><figref num="3">It is a high-level block diagram of an example of a network environment in which an embodiment of a storage gateway can be realized.</figref><figref num="4">It is a block diagram of an example network environment including an onsite storage gateway in a service customer network that acts as an interface between a service customer network and a storage service on a service provider network, according to at least some embodiments. ..</figref><figref num="5">FIG. 5 is a block diagram of an example service provider that provides storage services and hardware virtualization services to service provider customers according to at least some embodiments.</figref><figref num="6">It is a high-level block diagram which roughly shows the architecture of an example network environment and the data flow in it, in which one embodiment of a storage gateway is set as a cache gateway.</figref><figref num="7">It is a high-level block diagram which roughly shows the architecture of an example of a network environment and the data flow in it, in which one embodiment of a storage gateway is set as a shadowing gateway.</figref><figref num="8">It is a high level block diagram which roughly shows the bootstrap of a shadowing gateway in an example of a network environment according to at least some embodiments.</figref><figref num="9">It is a flow diagram of a bootstrap process for a shadowing gateway according to at least some embodiments.</figref><figref num="10">It is a flow diagram of a shadowing gateway that enters and then recovers from pass-through mode according to at least some embodiments.</figref><figref num="11">It is a flow diagram of a method for uploading, updating, and tracking blocks from a gateway to a remote data store according to at least some embodiments.</figref><figref num="12">It is a flow diagram of an optimized bootstrap process for a shadowing gateway according to at least some embodiments.</figref><figref num="13">Aspects of the storage gateway security model according to at least some embodiments are shown.</figref><figref num="14">FIG. 5 is a flow diagram illustrating at least some aspects of the gateway security model during activation, configuration, and operation of the storage gateway according to at least some embodiments.</figref><figref num="15">FIG. 3 is a high-level block diagram of an example networking environment showing service customers and service provider components or entities involved in the gateway activation process, according to at least some embodiments.</figref><figref num="16A">It is a process flow diagram which shows the interaction between the components shown in FIG. 15 during the gateway activation process according to at least some embodiments.</figref><figref num="16B">It is a process flow diagram which shows the interaction between the components shown in FIG. 15 during the gateway activation process according to at least some embodiments.</figref><figref num="17A">It is a flow diagram of the activation process from the viewpoint of the storage gateway according to at least some embodiments.</figref><figref num="17B">It is a flow diagram of the activation process from the viewpoint of the storage gateway according to at least some embodiments.</figref><figref num="18">It is a high-level block diagram which shows the gateway control architecture example which can be adopted in at least some embodiments.</figref><figref num="19">It is a flow diagram of a method for remote gateway management using gateway start connection and long polling techniques according to at least some embodiments.</figref><figref num="20">It is a flow diagram of a method for a gateway control server to broadcast a gateway request to its peer server according to some embodiments.</figref><figref num="21">It is a flow chart of an alternative method for delivering a gateway request to an appropriate gateway control server according to some embodiments.</figref><figref num="22">It is a flow diagram of a method for establishing, monitoring, and maintaining a gateway-initiated connection according to at least some embodiments.</figref><figref num="23A">It is a block diagram that roughly shows the architecture for a service provider network, including a gateway proxy plane, according to at least some embodiments.</figref><figref num="23B">A gateway control server that sends a message to a gateway through a gateway proxy plane is shown according to at least some embodiments.</figref><figref num="23C">A gateway that responds to a gateway control server request through a gateway proxy plane is shown according to at least some embodiments.</figref><figref num="23D">Demonstrates a ping message exchange for a gateway proxy plane according to at least some embodiments.</figref><figref num="24">The overall architecture for the cache gateway and its data I / O behavior is shown according to at least some embodiments.</figref><figref num="25">The overall architecture for the shadowing gateway and its data I / O behavior is shown according to at least some embodiments.</figref><figref num="26">FIG. 5 is a flow diagram of a method for writing to a write log on a block data store according to at least some embodiments.</figref><figref num="27">It is a flow diagram of a method for satisfying a read request according to at least some embodiments of a cache gateway.</figref><figref num="28">Computer cis may be used in some embodiments is a block diagram showing an Temu example.</figref>
0006In the present specification, the embodiments are provided by examples and exemplary diagrams for some embodiments. As described, one of ordinary skill in the art is not limited to the embodiments or figures described. You will understand that. In addition, the figures and detailed description thereof disclose embodiments. Not intended to be restricted to the particular form in which it was made, and conversely, the intent is defined in the appended claims. Understood to include all modifications, equalities, and alternatives contained within God and scope Should be done. The headings used herein are for organizational purposes only and are described. Or is not intended to be used to limit the scope of the claims. Used throughout this application As you can see, the term "may" has a compulsory meaning (ie, must). Rather than (meaning must), it has a permitting meaning (ie, ~ (Meaning possible). Similarly, "include", " The terms "including" and "includes" are Means include, but not limited to.
0007Methods, devices, and methods for providing a local gateway for remote storage And various embodiments of computer-accessible storage media are described. Storage gate Way embodiments are described herein through an intermediate network such as the Internet. A service that provides storage services to one or more customers of a service provider Explained in the context of the provider. The storage gateway is the customer's data center Installed on-premise (inside the building) on-premises Acts as a gateway between the customer's data center and storage services, Can be implemented as a virtual or physical appliance. Storage gateway is streak As an interface to a primary storage device provided remotely via a page service And can be set as a local cache for it and / or storage One implemented on the customer's network to the remote storage provided by the service It can be configured as an interface to shadow the next storage device. Storage gate Way is the front end of the appliance and is the standard device for the customer's application. It presents a data access interface and its data address on the back end of the appliance. Convert access to storage service request and follow storage service interface , The data can be transmitted to the storage service over the network. At least yes In some embodiments, the storage service interface is a web service interface. Can be implemented as an ace.
0008The embodiment of the storage gateway is an on-premises interface, a storage server. For virtually unlimited, flexible, extensible remote storage provided through the service Can be provided. The storage gateway is a traditional on-premises storage solution Can provide a cost-effective, flexible, and easily scalable alternative. Storage costs are declining, but the tube of traditional on-premises storage solutions The cost of science, as well as other software and hardware, remains relatively constant, Or it is increasing in some cases. The embodiment of the storage gateway is a service pro. Bida customers can be able to reduce the total cost of owning storage, at least how much Pass the management and other costs to the service provider.
0009In at least some embodiments, the storage service is a block storage technology. According to, the customer's data can be stored in the remote data store. At least some In an embodiment, the storage gateway is a block storage protocol (eg, i). SCSI, GNBD (Global Network Block Device), etc.), File strike Rage protocol (eg NFS (Network File Storage), CIFS (both) Internet file system), etc.), and / or object storage Rotocol (for example, REST (Representational State Tran) sfer: expression state transfer)) published on the front end for the customer's application Can be. Block storage protocols such as iSCSI are the basis of remote data stores Allows direct access to the underlying data blocks.
0010File strikes such as NFS or CIFS exposed by the storage gateway Written to a remote data store by an application via the rage protocol Files can be stored in remote data stores according to block storage technology. Stretch through public file storage protocols such as NFS and CIFS The gateway is stored in a remote data store according to block storage technology Also, the customer's data, they are uploaded from the gateway through the customer's network. Present as a file to the customer's application before being transmitted to the application. Example For example, public block storage protocols such as iSCSI block customers' Transmit to an application, so that application interprets the data block Demands that it handle any format that the application expects.
0011Block storage protocols such as iSCSI are low-level block storage pros Tocol and therefore more than file storage protocols such as NFS and CIFS Can also enable a wide range of use cases. The block storage protocol is usually Mic rosoft® SharePoint® and Oracle® Support for applications that write to block stores, such as databases The underlying stray for CIFS or NFS file servers that can enable It can also be set to provide the Therefore, at least go of the storage gateway In some embodiments, a block storage protocol such as iSCSI is the customer's app. It can be adopted as an interface exposed to applications.
0012Figure 1 shows service provider examples and services according to at least some embodiments. It is a high-level block diagram of a networking environment example including a customer example. Storage game The Toway 84 climbs one or more of several remote data storage features. To serve customer processes (s) 88 on Ant Network 80 Service Customer Local Network or Data Center (eg Client Network) Install, enable, and as a virtual or physical appliance within the 80) Can be set. Customer process 88 resides on client network 80 and is gated Via the data protocol of the way 84 data port (eg iSCSI protocol) Any hardware or software that can connect to and communicate with the storage gateway 84 It can be air and / or a combination thereof. Storage gateway 84 is like For example, as an on-premises storage device and / or customer process (s) 88 The strike provided by the client network 80 above and the service provider 60 It can serve as an interface to and from Rage Service 64. Storage service 6 In addition to 4, service provider 60 includes hardware virtualization services, but to it It was noted that other services, not limited to, could also be offered to the customers of service provider 60. I.
0013Customers of Service Provider 60 are referred to herein as Service Customers or Simply Customers. It can be on the local network or for one or more users on the network, Including one or more services provided remotely by service provider 60 , To provide networked computing services, such as the internet Computer network or multiple networks coupled to intermediate network 50 It can be any entity that implements Service customers include businesses, educational institutions, government agencies, Or a computer that provides users with networked computing services It can be any general entity that implements a network or multiple networks. Figure 1 indicates a single client network 80, but multiple client nets There can be work 80. Each client network 80 serves different service customers It is possible, or two or more client networks 80, for example, different branches of an enterprise. Different data centers for the same service customer, such as campuses with different companies or school organizations Or it can correspond to the area. Service provider 6 in at least some embodiments Each customer of 0 may have an account with service provider 60 and security credentials ( For example, an account name and / or identifier, password, etc.) may be provided, but Representing one or more customers (eg, client network administrator) using Includes but is limited to storage services provided by Service Provider 60 Not to manage customer resources provided by one or more services , Can log in to the interface to service provider 60 (eg web page) To.
0014An embodiment of the storage gateway 84 is hardware, software, or it. Can be implemented in combination of these. In at least some embodiments, storage gateways B 84 can be executed, for example, in a virtual machine instantiated on the host system. Can be implemented as a virtual appliance. In at least some embodiments, strike Rage Gateway 84 is a service customer data center (eg, a client network). In network80), a sir coupled to the local network infrastructure Downloaded on one or more computing devices, such as a bar system A virtual appla that can be installed, activated, and configured in any other way. Can be implemented as an ounce. As an alternative, the storage gateway 84 provides service assistance. In the customer's data center (eg client network 80), the local network Implemented as a dedicated device or appliance that can be coupled to the network infrastructure The dedicated device or appliance can perform the functions of the storage gateway 84. It may include software and / or hardware to represent. Figure 26 shows the storage game An example of a computer system in which an embodiment of Tway 84 can be implemented is shown. Less Also in some embodiments, the storage gateway 84 is a firewall 82 technique. Service provider through intermediate network 50 (eg internet) 60 Communicate with the network. Service Provider 60 Network is an intermediate network Front-end 6 through which network traffic to and from K50 passes Two technologies (for example, firewall technology, border router technology, load balancer technology, etc.) Please note that it is included.
0015At least some embodiments of the storage gateway 84 are data to the customer. Protection, as well as misuse and misuse of Gateway 84 by customers or third parties (eg) Can be implemented according to a security model that provides protection against illegal copying. Strike Communication between Rage Gateway 84 and Storage Service 64 is protected and encrypted obtain. The activation process will be explained later in this document, but it will be newly installed in it. Storage gateway 84 services to obtain security credentials Initiates a connection to the Provider 60 network and is identified against it. At least yes In some embodiments, during the activation process, the customer is at the service provider 60 of the customer. Log in to your account and provide the information used to register Gateway 84 as a service provider. Provide to Ida 60. However, the customer did not log in to Storage Gateway 84 and obeyed Customer security credentials and other account information are publicly available on Gateway 84. Not opened. This can minimize the security risk to the customer.
0016In at least some embodiments, the aspect of the security model is gateway 84. Is public to the customer process (s) 88 on the client network 80 External start connection to one or more open data ports (eg iSCSI port) To accept only. Storage gateways are all other to external processes Initiate a connection; no external process can initiate any other connection to the gateway. For example, in at least some embodiments, the storage gateway 84 is a gateway. Initiate other connections to Hay Management and Service Provider 60; Service Provider 60 Does not initiate a connection to gateway 84. As another example, client network 8 Network administrator process 90 of 0 to configure and manage gateway 84 , Cannot connect directly to storage gateway 84. Instead, Network Administrator Pro The configuration and management of Storage Gateway 84 by Seth 90 is, for example, Service Pro. Service provider 60 via console process 68 on the bidder 60 network Can be performed through. Therefore, in at least some embodiments, the client network Users, network managers, or processes on network 80 (eg, net) Work manager process 90 or customer process (s) 88) Cannot "login" directly to the gateway 84, on the service provider 60 network (For example, console process 68 and storage service 64) or something else Users, managers, or processes on your external network are also storage gateways It is also not possible to initiate a connection to Way 84. This is on storage gateway 84 If security credentials and other behavioral information are available on the client network 80 By process, or by an outsider or process, intentionally or intentionally Helps prevent you from being endangered.
0017The storage gateway 84 embodiment is one of several data store 66 features. Installed, activated, and storage service to provide one or more Can be configured for use with-bis 64. For example, the storage gateway 84 is Installed, enabled, configured, and streak to function as below Can be adopted in the service 64: -File system gateway. In this setting, the storage gateway is stray NAS storage interface for the service 64 (eg CIFS or NF) Acts as (using the S protocol). Remote data store 66 is a gateway 84 can be presented to the customer as an object store (eg REST) On the other hand, the data store 66 is implemented according to block storage technology. With this setting Remote data store 66 allows customers to write files to it, And as a virtualized file system where customers can read files from it , Can be presented to the customer. -Cloud volume gateway. In this setting, the storage gateway 84 is Volumes implemented on remote datastore 66 via Trage Service 64 ( Acts as an interface to singular or plural). Remote data store 66 Can be implemented using lock storage technology. Gateway 84 is flexible and inherently controllable Remote data that acts as a back-end storage device that provides unlimited primary storage capacity With volumes (s) (also called cloud volumes) on store 66 Provides a local network access point. With this setting, remote data Store 66 allows customers to localize volumes for reading and writing data from there. Can be presented to customers as a cloud volume system that can be mounted on. Shadowing gateway. In this setting, the storage gateway 84 is the customer's Shadow of write data (eg iSCSI write) to remote data store 66 Customer's application (eg, to provide hosting through storage service 84) For example, between the customer process (s) 88) and the customer's local data store 86) Functions as "bump in the wire". Remote data store 66 Can be implemented using block storage technology. With this setting, the storage gateway B 84 snaps the customer's local data store onto the remote data store 66 Acts as a shadowing appliance that shadows (s) obtain. This shadowing is transparent from the user's point of view on the local network Can be done If necessary or desirable, the customer may, for example, one of the customer's data. Restore part or all from snapshot (s) to local store 86 Snack customer data on remote data store 66 to recover, recover, or copy You can request or access a shot (s).
0018File system gateway and cloud volume gateway are both re-released Acts as a gateway to the mote data store, and both serve locally For example, frequently and / or recently used data can be cached. Note that they are similar in that. File system gateway and cloud volume In both gateways, reading data from the customer process is raw, if possible. It can be serviced from the cal cache, otherwise from a remote data store. versus In contrast, in shadowing gateways, data reading is done through the gateway. Passed to the customer's local data store. For the purposes of this document, File System Gate Ay and Cloud Volume Gateways shadow these embodiments. To distinguish it from Tway, it can be collectively referred to as a cache gateway.
0019[Example of storage gateway appliance architecture] Figure 2 shows a for storage gateways according to at least some embodiments. -Shows examples of textures and their components. Some of the components shown in Figure 2 are In the shadowing gateway embodiment when compared to the shrug gateway embodiment Note that it may not be used, or it may be used, or it may be implemented differently. I want to.
0020Block driver 10 interacts with customer process 88 on storage gateway 84. It becomes a face. Generally, the block driver 10 is read by customer process 88 (eg, read). Allows communication with the storage gateway 84 (via a write / write request). Su The storage gateway 84 is onsite for customer process 88, so process 88 From this point of view, the data seems to be stored locally. But storage Toway 84 is a remote data store 6 provided by storage service 64. Interface with storage service 64 to store data in 6. Cat For Shugateway, the primary data store is remote data store 66, but often Frequently accessed data can be cached locally by gateway 84 .. Reads can be satisfied from the local cache or from virtual data storage 66; write Only suitable for data blocks in local cache and / or virtual data storage 66 It is processed to be updated. Primary data strike for shadowing gateways A is local data store 86; reads are passed to local data store 86, Also, writes are not only sent to the local data store 86, but also virtual data storage 6 Shadowed to 6.
0021Block driver 10 captures read / write requests from customer process 88 and is key to it. Pass the request to the storage controller 12. In at least some embodiments, blocks Driver 10 is a block storage protocol (eg iSCSI or GMBD) As an interface to customer process 88. In some embodiments, the bro Block as an alternative to, or as an alternative to, the storage protocol interface Driver 10 has a file storage protocol interface (eg NFS or CIFS) can provide file system semantics for storage controllers 12 Can be used as an interface to. Figure 2 shows one block driver 10. However, keep in mind that there can be more than one block driver.
0022The storage controller 12 includes the block driver 10 and the cache manager 14. Acts as an intermediary with storage via. Storage controller 12 responsibility Sends read and write requests from block driver 10 to storage, as well as storage Including forwarding the callback to block driver 10 when the page responds with data obtain. Block driver 10 can also hold statistics such as the number of requests in progress.
0023In at least some embodiments, the stray on one storage gateway 84 The controller 12 is a cache manager 14 on another storage gateway 84. Can communicate with. In at least some embodiments, each storage gateway 84 A heartbeat message can be sent for discovery and failure detection. Given object Consistent hash to identify storage gateway 84 responsible for The storage gateway that can be used and the request to retrieve the data is targeted. Can be transferred to cache manager 14 on 84. Cache manager 14 strikes Respond by invoking the callback provided by Rage Controller 12 obtain.
0024In the cache gateway embodiment, the cache manager 14 is, for example, frequently Manages local cache 28, which provides storage for accessed data obtain. The local cache 28 is volatile inside the storage gateway 84 and / Alternatively, it can be implemented on non-volatile memory or, as an alternative, provided by the customer. Can be implemented at least partially on an external local data store 86. At least go In some embodiments, the local cache 28 is stored in the virtualized data storage 66. Represents data; writes from customer process 88 directly affect local cache 28 It may not affect you.
0025In at least some embodiments that employ multiple gateways 84, distributed b. -Cal cache can be used to identify the responsible cache that holds a given key Therefore, a consistent hash method may be used for the key. At least some In the embodiment, locality-aware reques t distribution) can be used to reduce communication between gateways 84 However, it may require additional load balancing.
0026All write requests to a given volume in remote data store 66 are for a particular game Can be turned to Tway 84 nodes. All write requests to the volume are specific gates Network partitioning may not be an issue as it will be forwarded to Way 84 node.
0027[Staging] In at least some embodiments, the cache manager 14 is a staging 16 It can contain components or can interface with staging 16 components. Staging 16 may include write log 18 or access to write log 18. Can have. In at least some embodiments, the data structure goes through write log 18. It can be built and used as a metadata store 26. Metadata store 26 is specific It may allow quick access to all writes to the block. Metadata store 2 6 applies mutations, for example, to different segments within a block Can be used to When write data is received from customer process 88, that data Is added to the write log 18. For example, offset and length related to blocks The metadata for the written data can be stored in the metadata store 26. At least Also in some embodiments, the write log 18 is either a linear or circular queue. It can be implemented as an implemented one-dimensional data buffer. In at least some embodiments Metadata store 26 is implemented as, for example, Berkeley Database Can be a key / value store. Both write log 18 and metadata store 26 Other embodiments may be used in some embodiments.
0028In a cache gateway embodiment, when a read is performed, the original block is localized. Can be retrieved from LeCash 28 or from remote datastore 66, write log 1 Before any pending changes indicated by 8 return the data to their respective customer process 88 Can be applied to.
0029In some embodiments, gateway 84 fails (eg crashes) And write in memory unless the data has already been written to local data store 86 Only data can be lost. In some embodiments, multiple gateways at the customer site 84 If there is another gateway 84 owned by the crashed gateway 84 Responsible for the key being a snapshot on the local data store 86 (if any) ) Can restore writes and start accepting requests directed to each volume .. In some embodiments, the write log 18 and / or the metadata store 26 is joke. Over two or more gateways 84, to provide durability and better durability Can be duplicated. In case of gateway 84 failure, one of the other gateways 84 Can take over the write log 18 and metadata store 26 of the failed gateway .. However, in at least some embodiments, the metadata store 26 is the owner's game. Can only be held on Tway 84. In these embodiments, the gateway 84 fails. If one of the other gateways 84 is to rebuild the metadata store 26 , The primary write log 18 can be taken over and analyzed.
0030In the cache gateway embodiment, the block fetcher 22 is requested to block. Fet the segment from remote data store 66 via storage service 64 Chi In at least some embodiments, the block fetcher 22 is caching. Because of lazy fetchin to fetch the complete block g) Technology can be adopted. Both cache gateway and shadowing gateway For those, block store 24 stores data from staging 16 Push-type distribution to remote data store 66 via screen 64. At least some In the embodiment, the block store 24 is delayed to deliver the block in a push-type manner. Shu technology can be adopted.
0031In at least some embodiments, during a read operation on the cache gateway , Block driver 10 needs to read, including volume ID, start offset and length The request is sent to the storage controller 12. In at least some embodiments The storage controller 12 converts the volume ID and offset into object keys. Can be replaced. Storage controller 12 caches read request information Can be passed to 4, but it will satisfy its read request from the appropriate local cache 28 You can try. If that data does not exist in the local cache 28, the request will block Transferred to fetcher 22, which transfers its data to remote data store 66 Fetch from the appropriate volume via storage service 64. Data is retrieved Then the local cache 28 is updated, the changes from write log 18 are applied, and read The response is returned to customer process 88. In at least some embodiments, a plurality of If locks are required, multiple, each showing a relative offset to each block A number of read responses may be returned. Sequential reads are detected in at least some embodiments Then, consecutive blocks can be fetched in advance.
0032In at least some embodiments, during the write operation, the block driver 10 is Responsible for a write request containing the user ID and write data for that volume Send to storage controller 12. The write data is written to the write log 18 and Metadata store 26 contains a reference to the modified data in buffer pool 20 Will be updated.
0033[Buffer pool] In at least some embodiments, the buffer pool 20 is a storage controller. It exists between 12 and the local data store 86. The buffer pool 20 has the following task One or more of these can be performed, but not limited to them. Some tasks are cached Note that it can only be applied to gateways: · Local data storage for write log 18 and local cache 28 (single) Or cache of data for logical offsets from their physical position on (or more). -Maintaining locks on the buffer during read and write operations. -For example, minimum frequency of use (LRU) based on eviction technology, etc. , Application of eviction technology to physical storage for local cache 28. this is, Note that it is not required for shadowing gateways. -For reads in each cache gateway, the requested data is locally cached. If not found in Shu 28, buffer pool 20 is booted from remote data store 66. It may communicate with the block fetcher 22 to fetch the lock. As an alternative, go In some embodiments, the block fetcher 22 strikes to fetch a block. Can communicate directly with Rage Service 64.
0034In at least some embodiments, the buffer pool 20 is a database (eg, for example. Berkeley database (BDB)) could be adopted as its metadata store 26 To. Table 1 below shows the metadata store 26 according to at least some embodiments. Indicates information that can be stored within. Restrict entries in Table 1 according to content or placement Note that it is not intended to be.
0035<tables num="1"><img id="000002" he="70" wi="159" file="JP6139718B2_D0001.tif" img-format="tif" img-content="drawing" /></tables>
0036In at least some embodiments, the physical disk offset is, for example, a 4MB boundary. At a fixed boundary, such as in. In at least some embodiments, this It contains boundaries for both data in the volume and in write log 18. Less Also in some embodiments, the write to a particular volume can be a sequential write. Therefore, fragmentation on disk does not need to be considered. "Chunk" is Note that it can correspond to, blocks, or one or more blocks.
0037Metadata store 26 has both S (snapshot) and C (chunk) entries These can include the storage controller 12 accessing the block through it. It needs to be kept up to date with the method it tries. For example, a block is initially , Can be referenced using the snapshot ID, but then always the chunk ID use. It can be stored in metadata store 26. The snapshot is complete And the storage controller 12 takes a snapshot using the snapshot ID You can refer to the block from; therefore, the C (chunk) entry in the metadata store 26 , Can be converted to the corresponding S (snapshot) entry.
0038[Operation of cache gateway] In at least some embodiments, when a read request is received, a write to the block Included log 18 entries or multiple entries are searched in metadata store 26. Reading If the request is satisfied with 18 or more write log entries All requested entries are searched in the metadata store 26 and read into the buffer Is flattened and the requested fragment is returned. Read request is write log 18 If you are not satisfied with using only a bird or multiple entries, then a cached data block The offset for (eg 4MB block) is calculated from the offset in the read request Is done. The location of the block is searched in the metadata store 26. The block is local If it is in the wash 28, the block is read from the local cache 28 and also , Otherwise it is fetched from remote data store 66. Requested write Log 18 entries are fetched, blocked in blocks, and requested as described above. Fragment is returned. If the block is fetched from remote data store 66, then Blocks are cached in the local cache 28 and stored in the metadata store 26 It will be recorded. The last access time to the block in local cache 28 is also updated To.
0039In at least some embodiments, when a write request is received, the next write log 18 Changes are recorded at offset, and the metadata, ie, offset and length, is metadata. Recorded in data store 26.
0040In at least some embodiments, once the block upload is complete, the block The latest version of the cache has been added to the local cache 28 (with any changes applied) , Recorded in metadata store 26. The previous version of the block is local If present in Shu 28, this block is marked as empty in Metadata Store 26. Be kicked.
0041In at least some embodiments, once the snapshot is complete, the metadata list A26 may need to be reorganized as described above. That is, its snapshot Block entries that belong to the corresponding snapshot on remote data store 66 Can be converted into an entry.
0042[Operation of shadowing gateway] In at least some embodiments, the read request is passed to the local data store 86. To.
0043In at least some embodiments, when a write request is received, the write data is next: Write log 18 offset, and the appropriate metadata for that write is metadata Recorded in Tastore 26. Write requests are also passed to the local data store 86.
0044In at least some embodiments, upload the block to remote data store 66 The upload process buffers to read write log 18 Call Le 20. Buffer pool 20 is physically off from logical write log 18 offset The metadata store 26 is used to perform the conversion to the set, and the data is then noted Read into the rebuffer. The buffer is then presented to the upload process. A The upload process uploads the block to the remote data store 66 and its broadcast Release the lock to buffer pool 20.
0045[Purge write log] In at least some embodiments, if write log 18 needs to be purged The buffer pool 20 is a write log for a volume that can be purged by write log 18. Get a fuset. In at least some embodiments, the write log offset is an example. For example, a database walkover that checks the offset for each entry ( It can be determined from the metadata store 26 by executing walk over). To purge write log 18, existing writes corresponding to the purgeable part of that log Only log entries can be marked as empty entries.
0046[Example of Embodiment] Figure 3 shows a high example of a network environment in which a storage gateway embodiment can be implemented. It is a level block diagram. Sir on intermediate network 100 (eg internet) The bis provider 110 may be one or more similarly coupled to the intermediate network 100. Service Customer Network (for example, Client Network (s) 1 50) Access to remote data store 116 via storage service 112 Provide Each client network 150 can serve different service customers Or two or more client networks 150, for example, different branch offices of a company If different data centers for the same service customer, such as campuses with different school organizations It can correspond to the area. Service customers can be businesses, educational institutions, government agencies, private organizations, or Providing networked computing services to one or more users A computer net that is connected to an intermediate network 100 such as the Internet. It can be any general entity that implements a work or multiple networks. How many In that embodiment, the storage service 112 is an interface (eg, a web server). Bis interface) can be provided, through which each service customer's client net Work (s) 150, features provided by storage service 112 Can be accessed.
0047Customer processes 154A and 154B are service customer client networks 1 Represents a physical and / or virtual machine or system connected to 50. Storage support As an example of the functionality provided by Service 112, the user may use customer process 154. Data collection in remote data store 116 through storage service 112 You can create and mount videos. View of users on client network 150 From a point of view, the data volume provided by the storage service 112 is that They can appear to be local storage; therefore, such data volumes It can be called virtual data volume 158. The virtual data volume 158 is remote One or more physical storage devices on which the datastore 116 is instantiated Or it actually maps to a storage system; this mapping is a storage service Processed by 112 and therefore the user's perspective on the client network 150 Seen from the perspective, it is transparent. The user of customer process 154 is on the desktop or on the device You can just look at the volumes mounted in the list. Customer process 154 The user is as if Volume 158 is a locally installed storage device. On the virtual data volume 158, create data, modify data, delete data, And can perform general data related functions.
0048Figure 4 shows the client network 250 and according to at least some embodiments. A service customer's club that acts as an interface to and from the storage service 212 Network including onsite storage gateway 252 on Iant Network 250 It is a block diagram of a network environment example. In at least some embodiments, storage Toway 252 is installed onsite in the service customer's data center It can be a file and / or block storage appliance.
0049The storage gateway 252 is, for example, as a file system gateway. Install, enable, and configure to act as a loud volume gateway Can be determined, total as a cache gateway or as a shadowing gateway Is called. The file system gateway (for example, CIFS or NFS protocol) As a NAS storage interface for storage service 212 (using Works. Remote data store 216 is actually implemented as block storage However, it can be presented to the customer as an object store (eg REST). Cloud Volume Gateway is a virtual provided by Storage Service 212 Acts as an interface to the volume storage gateway. Volume Storage can be implemented as block storage. Gateway 252 is flexible It functions as a back-end storage device that provides an essentially unlimited primary storage capacity. Local net with mote data store 216 (also known as cloud volume) Provide a work access point. Shadowing gateway is the customer's write day Remote provided by storage service 212 for data (eg iSCSI write) Customer's application and customer's to provide shadowing to storage Acts as a "bump in the wire" to and from the local data store. The remote data store 216 can be implemented as block storage.
0050In the cache gateway embodiment, the storage gateway 252 secures the data. Frequently accessed while fully encrypted to facilitate return to service provider 210 A local cache of the data to be stored can be stored on the local data store 254. Similarly , Shadowing gateway embodiment securely encrypts write data and services The move to provider 210 can be accelerated. This compared to a standard internet connection Accelerated data movements include, for example, data deduplication, compression, parallelization, and TCP wins. It can be achieved using one or more of the dow scaling techniques. Storage game The Tway 252 is typically on-site as a primary or backup storage device. The cost, utilization, maintenance, and provisioning headaches associated with managing storage arrays Species can be significantly reduced. Storage gateway 252 is a customer, otherwise the company Can be stored within expensive hardware (eg NAS or SAN hardware) Place hundreds of terabytes to petabytes of data in a cost-effective appliance This can be achieved by changing. With storage gateway 252, customers Durable, available and scalable, provided by Service Provider 210 Onsite storage (cassi) while leveraging a distributed storage infrastructure Local cache maintained by gateway 252 in the gateway embodiment Can benefit from low access latency (provided by Shu).
0051The embodiment of the storage gateway 252 is a customer's onsite application and system. Can cooperate with murless. In at least some embodiments, the customer is a SAN (iSCS). I), NAS (NFS, Microsoft® CIFS), or objects Configure storage gateway 252 to support REST storage Can be. Provided by Storage Gateway 252 in at least some embodiments The provided iSCSI interface is Microsoft® ShareP Onsite products such as oint® and Oracle® databases May enable integration with lock storage applications. At least some implementation In form, the customer is Windows®, Linux®, and UN. File Stray across environments, including but not limited to IX® environments NFS and C provided by Storage Gateway 252 to integrate IFS interface can be used. In at least some embodiments, storage The Toway 252 can also be configured to support REST-based requirements.
0052In at least some embodiments, the storage gateway 252 is the customer's data. Servers coupled to the Client Network 250 infrastructure at the center -Downloaded onto one or more computing devices, such as a system A virtual device or device that can be installed, activated, and configured in one or other ways. Can be implemented as a priority. Alternatively, the storage gateway 252 is a cryo Dedicated equipment or apply that can be coupled to the Ant Network 250 infrastructure Can be implemented as an anchor; its dedicated device or appliance has the function of a gateway It may include software and / or hardware that can be implemented on it.
0053In at least some embodiments, the storage gateway 252 is an intermediate network. Communicate with service provider 210 via the Internet (for example, the Internet) To. Binding of storage gateway 252 to intermediate network 200 is a large amount of data However, there is an intermediate network between the storage service 212 and the storage gateway 252. Generally, the client network of service customers as it can be communicated through the service customer. May be via the high bandwidth connection provided by 250. For example, during peak hours, the connection is 1 Must support data transfer above 00 Mbit / s (100 Mbit / s) obtain. However, in at least some embodiments, techniques such as data deduplication strike When uploading data from Rage Gateway 252 to Storage Service 212 Can be adopted to reduce bandwidth utilization, so more customers' bandwidth can be used for other purposes May be available on the way. Data weight that can be adopted in at least some embodiments An example of multiple exclusion technology is "RECEIVER-SIDE DATA DEDUPLICATI. ON US Patent Application No. 12 / 981,3 entitled "IN DATA SYSTEMS" No. 93 and "REDUCED BANDWIDTH DATA UPLOADIN US Patent Application No. 12 / 981,39 entitled "G IN DATA SYSTEMS" It is described in No. 7.
0054In at least some embodiments, the client network through the intermediate network 200 The bandwidth on the connection between the network 250 and the service provider 210 is the storage Network on Internet 252 and, for example, on Client Network 250 Can be assigned to other customer applications via the admin process 260. Strike Rage Gateway 252 has changed (up to new) according to, for example, data deduplication technology. Data (or changed) to storage service 212 continuously or almost continuously Can be uploaded. However, the rate of change of data on the client network 250 is Can change over time; for example, during the day, customer process write throughput increases However, at night, write throughput can be low. Therefore, when busy with a high rate of change Is high enough to keep up with the bandwidth allocated to storage gateway 252 If not, you may be late for uploading changed data; Storage Gateway 252 After that, you can catch up when the rate of change is not so high and you are not very busy. At least Also in some embodiments, the storage gateway 252 lags above a specified threshold. If so, storage gateway 252 may request additional bandwidth allocation. Less Also in some embodiments, the storage gateway 252 is more if necessary. An alarm can be issued to request more bandwidth.
0055Figure 4 shows the direct connection between the storage gateway 252 and the storage service 212. Is shown, but the connection between the storage gateway 252 and the storage service 212 Note that the continuation can go through the local network 256.
0056In at least some embodiments of storage gateway 252, on demand Large blocks of data rather than retrieving data from mote data store 216 Or chunks, even entire volumes of data localized to local data store 254 Can be cached. The storage gateway 252 is often used for data, for example, A local cache of data to be accessed or important data can be held on it. Can include physical data storage and / or memory (local data store 254) Or you can access it. Local data store 254 is volatile or non-volatile It can be a primary storage device or memory, or a combination thereof. Frequently accessed Keeping a local cache of your data can help many or most of your data Rather than getting data from remote data store 216, Seth has a local cache Data access time to customer process 258 is generally modified as it can be serviced from Can be good. However, the remote data store 216 is a client net for service customers. Can act as a primary data store for work 250; therefore storage gateway B. 252 regularly, irregularly, or continuously, b. -To upload from Cal Cache to remote data store 216, and required To download the requested data from the remote data store 216, depending on , Can communicate with the storage service 212 over the intermediate network 200.
0057In Figure 4, the remote data store 216 storage devices (218A, 218B, 218C, ...) means that the remote data store 216 is the local network of the service provider 210. On or to some storage devices or systems connected to network 214 Show that it can be implemented across. Therefore, the data of the service customer is the "back end". Can be interspersed with the above two or more physical storage devices or systems. Back-end storage device, etc. It can be a multi-tenant device shared with customers, but it is not always necessary. However , Users and processes on the client network 250, mentioned in connection with Figure 3. From the perspective of, client data is presented as a virtual volume or file Can be done.
0058In at least some embodiments, as described in connection with FIGS. 3 and 4, Service providers also offer hardware virtualization technology and possibly other virtualization technologies to their customers Can be provided. Service provider 200 has a block storage function (ie, block) Various virtuals, including block storage technology that provides customers with a Kubase storage system) It can provide computing technology and virtualized storage technology. Service provider A virtual computer implemented according to the hardware virtualization technology provided by Da 200. The arting environment or system may be supported by block storage technology. Block storage technology, for example, supports block-level storage capabilities. The structural and functional details of the volume to be loaded, and it provides storage availability Operating on a virtual computing system (or other system) Virtual computing through standard storage calls, making it independent of the system It may provide a virtualized storage system that can interact with the system.
0059The storage gateway 252 embodiment is for onsite customer applications. Virtualized Computing and Strikes Provided by Service Provider 200 Resilient "cloud-based" computing and that can be integrated with Rage technology Provide customers with access to storage resources. For example, for SAN storage Customers using the Age Gateway 252 have a matching, point-in-time block of their data. You can take a snapshot of the base. These snapshots are then Bro Requires high I / O and low latency data access provided by dock-based storage systems Hardware virtualization technology application or instance (eg, in Figure 5) Can be processed by virtual computing system 264). As another example The customer has a storage gateway 252 for NAS storage, NFS or CI Can be configured via the FS file protocol and also a hardware virtualization technology instance You can take a point-in-time snapshot of their file data accessible from.
0060In some embodiments, REST provided by storage gateway 252 Objects written using the base interface are sent to the service provider Directly from more offered virtualized storage technology via HTTP or other protocols Integrated content delivery that can be accessed or provided by a service provider Can be delivered using. In some embodiments, the customer is a hardware virtualization technology a. By virtual storage technology for parallel processing of these objects on the stance The highly scalable, distributed infrastructure provided by the company is also available.
0061Figure 5 shows storage services and hardware according to at least some embodiments. A service provider that provides service virtualization services to service provider customers It is a block diagram of an example. The service customer client network 250 is, for example, a diagram. Client network 250 and service providers as described in connection with 4. One or one that acts as an interface to the 210 storage services 212 It may include multiple storage gateways 252. Service client (single or multiple Number) may have access to one of the services provided by service provider 210 , Any administrator, user, or process.
0062Hardware virtualization technology hosts multiple operating systems Computer 2 Can be run simultaneously on 92, ie as virtual machine (VM) 296 on host 292 It can be done. The VM 296 is, for example, rented or re-rented to a customer of service provider 210. Can be Hypervisor on host 292, ie Virtual Machine Monitor (V) MM) 294 presents a virtual platform to VM 296 on host 292, V Monitor the execution of M 296. Each VM 296 provides one or more IP addresses Can be; VMM 294 on host 292 gives the IP address of VM 296 on the host Can be recognized. The local network of service provider 210 sends the packet to the VM Internet destinations from 296 (eg service on client network 250) To the client (s) 262), and to the Internet source (eg, for example) Route from service client (s) 262) to VM 296 Can be set to
0063Service provider 210 is an intermediate network over local network 256 Service customer client network 250 combined to 200, intermediate network Hardwork coupled to the local network of K200 and Service Provider 210 Ability to implement virtual computing system 264 through the carrier virtualization service 290 Can provide power. In some embodiments, the hardware virtualization service 290 is in. A surface, eg, a web service interface, can be provided, via which a service Bisclient 262 to the functionality provided by hardware virtualization service 290 Can be accessed. In service provider 210, each virtual computing system 2 64 is offered to service customers by leasing, renting, or otherwise, Host 29 2 Can represent a virtual machine (VM) 296 on the system.
0064From an instance of virtual computing system 264, the user previously described As you can access the functions of the storage service 212. Therefore, it is shown in Fig. 5. As such, in the embodiment of the virtualization system, the client depends on the service provider 210. Virtual computing system 264 implemented on VM 296 Generate a cal instance to install the virtual computing system 264 locally Remote data store 216 implemented by service provider 210 from chest of drawers It may be possible to access data from and store data there.
0065As mentioned above, one or more storage gateways 252 are client networks. Can be instantiated on network 250. At least one of gateways 252 At least some data, such as frequently accessed or important data It can be a cache gateway embodiment that caches locally. Storage game The way (s) 252 is, for example, a primary store of data (remote data). New or modified so that store 216) is retained in the cache gateway embodiment To upload corrected data from local cache, or new or modified The corrected data (write data) can be remoted within the shadowing gateway embodiment. Upload to local primary datastore snapshot on datastore 216 Storage service 21 through one or more high bandwidth communication channels Can communicate with 2.
0066[Cash gateway embodiment] In FIG. 6, one embodiment of the storage gateway is a file system gateway. Or as a cloud volume gateway, a network environment example -A high-level block diagram that roughly shows the texture and the data flow in it. These gateways may be collectively referred to as cache gateways. At least go In some embodiments, the storage gateway 252 is the service customer's data center. Files and / or block storage apps installed onsite Can be a ally. In Figure 6, the storage gateway 252 is, for example, a file. Acts as a system gateway or as a cloud volume gateway As such, it can be installed, activated, and configured. File system gateway , For storage service 212 (eg using CIFS or NFS protocol) And function as a NAS storage interface. Remote data store 216 , Implemented as block storage, but for customers object stores (eg, Can be presented as REST). Cloud Volume Gateway is a storage service As an interface to virtualized volume storage provided by S212 Function. Virtualized volume storage can be implemented as block storage. Ge The Tway 252 is a back end that provides flexible, essentially unlimited primary storage capacity. Remote data store 216 (also called cloud volume) that acts as a storage device Provide a local network access point with.
0067Once Storage Gateway 252 is installed, enabled, and configured, Network administrator process 260 of client network 250 is remote day On the Tastore 216, through the storage service 212, for example, a new data volume You can create a 270 or mount an existing data volume 270. Bo Ryumu creation request and other service request are service provider front end 280 Can be created for service 212 via. Front end 280 is storage It can also manage the connection to and with the gateway 252. Front end 280 , Firewalls, perimeter routers, load balancers, gateway servers, gateways Clients for iProxies, console processes, and storage services 212 Publish to network (s) 250 and strike its storage service 212 Required to interface with Rage Gateway (s) 252 One of any common networking equipment and / or processes that can be Or it may include, but is not limited to.
0068In at least some embodiments, the storage gateway 252 is a service pro. Initiate all connections to Service Provider 210 via Bider Front End 280 However; service provider 210 does not initiate a connection to gateway 252. further, Network administrator process 260 does not initiate a direct connection to gateway 252; eg For example, network administrator process 26 to configure and manage gateway 252 Access to gateway 252 by 0 is service provider front end 280 Pass through service provider 210 via.
0069The storage gateway 252 has one or more data ports (eg, iSCS). I port), customer process (s) on client network 250 2 Published on 58. Customer process 258 resides on client network 250 and Data protocol for gateway 252 data port (eg iSCSI protocol) Any hardware or software that communicates with the storage gateway 252 via) It can be air and / or a combination thereof. Customer process 258, for example, Mi crosoft® SharePoint® and Oracle ( Storage applications such as databases (registered trademarks), servers (eg SQL) Server, Microsoft® Exchange® server Database applications (eg SQL database applications, etc.) And Oracle® Database Applications), Microsoft (Registered Trademark) Exchange® application or storage device Clients that work to communicate with the 252 data ports (s) Up to any other application running on one or more devices on the network 250 Or it can be a process. The customer process is referred to herein as Client Network 2 Any software pro that may be running on one or more devices in 50 Including Seth; the underlying hardware on which a process runs is that of a process. Instead, connect to the storage gateway 252 data port (s) And note that they can be involved in or carry out communications.
0070Mounted volume 270 is customer pro by storage gateway 252 Can be presented to Seth (s) 258. Customer process (s) 258 Then, for example, according to the iSCSI protocol, to the storage gateway 252 So read and volume from volume 270 through the exposed data port Can write to 270. Storage gateway 252 is volume 270 Handles all read and write requests for. Bo on remote data store 216 Ryumu 270 acts as a primary data store, but storage gateway 252 also , Local cache of frequently accessed data on local data store 254 Can be stored. The local data store 254 is inside the storage gateway 252. Storage gateway provided by service customers on storage hardware 25 It can be implemented on two external storage hardware, or a combination thereof.
0071For reads, the storage gateway 252 first receives a given read as a local key. You can check your local cache to see if you're happy with the hash To. Storage gateway 252 if read is unsatisfactory from local cache Can request data from storage service 212, but it is the requested data ( Alternatively, remote data (blocks or chunks of data containing the requested data) Get from store 216 and return the requested data to storage gateway 252 .. The storage gateway 252 receives data from the storage service 212. Blocks or chunks can be stored in the local cache.
0072For writes, the storage gateway 252 b. -Can be written to the cal cache. In at least some embodiments, the write data is , Can be added to the block-based write log implemented in the local cache. Strike Rage Gateway 252 is a local cache on a regular, non-regular, or continuous basis. To upload new or modified data in the store to the primary data store 216 In addition, the service provider 210 communicates with the recipient data upload process (not shown). It may include a trusted sender-side data upload process (not shown). Book of write data Uploading from the included log is from the start process to the local data store 254 Can be executed asynchronously with the processing of read and write operations. At least some embodiments In the state, this upload process involves data deduplication, compression, parallelization, and TCP. One or more of the window scaling techniques may be adopted. As shown in Figure 6, the small amount An example of a data deduplication technique that can be employed in at least some embodiments is U.S. Patent Application No. 1. It is described in 2 / 981,393 and 12 / 981,397.
0073Remote data store 216 can provide essentially unlimited storage, but locally The cache can be limited in size. Therefore, the storage gateway 252 An older and / or relatively inactive database in the local cache Remove, replace, locks with newer and / or active data blocks, Or it can be overwritten.
0074[Shading gateway embodiment] In Fig. 7, one embodiment of the storage gateway is installed as a shadowing gateway. The architecture of the network environment example and the data flow in it are roughly defined. It is a high-level block diagram which shows. In Figure 7, the storage gateway 252 is the customer's Write data (eg iSCSI write) provided by storage service 212 Customer application to provide shadowing to remote storage As a "bump in the wire" between the customer and the customer's local data store Installed and enabled to act as a shadowing gateway to work And set. Remote data store 216 is implemented as block storage It can be.
0075In the embodiment shown in FIG. 7, the local data store 254 is the remote data store 216. What is the cache gateway embodiment in FIG. 6 in which is functioning as a primary data store? In contrast, customer processes (s) on client network 250 258 Acts as a primary data store for. Storage gateway 252 is shadow Once installed, enabled, and configured as an Ing Gateway, Stray The gateway 252 has one or more data ports (eg, iSCSI port). Publish to customer process (s) 258 on client network 250 To do. Customer process (s) 258 on client network 250 Then, through the storage gateway 252 data port (s), raw Read from cal data store 254 and write to local data store 254 Get it. Customer process 258 resides on client network 250 and gates Via the data protocol of the 252 data port (eg iSCSI protocol) , Any hardware, software, and communication with the storage gateway 252 And / or a combination thereof. Customer process 258, for example, Microso ft® SharePoint® and Oracle® Storage applications such as databases, servers (eg SQL server, Microsoft® Exchange® Server, etc.), Day Tabe application (for example, SQL database application, and Or acle (registered trademark) database application), Microsoft (registered trademark) ) Exchange® application or storage gateway 252 Client network that works to communicate with one or more data ports Any other application or processor running on one or more devices on the 250 Can be The customer process is, as used herein, 1 in the client network 250. Includes any software process that may be running on one or more devices But; the underlying hardware on which the customer process runs replaces the process Connections to and to storage gateway 252 data ports (s) Note that it can be involved in or carry out communication.
0076Read and write requests are made via gateway 252 data ports (s) Can be received. For reads, the request may cause further interference by gateway 252. Or it can be passed directly to the local data store 254 without any processing and the requested data It can be passed directly from the local data store 254 to the customer process 258. Local data Write requests directed to store 254 are also localized by storage gateway 252. Passed to data store 254. But write request to local data store 254 In addition to passing, the storage gateway 252 is new as indicated by the write request. Or update data to a remote data store via storage service 212 Can shadow to 216.
0077In at least some embodiments, the new or updated data is remote data. To shadow the tor 216, the storage gateway 252 is remote Uploaded to datastore 216, for example, in a first-in first-out (FIFO) write log. The write data to be loaded can be stored or buffered locally. At least yes In some embodiments, write logs can be implemented in block storage format, The write log contains one or more blocks (eg, 4MB blocks). Write request The write data received in can be added to the write log. From two or more write requests Write data can be written to the same block in the write log. Books related to blocks Metadata for inclusive data (eg offset in write log blocks and Length, as well as offset in the data store of interest), ranks as a metadata store Can be paid.
0078Storage gateway 252 is local, periodically, irregularly, or continuously. Shadow the write data stored in the remote data store 216 from the write log At service provider 210 to upload to the loaded data volume Sender data upload that communicates with the recipient data upload process (not shown) It may include a process (not shown). Uploading write data from the write log is Handling read and write operations from the start process to local data store 254 Can be executed asynchronously with. The upload process blows the write data from the write log Can be uploaded by clicking. When the write log block is uploaded successfully, it corresponds Block can be marked as free in the write log.
0079In at least some embodiments, the upload process is data deduplication, compression. , Parallelization, and one or more of TCP window scaling techniques can be adopted To. As shown in FIG. 7, a data deduplication technique that can be employed in at least some embodiments. Examples of procedures are described in U.S. Patent Applications 12 / 981,393 and 12 / 981,397. Has been done.
0080Service provider front end 280 connects to storage gateway 252 Note that you can manage. In at least some embodiments, storage games Tway 252 connects to service provider 210 via front end 280 Service provider 210 does not initiate a connection to gateway 252. Fu Lontoend 280 is a firewall, perimeter router, load balancer, gateway Servers, gateway proxies, console processes, and storage services Publish 212 to client network (s) 250 and its storage Service 212 interfaced with storage gateway (s) 252 Any common networking equipment and / or software that may be needed to connect Can include, but is not limited to, one or more of rothes.
0081In at least some embodiments, the storage gateway 252 is a service pro. Initiate all connections to Service Provider 210 via Bider Front End 280 However; service provider 210 does not initiate a connection to gateway 252. further, Network administrator process 260 does not initiate a direct connection to gateway 252; eg For example, network administrator process 26 to configure and manage gateway 252 Access to gateway 252 by 0 is service provider front end 280 Pass through service provider 210 via.
0082Provided by Storage Gateway 252 as a shadowing gateway The shadowing behavior is from the user's point of view on the client network 250. For example, it can be effectively transparent. Customer process (s) 258 Data port exposed by storage gateway 252 on network 250 Read and write to (s) (for example, iSCSI port) To do. From the point of view of customer process 258, storage gateway 252 is arbitrary It can look like another data target (eg an iSCSI target). Data port Read from customer process (s) 258 received on (s) The request is propagated to the local data store 254, which acts as the primary data store. Customer process (s) received on a data port (s) 258 The write request from is propagated to the local data store 254 and the remote data store 2 Shadowed to 16. The shadowing behavior of gateway 252 is the primary data Without significantly affecting the performance of the store or client network 250 , Can be run in the background.
0083For the "bump in the wire" shadowing gateway configuration shown in Figure 7. An example of a use case for this is disaster recovery. Storage gateway 252 Updates data from client network 250 to storage service 212 Send, but it's also called Snapshot 270, Shadow Volume or Store data in multiple volumes. Data is in block storage format Can be stored in snapshot 270. Data also ranks as local data store 254 Will be delivered. Corruption or loss of some or all of locally stored volumes If something happens that results, the corrupted or lost data will be in data store 2 Can be recovered from snapshot 270 of volumes stored within 16. storage Provider 210 is a customer network administrator (for example, a network administrator process). Shadowed on remote data store 216 through it (via 260) A snapshot of some or all of a locally stored volume from a volume It may provide an interface that may require a recovery of 270. At least some embodiments In the state, the shadowed volume from which the data is recovered is as up-to-date as possible. Before recovering snapshot 270 of data to ensure that At least part of the write log maintained by Rage Gateway 252 is remote Can be uploaded to data store 216. In some cases at least some Data is recovered directly from the write log maintained by storage gateway 252 Note that it can be done.
0084[Customer process-Gateway communication] As mentioned above, the customer administrator can, for example, go through the network administrator process 260. Through the service provider 280 front end to configure the toway 252 , Can communicate with storage gateway 252 (eg shadowing gateway) .. In at least some embodiments, one or more customer processes 258 are gated. Through the service provider 280 front end to make a request for way 252, It can also be configured to communicate with the storage gateway 252. For example, customer process 258 is a storage gateway through the service provider 280 front end It can be a SQL server configured to communicate with 252.
0085[Shadowing gateway boot strap technology] As shown in Figure 7, the storage gateway 252 is the shadowing gateway. Once installed, enabled, and configured, Storage Gateway 252 Has one or more data ports (eg, iSCSI port), client network Publish to customer process (s) 258 on network 250. Client Customer processes (s) 258 on the network 250 are then stored Local data store 25 via the way 252 data port (s) Can read from 4 and write to local data store 254. Read and A write request was passed to local data store 254 and indicated by that write request Write data is a snapshot of the local data store (s) 272 Shadowed to remote data store 216 so that it can be updated.
0086But when it was first installed, activated, configured, or for some reason Sometime after going offline in, the shadowing gateway is the customer's network Once online, a snapshot (up to singular) on remote data store 216 Or more than one) There can be data in the local data store 254 that is not in 272. Therefore , At least some embodiments are bootstras for shadowing gateways. A snapshot (s) may be provided in the meantime, while the local Input and / or to accurately reflect the data currently on the data store 254 At least some data from local data store 254 is remoted so that it can be updated Can be uploaded to the data store 216.
0087Figure 8 shows shadows in an example network environment, according to at least some embodiments. It is a high-level block diagram which roughly shows the bootstrap of the ing gateway. Su Trage gateway 252 shadows on client network 250 When online as a toway, gateway 252 takes snapshot 272 To match the local data store 254 with the remote data store 216 It can be determined that there is data in the local data store 254 that needs to be loaded. The gateway 252 upload process is then the local data store 254? Remote data store 21 in service provider 210 of these blocks of data You can start uploading to 6. The storage gateway 252 also has that data Expose the port to customer process (s) 258 and local data strike Start receiving and processing read and write requests directed to 254 and write Start caching the new write data indicated by the request into the write log and write Can start uploading write data from embedded logs to remote data store 216 To. Uploading data from the local data store 254, in that way, strikes Rage gateway 252 provides its shadowing capabilities to client network 250 It can run in the background while running on. Local data store 254 When the data upload from the storage gateway 252 is completed, the storage gateway 252 will display the data. Continue executing the doing function.
0088Figure 9 shows for a shadowing gateway, according to at least some embodiments. It is a flow chart of the bootstrap process of. Shadowing game, as shown in 300 Tway goes online on the customer's network. For example, storage gateway A new instance of is installed as a shadowing gateway on the network It is tolled, activated, and set. As another example, the shadowing gateway An existing instance can go online after being offline for some reason; Customer processes (s) read data while the way is offline And may have communicated directly to the local data store for writing. Another example As the shadowing gateway, the shadowing behavior is in the meantime, for some reason Temporarily interrupted at (for example, because the write log is full), pass-through mode It is possible to enter the screen and exit the pass-through mode to resume the shadowing operation. There is sex.
0089As shown in 302, the shadowing gateway can, if necessary, of existing data. You can start uploading from a local data store to a remote data store. For example , This is the new shadowing gateway, the local data store has already thrown data If so, existing data in the local data store will have a matched snapshot. Must be uploaded to a remote data store so that it can be generated. Another example Exiting pass-through mode and bringing the existing shadowing gateway back online When the shadowing operation is resumed, new data is sent to the local data store. May have been written, so take a snapshot on a remote data store , Must match the current data on the local data store.
0090As shown in 304, the shadowing gateway is exposed on the customer's network. Read from the customer process through the gateway data port (s) And can start accepting writes. As shown in 306, the shadowing gateway , Start caching the write data from the write to the write log, as shown in 308 Can start uploading write data from the write log to a remote data store To.
0091Uploading data from a local data store, started at 302, is a shadow The ing gateway accepts read and write requests and performs its shadowing capabilities It can run in the background while running on the customer's network. Local de Once the data upload from the data store is complete, the shadowing gateway will Continue executing the shadowing function.
0092Note that the order of the elements in Figure 9 can be different. For example, element 302 requires It can be executed after any one of the elements 304 to 308. In other words, shadow in Before the gateway starts uploading existing data from the local data store Can start accepting reads and writes, as well as performing its shadowing function ..
0093FIG. 10 enters pass-through mode according to at least some embodiments, It is a flow chart of the shadowing gateway that recovers from the above. Shadow as shown in 320 The ing gateway interrupts its shadowing function (ie, of the write data). You can enter pass-through mode by stopping caching and uploading), Meanwhile, reading from the customer process on the customer's network to the local data store Continues accepting and servicing receives and writes. The gateway is Shadowy If it detects any condition that could cause the function to fail, it may enter pass-through mode. One case As the shadowing gateway is full of write logs and uploads successfully If it detects that it cannot, it can enter pass-through mode. Gateway detected The condition can be alerted to the local network administrator; the administrator is then indicated by the alert Can deal with problems. For example, an administrator can allocate more memory to write logs, And / or more bandwidth can be allocated to the gateway upload process. tube The scholar can then notify the gateway that the problem has been addressed.
0094Shadowing gateway detected, for example, causing pass-through mode You can exit pass-through mode by receiving an instruction that the problem has been addressed If determined to be possible, the gateway may resume shadowing, as shown in 322. (Ie, start caching and uploading write data).
0095Localrs that have not been uploaded to the remote data store when exiting pass-through mode There can be data in the data store. Gateway writes during pass-through mode Only continues to receive and process requests, so new data is written to the local data store It may have been squeezed. Therefore, the shadowing gateway is shown in Figures 8 and 9. Run the bootstrap as shown in, and in pass-through mode as shown in 324 Remote data from local data store to recover at least some data Can be uploaded to Tastore.
0096Optimized for shadowing gateways, at least in some embodiments Bootstrap process from local data store to remote data store It can be adopted to reduce the amount of data loaded. Optimized bootstra The process blocks blocks of data that have already been uploaded to the remote data store. It can be detected and therefore avoid uploading blocks that have already been uploaded. Optimized bootstrap process from gateway to remote data store Generated and generated for the storage gateway process during a typical upload of data And keep track of the data available.
0097FIG. 11 shows remote data from the gateway according to at least some embodiments. A flow diagram of how to upload, update, and track blocks to the store. Connoisseur During normal gateway operation, the gateway tells the service provider, as shown in 360. Then, write data to a remote data store, specifically to a storage service. Upload. The storage service receives the write data, as shown in 342. , Each block (single or plural) (eg 4MB block) is remote data Get from the store. The storage service then writes write day, as shown in 344. Modify each block (s) according to the data, and modify the block (single) Upload the number or more) back to the remote data store with the new version name .. For each modified block, a toe indicating the modified block, as shown in 346. Kun is sent back to the storage gateway. The storage gateway is these Track Khun; every time a block is modified, the modified reference block is stray Must be sent to the service.
0098As shown in 348, the storage gateway is a regular service provider. Update the token manifest on a targeted or irregular basis , Can purge at least some of the locally tracked tokens. Storage gate Ay may need to track a large number of tokens. At least in some embodiments , Manifest tracks a large number of tokens locally from the storage gateway The burden of need can be removed. Storage gateway, manifest Stray regularly or irregularly to renew with tokens received by Toway The service can be called and each locally stored token can be purged.
0099In at least some embodiments, the optimized bootstrap process is a manifold. Which block by making a call to check the hash of each block in the fest To determine which blocks have been uploaded and which blocks have not been uploaded Using the manifest, which block indicated by the manifest is the local data Which block indicated by the manifest as to which block matches the block on the store Does not match the block on the local data store or needs to be uploaded accordingly Can determine if there is. In other words, which manifest is on your local data store Detect if a block is a dirty block and which is not a dirty block Used for In this way, the optimized bootstrap process is a mani Attempting to determine which blocks have already been uploaded via the fest , Blocks that have already been uploaded will not be uploaded again, only dirty blocks Make it uploaded. Optimized boo in at least some embodiments Blocks (dirty) that the tostrap process determines needs to be uploaded Reduce the amount of data actually uploaded from dirty blocks) Data deduplication technology is applied when uploading these blocks to obtain.
0100Figure 12 shows the shadowing gateway according to at least some embodiments. It is a flow chart of the optimized bootstrap process for. Bootstrap process For example, when the gateway exits pass-through mode, the shadowing gateway It is started against a. Blocks from the local data store, as shown in 360 To be acquired. Manifs that can be stored on remote data stores, as shown in 362 Est is the current block is a dirty block that needs to be uploaded Can be checked to determine if. At 364, the current block is manifest Follow data deduplication techniques as shown in 366 if dirty according to At least part of the block can be uploaded to a remote data store. The method is Then proceed to 368. At 364, the current block is dirty according to the manifest If not, the method goes directly to 368. In 368, if more blocks are processed The method returns to element 360 to process the next block. If not, Boots The trap process ends.
0101[Storage gateway security model] The embodiment of the storage gateway is data protection for the customer, as well as the customer or Protection against third party misuse and misuse of the gateway (eg, illegal copying) It can be implemented according to the security model provided. Figure 13 shows at least some fruit An aspect of the storage gateway security model according to the embodiment is shown.
0102In at least some embodiments, one aspect of the security model is storage gaming. Gateway 84 for use by Toway 84 in communication with service provider 60 Client network 8 without security credentials or other identifying information for It is to be delivered on 0 and installed first. The activation process can be adopted, Through it, the storage gateway 84 on the customer network is the service provider You can register for 60. In at least some embodiments of the activation process, storage Toway 84 will be responsible for each customer account in order to obtain the required security credentials. Connecting with service provider 60 as the correct gateway to und (eg, Initiate SSL (Secure Socket Layer) / TCP Connection) and Service Provider 60 Can identify itself to. During the activation process, the service customer faces gateway 84 And specify the name. In at least some embodiments, the service customer is a service customer. When logging in to the customer's account with Robida 60 and registering Gateway 84 Information used, including but not limited to the gateway name, to service provider 60 provide. However, the service customer did not log in to the storage gateway 84 and obeyed. For service customer security credentials and other account information, see Gateway 8 4 Not published on. This can minimize security risks to service customers To. This gateway name is the gateway 84 and other metade related to service customers. Stored by service provider 60 along with data, each gateway 84 Can be used in tracking and identification of. Service customers on client network 80 You may have one or more gateways 84 installed and activated, each one Note that it has a distinguished name and other metadata. Figures 15 to 17B show Further explained in the section entitled "Storage Gateway Activation Process" below However, the activation process that can be adopted in at least some embodiments is shown. Activation At Rothes, gateway 84 initiates a connection to service provider 60 and gateways Metadata about the Hay 84 platform, along with the public key, service provider Can be offered to 60. Service provider 60 is then used in the activation process, A temporary unique activation key may be provided to gateway 84. In addition, service customers Through the service provider console process to enable gateway 84 , May be required to log in to the customer's account; in that way, Gateway 8 4 can be matched against the account of the service customer attempting to activate gateway 84 To. Security gained through the activation process by Storage Gateway 84 Credentials and other metadata (eg, customer-provided gateway name), then the game Service Pro to identify Tway 84 to Service Provider 84 processes By storage gateway 84 in communication with various processes of the Bider 60 network Can be used.
0103In at least some embodiments, another security model, as shown in FIG. Aspect is that the storage gateway 84 is a customer processor on the client network 80. One or more data ports exposed to (s) 88 (for example, one or more) Accepts only externally initiated connections to the iSCSI port). Storage gate Ay does not accept connections initiated from other outsides and all necessary connections to external processes To start. For example, in at least some embodiments, the storage gateway 84 Is at least one secure connection to service provider 60 92 (eg SSL) Start Cure Socket Layer) / TCP Connection); Service Provider 60, However, the connection to gateway 84 cannot be started. Gateway start connection and at least Remote gate using long polling techniques that may also be used in some embodiments Examples of methods for way management are shown in FIGS. 18-20.
0104Further, as shown in FIG. 13, in at least some embodiments, a service customer (eg, an example). For example, the network administrator process 90) configures and manages the gateway 84. Therefore, do not connect directly to the storage gateway 84; instead, the storage gateway Configuration and operation requests for 84 are made through service provider 60, which is The request is gated through a secure communication channel 92 initiated by gateway 84. Pass it to Ay 84. For example, as shown in FIGS. 18 to 21, the settings for the gateway 84 And action requests are made through a console process on the service provider 60 network. Can be performed by or through network administrator process 90. Little At least in some embodiments, the console process is directed at the customer's gateway 84. Received, received configuration or operation requests maintain gateway-initiated connection 92 Transferred to the toway control plane. The gateway control plane is the subject of the request The current connection to gateway 84, eg, maintained on a particular gateway control server The connection is searched for, and the request is forwarded to the gateway 84 through the connection.
0105Therefore, in at least some embodiments, a user, network administrator, or The customer's process can also initiate a connection directly to the storage gateway 84. You also cannot "log in" and the operator on the service provider 60 network External people such as processes or processes also initiate a connection to storage gateway 84 Can not. This, along with other aspects of the gateway security model, is a storage device. Security credentials and other behavioral information on the Toway 84 can be found on outsiders or processes. It can help prevent you from being intentionally or unintentionally endangered by a computer.
0106In another aspect of the security model, gateways are enabled and running, strays. All communication between the gateway and the storage service can be protected and encrypted .. As mentioned earlier, one aspect of the security model is storage gateways and storage. Communication with the service is a secure gateway-initiated connection (eg SSL / TCP connection) ) To be executed. Encryption technology, such as public / private key encryption, is a game Can be used for communication over a secure connection at the start of the toway.
0107Figure 14 shows the activation of the storage gateway according to at least some embodiments. , Configuration, and operation, at least some aspects of the gateway security model It is a flow chart which shows. As shown in 400, the storage gateway is the customer network. Can be instantiated on the desktop. For example, an instance of a storage gateway To generate, the storage gateway usually serves behind a firewall. Virtual or physical appliance on the customer's local network or data center Can be installed as. For example, in at least some embodiments, storage The gateway is one, such as a server system on the service customer's local network. Can be downloaded on one or more computing devices, or otherwise Can be implemented as a virtual appliance that can be installed. Alternatively, storage Tway is a dedicated device or app that can be coupled to the service customer's local network. Can be implemented as a liance; its dedicated device or appliance is a storage gate It may include software and / or hardware that implements the functionality of the way. To 402 As shown, the instantiated storage gateway identifies the gateway. To obtain gateway security credentials, service providers and advisors Start the activation process with the customer. In at least some embodiments, security The authentication information includes a certificate signed with the public key provided by the gateway. Activation process example Will be described below in connection with FIGS. 15-17B. The activation process is done by the gateway When first installed on the customer network, it can be initiated by the gateway, Also, at other times, for example, the gateway device is upgraded, maintained, or something. It can also be started when the power is turned on after it was turned off for other reasons Please note. The storage gateway is a service, as shown in 404 in Figure 14. Establish a secure connection to your provider. Can be used in at least some embodiments Examples of methods for gateway start connection using polling technology are shown in Figures 18 to 21. Shown in. As shown in 406 of Figure 14, the customer is the service provider console. Set up and operate the storage gateway throughout the process. Gateway start contact Sequel and using long polling techniques that may be used in at least some embodiments , Examples of methods for remote gateway management are shown in FIGS. 18-21. Figure 14 As shown in 408, the storage gateway is, for example, a storage service. Yes to identify the gateway to the service provider to communicate with Rothes Gateway security credentials obtained during the activation process and possibly other meta Use your data to communicate with your service provider.
0108[Storage gateway activation process] Embodiments of the storage gateway include, for example, as an on-premises storage device and Service customer network and storage service provided by the service provider It can function as an interface to and from the screw. In at least some embodiments The storage gateway is the customer's local network infrastructure in the customer data center One or more compute, such as a server system coupled to a structure Virtual devices that can be downloaded or otherwise installed on the device Or it can be implemented as an appliance. Alternatively, the storage gateway is the customer's Dedicated equipment or application that can be coupled to the local network infrastructure Can be implemented as Dedicated device or appliance realizes gateway function Software and / or hardware may be included.
0109In at least some embodiments, the stray after the gateway is installed The gateway must be enabled by the service provider in order to use the gateway There is a need. This section describes storage gateway identification, authentication, and authorization. Describes the methods that can be performed during gateway bootstrap or activation. Gate The way activation method identifies the storage gateway and provides the customer's service provider. Associated with da account. However, customer credentials are striking during the activation process. Not published to the gateway. In at least some embodiments, the customer serves Used to register the gateway 84 by logging in to the customer's account with the provider Providing information to service providers, including but not limited to gateway names .. However, the customer does not log in to the storage gateway and therefore the customer's security Authentication information and other account information will not be published on the gateway. This is the customer Security risk to In at least some embodiments The service provider account used by the customer in the activation process is shown in Figure 5. As other storage resources and hardware provided by the storage service Includes, but is limited to, virtualization hardware resources provided by the hardware virtualization service No, care to manage other resources provided to customers by the service provider It can be the same account used by the customer.
0110Figure 15 relates to the gateway activation process according to at least some embodiments. A network that represents the service customer and service provider components or entities that you give. It is a high-level block diagram of an example of an environment. These participants are storage gates Way 84, Network Administrator Process 90, Console Process 68, and Gate It may include, but is not limited to, the way control 70. Storage gateway 84 is usually , Behind the firewall, service customer's local network or data center -As a virtual or physical appliance on (eg client network 80) Can be installed. For example, the storage gateway 84 is, for example, in a virtual machine. Can be a virtual appliance running on a server on client network 80 It can be downloaded and instantiated on the desktop. Service provider 60 network Console process 68 on the software, for example, to sign on to the customer's account From a device on client network 80, or to client network 80 Accessed by or through network administrator process 90 from an external device It can be possible. For example, console process 68 depends on service provider 60 Network management to view and manage the accounts and resources provided Accounts for each service customer through network administrator process 90 You can sign on through it to a web interface or some other interface Can provide a service. Service Provider 60 Network Gateway Control 70 Process Su or Plain is an instrument in one or more customers of Service Provider 60 Added to one or more storage gateways (s) 84 Can perform trace and management functions. Gateway control 70 and console process 68 , For example, one or more server computers on the service provider 60 network It can be mounted on the data device. In at least some embodiments, gateway control 70 It may be used to provide load balancing and high availability, two or more Getou including E b Control Server It can be implemented as a control plane.
011116A and 16B have gateways according to at least some embodiments. It is a process flow diagram which shows the interaction between the components shown in FIG. 15 during the activation process. Yes The activation process involves two points of interaction from the customer's point of view. The first is Figure 16A The customer interacts with the gateway 84 as shown in. The second is as shown in Figure 16B. , The customer interacts with the Service Provider (SP) Console 68.
0112Figure 16A shows the customer (to network administrator process 90 in Figure 15) during the activation process. Represented by), Gateway 84, and Service Provider (SP) Gateway Shows the interaction between controls 70. Gateway 84 is installed and / or electrical After the source is turned on, gateway 84 generates a public key (eg RSA key pair) and The hardware of the device on which the gateway 84 is installed and / or Collect metadata about the software. For example, the metadata is IP address, M It may include an AC address, or other hardware and software characteristics of the device. Game Tway 84 then passes the public key and metadata through, for example, HTTP POST. Then, it is disclosed to gateway control 70. Gateway control 70 is enabled accordingly It can generate an activation key and return its activation key to gateway 84. The activation key is Global It can be a unique identifier (GUID), eg, a randomly generated number of N bits. Ge The gateway control 70 uses the activation key as the public key and the key obtained from the gateway 84. Can be stored with data.
0113After receiving the activation key from gateway control 70, gateway 84 is gatewayed. B 84 At a fixed port (IP address: port) on the VM or device, the client Provides an activation key within network 80. Customers then network administrator professionals Access to the fixed port of gateway 84 to get the activation key via Seth 90 The access is the activation key in the query string and the service provider (SP) ) Redirected to Console 68 process.
0114In at least some embodiments, the activation key is for a period of time or duration (eg, for example). , 30 minutes), then the activation key expires. At least some implementation In the form, the activation key is valid only for the specified lifetime, so the expired activation A background garbage collection process that removes keys is a service provider May be offered at Ida 60. Survival for activation key in at least some embodiments The period is gateway 84 on the service provider 60 side to handle the boundary example. May be longer than above (eg 45 minutes on the service provider 60 side, Gateway 30 minutes on B 84).
0115Figure 16B shows the customer (to network administrator process 90 in Figure 15) during the activation process. Represented by), Service Provider (SP) Console 68, and Service Provider Shows the interaction between Ida (SP) gateway control 70. Network administrator process When 90 gets the activation key from gateway 84, gateway 95 is served by the customer An activation key can be used to add to the provider 60 account. SP Consaw After being redirected to Le 68, the customer goes to that account (for example, network administrator) Login (via process 90) and gateway 84 against gateway control 70 The enable key from the query string is used to fetch the exposed metadata .. At least part of this metadata (eg, through network administrator process 90) Is displayed to the customer. Returned from gateway control 70 to SP console 68, customer The metadata displayed in 90 was previously in gateway control 70 by gateway 84. Notify customer 90 about gateway 84 to be activated, which is the metadata provided to Can be used to The metadata displayed is the one indicated by that metadata. Each gateway 84 is installed on the customer's network It can be confirmed to the customer 90 that it is 84. For example, the IP address of gateway 84 is What can be displayed and confirmed by customer 90 is the IP address of gateway 84. Furthermore Credentials obtained from customer 90 to log in to your account (eg customer account) The count number and / or other customer identification information) is the customer 90, each gateway Authenticate as a customer who owns 84 and associate customer 90 with each gateway 84 Can be used when
0116Customer 90 receives additional information via SP Console 68, eg Gateway 84. You may also be prompted to enter a name for it. After looking at and verifying the displayed metadata, Customer 90 requires, for example, a "confirm" or "activate" or "register" user interface Gateway 84 gateway via SP console 68 by selecting the element B. Registration to Control 70 can be approved. Customer 90 gateways with SP console 68 If you authorize 84 registrations, SP Console 68 will have the activation key obtained from Customer 90. Can be passed to gateway control 70. For example, customer-provided name and advisor for gateway 84 Customer information, such as the customer account ID, can also be passed to the gateway control 70. Customer-provided The activation key was previously provided to gateway control 70 by gateway 84. -It is collated against. Customer information (for example, the name of gateway 84) is, for example, game By gateway control 70, along with the metadata previously provided by Tway 84 Is stored.
0117In at least some embodiments, SP console 68 and SP gateway control 70 All exchanged between and between gateway 84 and SP gateway control 70 The data can be encrypted. In at least some embodiments, the customer's credentials, access Top secret data such as sesqui or private keys are not passed within the activation process.
0118Referring again to FIG. 16A, in at least some embodiments, the SP gateway system 70 is responsible for maintaining all information regarding the registration and activation of Gateway 84. .. Gateway 84, in the meantime, asks for information to generate a certificate signing request (CSR). And poll the SP gateway control 70 continuously. SP, as shown in Figure 16B Gateway control 70 receives authorization from customer 90 via SP console 68 and consults When the customer-provided activation key is matched against the activation key provided by gateway 84, As shown in FIG. 16B, the SP gateway control 70 receives customer information from customer 90. By providing metadata that includes, but is not limited to, at least a portion of Can respond to GET requests on way 84. Gateway 84 then generates a CSR and Send to SP gateway control 70. In response to the CSR, SP Gateway Control 70 Generate a certificate and sign it with the previously provided public key of gateway 84. In at least some embodiments, the certificate is the customer and / or gateway information, For example, it may include a customer account ID and 84 customer-provided gateways. SP game Tway Control 70 is then encrypted with the public key previously provided by Gateway 84. Respond by sending a personalized, self-signed certificate to gateway 84. Certificate Is then authenticated in future communications from gateway 84 to service provider 60. Can be used for
0119In at least some embodiments, the customer uses the same activation key to multiple gates. System / hardware specific information to help prevent enabling Way 84 Also with the activation key exposed by gateway 84 to SP gateway control 70 Can be included in.
012017A and 17B show storage games according to at least some embodiments. It is a flow chart of the activation process from the viewpoint of Tway. As shown in Figure 17A, 500 After the gateway is installed and / or powered on, the gateway B checks the persistent storage device to determine if it is already enabled. For example, the gateway powers for upgrade, maintenance, or some other reason. May have been cut. If the gateway is enabled, the activation process is , Proceed to element 530 in Figure 17B, where the gateway is set up from the SP gateway control. You can get fixed information.
0121In 500 in Figure 17A, if the gateway was not previously enabled, the activation process Goes to element 502 in Figure 17A, where the gateway requests a certificate signing request (CSR). Check if you have any persistent customer information to generate. Gateway If a has persistent customer information, the process proceeds to element 520 in Figure 17B. Gate If the way does not have persistent customer information, the process proceeds to element 504 in Figure 17A. At 504, the gateway generates a public key (eg, an RSA key pair). gateway Is the hardware of the device on which the gateway is installed and / or Metadata about the software can also be collected. For example, the metadata is the IP address, It may include a MAC address, or other hardware and software characteristics of the device. Ge Tway then puts the public key and metadata on the SP gateway, as shown in 506. B. Publish to control. At 508, the gateway issues the activation key from SP gateway control Receive. At 510, the gateway is a fixed port on the service customer's network (I) Provide activation key on P address: port).
0122The gateway then generates a CSR, as shown in Figures 17A 512-516. SP gateway control can be polled for customer information needed for this. client The information may include the customer's account ID and the customer-provided name for the gateway. Not limited to this. At 512, the gateway is, for example, one minute or some other period, Can pause and then check if information is being received from SP gateway control Can be At 514, if no information is received, the gateway, as shown in 516 Checks if the activation key has expired. At least some embodiments The activation key is valid for a period of time or lifetime (eg, 30 minutes) and its After that, the activation key expires. At 516, if the activation key has not expired, the activation process Returns to element 512 in Figure 17A to continue polling for SP gateway control. To. At 516, if the activation key has expired, the activation process will issue a new activation key. Return to element 504 in Figure 17A to get from the SP control plane.
0123514 in Figure 17A, valid if customer information is being received from the SP gateway control The transformation process proceeds to element 518 in Figure 17A, where the gateway persists customer information. Store in memory. In at least some embodiments, the received customer information is encrypted Therefore, the gateway may decrypt the information before storing it. The process, Then proceed to element 520 in FIG. 17B.
0124Referring to Figure 17B, at 520, the gateway it already has a certificate. You can check if. At 520, if the gateway already has a certificate, Rothes can proceed to element 530 in Figure 17B, where the gateway SP gates the configuration information. Can be obtained from way control. At 520, if the gateway does not have a certificate, pro Seth proceeds to element 522. At 522, the gateway generates a CSR and S the CSR Send to P control plane. At 524, the gateway responds to the receipt of the CSR and SP Receive a security certificate from the control plane; the certificate is sent to the gateway Can function as curity credentials. At 526, the gateway provides the activation key ( (See step 510 in Figure 17A) can be disabled. At 528, the gateway is enabled To maintain the information obtained in the process (certificate, customer-specified gateway name, etc.) In, its current state can be saved.
0125At this point, the activation process is complete. At 530, the gateway SPs the configuration information Can be obtained from gateway control. In at least some embodiments, the customer gates When notified that the way has been successfully activated, the customer is in via the SP console. Stalls and enabled gateways can be configured. SP console, it's the customer You can log on to a user interface, such as a web interface, for a customer. Provide to your account and select a gateway (which can be identified by a customer-specified name) , You can specify the settings for that gateway. In at least some embodiments, S The P console communicates this setting to the SP gateway control, which in turn is the gateway. B. Specified via a connection initiated by itself (eg, and an SSL / TCP connection) Set up the gateway.
0126[Activation key security] The activation key is public on the service customer's network, as shown in 510 in Figure 17A. Is it a customer in the query string, made available with an open IP address and unencrypted? Can be passed to the SP console. The activation key has a limited lifetime and the IP address is Known only to customers, there is still a short period of time when the activation key is exposed on the IP: port .. The activation key is the metadata that the gateway also exposes to SP gateway control. Without it, it would be useless on its own, but the gateway has been somewhat vulnerable for this short period of time. It can be. In at least some embodiments, the customer is a malicious user or processor. Helps get the activation key to prevent someone else from activating the gateway For this purpose, security groups or other security measures may be utilized. In addition, customers Must log in to the SP console process to activate the gateway So the gateway can match the customer account trying to activate it.
0127[Remote gateway management using gateway start connection] Embodiments of the storage gateway include, for example, as an on-premises storage device and A network of service customers and a storage server provided by the service provider It can function as an interface to and from the screw. In at least some embodiments The installed storage gateway is a gate implemented by the service provider It can be activated, tracked, configured, and managed through way control technology. Figure 18 is less Also shows an example of a gateway control architecture that can be adopted in some embodiments. It is a high level block diagram. In at least some embodiments, as shown in FIG. , Gateway control 70 is two or more gateway control servers 74 (eg, gates). It may include a group of way control servers 74A, 74B, 74C, ...). Multiple games The toway control server 74 can provide load balancing and high availability. Given in operation Occasionally, certain installations and activated strikes on the service customer's network 80 Rage gateway 84 is connected to a particular one of gateway control servers 74. However, the storage gateway 84 will at some other time be a different gateway control server. Note that it can be connected to bar 74.
0128The gateway control server 74 currently connected to the storage gateway 84 Send a request or command over intermediate network 50 to storage gateway 84 By doing so, the storage gateway 84 can be managed. Storage gateway 8 Requests initiated from gateway control server 74 to manage 4 are configuration change requests And may include, but are not limited to, operation requests. But storage gateway 84 Can be deployed behind the client network 80 firewall, so the gate Way control server 74 will not be available unless an exception rule is created for gateway 84. It is possible that the gateway 84 cannot be reached from outside the firewall. In addition, few At least in some embodiments, the security mode for the storage gateway 84 Dell includes, but is not limited to, service provider processes, external processes It can indicate that you are not allowed to initiate a connection to the gateway 84.
0129In at least some embodiments, the service provider connects to gateway 84. While implementing a security model that does not allow you to start Allow bar 74 to send a request or command to storage gateway 84 For remote gateway management using gateway start connection and And equipment is provided. In the remote gateway management method, the gateway makes a connection request Initiate a connection to the service provider by sending. At least some fruit In the embodiment, the connection to a specific gateway control server 74 via the load balancer 72. Is established. However, the gateway 84 sends a request message to the gateway start connection. Do not send to the service provider via. Instead, a service provider (eg, The gateway control server 74) is the gateway 8 while the gateway 84 waits for a response. Holds the connection hold request sent to 4. Requests to gateway 84, for example, Work administrator process 90 or gateway 84 is installed on it Serving when received from some other process on the client network 80 The provider (eg, gateway control server 74) makes a request to the service provider. The gateway start connection that the data (for example, gateway control server 74) keeps Send to gateway 84 via. Gateway 84 also responds to requests It may be sent to service provider 80 via a toway start connection.
0130In at least some embodiments, a connection from gateway 84 is established for it. The gateway control server 74 (for example, gateway control server 74A) is The connection can be registered with the registration service 76. Gateway control server 74 against it Upon receiving a request for gateway 74 that does not hold a connection, the gateway controller Bar 74 finds which gateway control server 74 holds the connection In order to make an inquiry to the registration service 76, the request is kept connected to the gateway 84. It can be transferred to the gateway control server 74 that it has. In some embodiments, as an alternative And the gate that receives the request for gateway 74 that does not hold a connection to it The way control server 74 makes the request to two or more other gateway control servers 84. Can simply be broadcast to.
0131In at least some embodiments, the service provider 80 is the gateway start contact. A ping process may be employed to monitor the continuation. In the ping process The gateway control server 84, which maintains a connection to the way 74, periodically or irregularly A ping message can be sent to gateway 84. Gateway 84 is a ping Respond to the sage. Gateway 84 pin for a specified timeout period When it detects that it is not responding to the g message, the gateway control server 74 contacts it. The continuation can be interrupted, and registration service 76 can unregister the connection.
0132In at least some embodiments, ping messages are gated at regular intervals. Can be sent to EYE (singular or plural) 74. At least some embodiments have a connection Short intervals to untrusted gateway 84, and connections are generally reliable Ping messages will be sent to the gateway at longer intervals , The ping interval can be adjusted according to the reliability of the connection of a particular gateway 84. pin The g interval over time for a given gateway 84, when the connection remains reliable It can increase and decrease for a given gateway 84 where the connection is unreliable.
0133In at least some embodiments, the gateway 84 is the gateway-initiated connection. Can detect whether is terminated or interrupted. Check that the connection is closed Upon issuing, Gateway 84 makes another connection request to the service pro to reestablish the connection. Can be sent to the bidder 80. The connection is a different gateway than the one that previously held the connection B. Note that you can reconnect to Control Server 74. At least some embodiments In the state, gateway 84 monitors the ping message and the ping message points. By determining that it has not been received through the connection during the specified timeout period Therefore, it can be determined that the gateway start connection is interrupted.
0134In this way, in the remote gateway management method, the gateway 84 is a service. Establish a connection to your provider to anticipate requests (s) from your service provider Period and wait. The service provider keeps the connection hold request for gateway 84 To have. Upon receiving a request for gateway 84, the service provider issues the request. To each gateway through the gateway start connection. Service provider Both Ida and the gateway monitor and manage the connection, and the connection is medium for some reason If it fails, the interruption is detected and allows gateway 84 to reestablish the connection. To.
0135Figure 19 uses a gateway-initiated connection according to at least some embodiments. It is a flow chart of the method for remote gateway management. As shown in 600, Gateau Ay establishes a connection to the gateway control server via a connection request. For example As shown in Fig. 18, the gateway is contacted with the gateway control server by the connection request. Outbound SSL / TCP connections can be established through a load balancer. 602 in Figure 19 As shown in, when the connection to the gateway is established, the gateway control server is connected. Keep the continuation and maintain that connection. As shown in 604 in FIG. 19, the gateway controller The bar can receive requests to the gateway. For example, gateway control server 74 makes a configuration request or an operation request to the gateway 84, as shown in FIG. Received from each network administrator process 90 via console process 68 .. After the gateway control server receives the request to the gateway, the gateway system Your server makes the request through the gateway start connection, as shown in 606 in Figure 19. , Forward to the gateway.
0136Seeing Figure 18 again, the service customer is facing the storage gateway 84 shown. Service Provider Console 60 to initiate a configuration change request or operation request Can be accessed. For example, a network administrator may go through network administrator process 90. Through, the request may be sent via console process 68. Console process 68 Then sends the request to the gateway control server 74 behind the load balancer 72. obtain. However, the gateway control that console process 68 sends a request to it Server 72 is a gateway control server that holds a connection to each gateway 84. It may not be bar 72. For example, gateway control server 72B is gateway B. It is possible to maintain the connection to 84, but on the other hand, the request to gateway 84 is a gateway system. Can be sent to your server 72A. Therefore, it receives a request from console process 68 The gateway control server 72 (for example, gateway control server 72A) makes a request. A gate that holds a connection to gateway 84 for delivery to the appropriate gateway 84 Need to forward request to way control server (eg gateway control server 72B) There can be. Thus, at least some embodiments are gateway control servers. 72 (eg server 72A) received from console process 68, by request Gateway control server that currently holds a connection to the specific gateway 84 A request to a particular gateway 84 can be delivered to 72 (eg, server 72B). A method or a plurality of methods may be provided.
0137In some embodiments, server 72 makes a connection to it to achieve this. Gateway control server 72 that receives requests for gateway 84 that it does not hold (For example, server 72A) is for all of its peer gateway control servers 72. , The request can be broadcast. FIG. 20 shows the gateway according to some embodiments. B. For the control server to broadcast the gateway request to its peer server It is a flow chart of the law. As shown in 620, each gateway control server 72 is instant Once generated, server 72 may register with registration service 76. Gateway control sir When the bar 72 exits, the server 72 is unregistered from the registration service 76. Registration Service 76, for example, by database service or distributed storage service Can be sponsored. Gateway control server 72 (eg, server), as shown in 622. 72A) for gateway 84, where server 72 does not have a connection to it. Can receive requests. Broadcast request to its peer gateway control server 72 To do so, the gateway control server 72 (eg, server 72A) is shown in 624. As such, its peer gateway control server 72 (eg, servers 72B and 72) Registration service 76 may be polled to discover C). Gateway control server -72 (eg server 72A) then, as shown in 626, registration service 76 It can forward gateway requests for all of the servers 72 found through. On request The gateway control server 72 that currently holds the connection to the indicated gateway 84 (For example, server 72B) may then send the request to its respective gateway 84. To.
0138Figure 21 shows the appropriate gateway control server according to at least some embodiments. It is a flow chart of an alternative method for acquiring a gateway request for. As shown in 640 From gateway control server 72 (eg server 72B) to gateway 84 Upon receiving the connection request, server 72 registers pairing with gateway 84. Register within 76. Gateway control server 72 (eg, sir) as shown in 642 Bar 72A) to gateway 84 where server 72 does not have a connection to it Can receive requests for. Server 72 holds a connection to it, as shown in 644 Gateway control server 72 that receives requests for gateway 84 that is not (for example, For example, server 72A) then has a game that currently holds a connection to gateway 84. Registration service to find Tway Control Server 72 (eg Server 72B) You can query 72 and then submit the request to Registration Service 76, as shown in 646. It can be forwarded to the gateway control server 72 indicated (eg, server 72B). .. A gateway control server that currently holds a connection to gateway 84 as indicated by the request Bar 72 (eg, server 72B) then makes a request via a gateway-initiated connection. Can be sent to the corresponding gateway 84, respectively.
0139In at least some embodiments, the request is delivered to gateway 84, thereby When processed, the status is the connection from gateway 84 to that gateway 84. Is returned to the gateway control server 72 (for example, server 72B) that currently holds the This is followed by the gateway control server where it received the request previously forwarded from it. Status is returned to -72 (eg, server 72A), which in turn is then its stator. Returns to console process 68. Console process 68 is then the result of the request The customer process that initiated the request (eg, network administrator process 90) Can be provided to. If the request cannot reach the target gateway 84 for some reason For example, the gateway 84 indicated by the request is unavailable or found If this is not possible, console process 68 has initiated the request, indicating that the request has failed. It can be provided to a customer process (eg, network administrator process 90). Customer process May retry the request if necessary or desired.
0140Figure 22 establishes and supervises a gateway start connection according to at least some embodiments. It is a flow chart of a method for visualizing and maintaining. As shown in 660, the gateway is Can be instantiated on the client network. As shown in 662, Instagram After the service is generated, the gateway makes a secure connection to the service provider (for example, SSL). Make a connection request to the service provider to establish a cure socket layer) / TCP connection) Send to Ida. In at least some embodiments, the game in the service provider The toway control process can hold the connection and register the connection as shown in 664. You can register with us. The request to the gateway received by the service provider is It can then be forwarded to the gateway through the gateway start connection.
0141As shown in 666, the gateway control process can disrupt the connection. For example, few At least in some embodiments, the gateway control process is periodic or non-regular. You can ping the gateway through the connection and the gateway does not respond to that ping If it detects, the connection may be interrupted. Gateway if registered with the registration service The control process may unregister the connection.
0142As shown in 668, the gateway may detect that the connection is interrupted. example For example, in at least some embodiments, the gateway control process is periodic or informal. Periodically, the gateway can be pinged through the connection. The gateway is a service provider The connection is interrupted by determining that the ping from Ida is not being received through the connection It can be detected that it has been done.
0143Either on the service provider side or on the client network / gateway side Other methods for detecting interrupted connections may be employed in some embodiments. Please note that.
0144[Gateway proxy] Figure 18 above shows a gateway control play that includes multiple gateway control servers 74. Indicates a service provider network that includes gateway control 70 implemented as .. In at least some embodiments, the service provider network has multiple games. Gateway to communicate with storage gateway, including toway proxy nodes It may include a gateway proxy plane that can be used by the control plane. Gate Way proxy holds gateway start connection to gateway control server 74 And can be used to manage. Gateway 84 connects to the gateway proxy Start the continuation; the gateway proxy can maintain the communication channel to gateway 84, Not only does it help prevent misuse of multiple copies of the same gateway 84, but it also helps. The service between the service provider (eg, gateway control server 74) and the gateway Can help ensure a safe replacement of the sage.
0145[Interaction between gateway and proxy] Figure 23A shows the gateway proxy plane according to at least some embodiments. A block that outlines the architecture for a service provider network, including It is a figure. The gateway proxy plane is two or more proxy nodes 700, proxies Sistore 702, client-side interface process exposed to external network (CIP) 720, and proxy node 700 and game not exposed to external network Server-side interface pro with toway control server (s) 74 It may include Tocol (SIP) 710. In some embodiments, gateway proxy 7 00 is real on the same physical device as the gateway control server (s) 74 Can be disguised. In another embodiment, the gateway proxy 700 is a gateway control server. It can be mounted on a device separate from the bar (s) 74.
0146The installed and activated storage gateway 84 is a gateway protocol. CIP 70 secure connection request to Synode 700 (eg SSL / TCP connection request) Start through 0. Proxy node 700 receiving a connection request (in this example, a proxy Node 700B) is the gateway identifier of gateway 84 that initiated this connection and Proof of the gateway associated with the connection request to find the customer account identifier Inspect the book. The customer and gateway 84 are the gateway identifier from the certificate and Can be authenticated using the customer account identifier. After customer and gateway 84 authentication , Proxy node 700 then communicates with gateway 84 to which it is connected, positive Proxy store 70 that it is an authoritative proxy 700 Publish to 2. Proxies (eg, proxies 700A and 700B) are specific Proxy store 7 to discover other proxies that currently hold a connection to the gateway You can make an inquiry to 02.
0147In at least some embodiments, the proxy store 702 is actually a database. Can be revealed. The database can be either a distributed database or a centralized database. In at least some embodiments, the proxy store 702 may store the following associations: Ru: (Gateway ID, account ID, proxy endpoint)
0148If the message is sent to gateway 84, proxy 700 will be which proxy 7 02 to proxy store 702 to find out if it has a connection to gateway 84 Can make inquiries. In at least some embodiments, the game is in the proxy store 702. There is only one entry per Tway 84.
0149[Gateway control server and proxy interaction] Figure 23B shows the gateway proxy plane according to at least some embodiments. Indicates a gateway control server that sends messages to the gateway through. Figure 23B As shown in, in at least some embodiments, the gateway control server 74 It may have messages that need to be sent to a particular gateway 84. Gateway system Your server 74 sends messages through SIP 710 to the gateway proxy node. Send to 700. Proxy node 700 receiving the message to gateway 84 When holding a connection, proxy node 700 gates messages through that connection. Transfer to Way 84. However, the proxy node 700 that receives the message is gatewayed B. If the connection to 84 is not maintained, the proxy node 700 is which proxy node 70. To proxy store 702 to determine if 0 holds the connection to gateway 84 Make a query and send a message to its official proxy node 700 (in this example, a proxy) Transfer to 700B). The official proxy node 700 then, over that connection, Forward the message to gateway 84.
0150Figure 23C shows the gateway proxy plane according to at least some embodiments. Indicates a gateway that responds to gateway control server requests through it. At least go In some embodiments, the response from gateway 84 to gateway control server 74 is Starting at CIP 720, which receives the response from gateway 84, as shown in Figure 23B. Follows the reverse route that the request from gateway control server 74 to gateway 84 followed. It can be. The CIP 720 is the proxy node it received the request from (proxy 7) Send a response to 00B). Proxy 700B is the gateway control server which response Note that we don't know what to do with 74. Proxy 700B it makes a request there Make a request by sending a response to the proxy node (proxy 700A) received from Complete. Proxy 700A then responds to the gateway control server that initiated the request. Send to bar 74.
0151[Connection monitoring and management] In at least some embodiments, a proxy is used to manage gateway-initiated connections. The ping process used in can be implemented. In at least some embodiments The gateway 84 has a secure connection to the gateway proxy 700, as mentioned above. For example, initiate an SSL / TCP connection via CIP 720. Gateway proki The 700 may send ping messages to gateway 84 on a regular or non-regular basis. To. Each ping message can contain a timeout; gateway 84 within a time interval If it does not receive a ping, it closes the current connection and connects via CIP 720 Start again. At any point in time, at least in some embodiments, the proxy There is only one proxy gateway mapping in store 702. Gateway If the proxy 700 sends a ping and does not get a response from gateway 84, it is , Close that connection to gateway 84.
0152Gateway proxy 70 on all pings, at least in some embodiments 0 queries proxy store 702 and another proxy 700 goes to gateway 84 By determining if it exposes its connection to a given gateway 84 Check if it is a legitimate proxy for it. Where it's not a formal proxy If so, Proxy 700 closes the connection to Gateway 84. This is proxy node 70 If multiple connections to 0 are initiated by the same gateway 84, for example Toway 84's certificate is copied to another gateway and both gateways connect Can be dealt with if you try to start.
0153Figure 23D shows the gateway proxy plane according to at least some embodiments. Indicates a ping message exchange for. In at least some embodiments, Gateau With respect to the way proxy, ping is end-to-end ping. Reason to ping The reason is that the TCP "keepalive" function has a minimum interval of 2 hours, while the embodiment That connection timeouts or terminations may need to be detected at shorter time intervals To.
0154In at least some embodiments, the ping follows the path shown in Figure 23D. Ge The toway proxy node (in this example, proxy 700B) pings the message Send via SIP 710. The message is on the gateway proxy node 700 You'll hit one of them, the proxy 700A in this example. Proxy 700A is a proxy Formal proxy 7 to gateway 84 by querying Sister 702 Find 00 (proxy 700B in this example) and ping proxy 700 Transfer to B. Proxy 700B forwards the message to gateway 84 and gateways The response from Way 84 follows the same path. Proxy 7 in at least some embodiments When 00B gets the response to the ping from gateway 84, it is that gate Increase its ping interval for way 84. When gateway 84 connection is interrupted, p The ing interval can be reset to the minimum value. In this way, poor gateways and proki Connections between them tend to be pinged more often.
0155Proxy 700 first sends a ping message to SIP 710 to p The end-to-end ping method described above, which initiates an ing message, is a gateway. It can help ensure that the proxy node 700 is reachable from the control plane. If the ping fails, the proxy 700 will do it (for example, due to network partitioning). Can close the connection to gateway 84, assuming that is not reachable from the control plane ..
0156[Remote gateway management using long polling connection] In some embodiments, long polling techniques are used for gateway-initiated connections. Can be done. Seeing Figure 18 again, long polling is a server (eg, Gateau). From the control server 74) to the client (eg storage gateway 84) It is a polling technology that emulates push-type distribution of information. Long polling technology Now the client (eg storage gateway 84) is the server (eg game) A standard client by initiating a long poll connection to the Tway Control Server 74) Request information from the server, as in server polling. But sir If the bar does not have any information available to the client, give an empty response Instead of sending, the server holds the client's request and the information is on that client Wait for it to become available. When the information becomes available, the server (eg, get) The control server 74) can respond to the client's long polling request. Response contains information sent to the client (eg, Storage Gateway 84) Mu.
0157In the gateway start connection method using long polling, gateway 84 is Establish a connection to the gateway control server 74 via a polling request. example For example, gateway 84 is load balanced by a long poll request, as shown in FIG. Outbound SSL / TCP connection to gateway control server 74 through device 72 Can be established. Gateway control server 74 keeps the request and keeps its connection .. The gateway control server 74 receives the request for the gateway 84. For example , The gateway control server 74 is installed for the gateway 84 as shown in FIG. Console request or operation request from each network administrator process 90 Can be received via Rothes 68. Gateway control server 74 pairs to gateway 84 After receiving the request to make the gateway control server 74, the gateway long poll Send a response to the request; the response is a request to gateway 84 (eg, set up) Includes fixed request or operation request). In some embodiments, as an alternative, a gateway system Your server 74 gates the received request without responding to the long poll request. A gateway on an established connection to the gateway maintained by the way control server Can be sent to B84.
0158[Block storage I / O operation on the storage gateway] The embodiment of the storage gateway is a cache gateway or a cache gateway as described above. It can be realized as a shadowing gateway. In the embodiment, the cash gate Ay has on-premises (local) storage for the most frequently accessed data And also the remote provided by the storage service for essentially infinite total capacity It can be thought of as an on-premises block-based appliance that utilizes storage. Figure 6 shows an example of a network environment in which one embodiment of a cache gateway is realized. It is a high-level block diagram which roughly shows the texture and the data flow in it. Ki The hash gateway is the local network of service customers and service providers. Can act as an interface to and from storage services in your network. In at least some embodiments, the cache gateway is an iSCSI interface. The device can be exposed to processes on the customer network, but in some embodiments, other devices The interface can be exposed. Therefore, the cache gateway is a client Data interface target operating within the network (for example, iSCSI server) Can look like (get), for example, a cache gateway is a client network It can appear as a storage array on the screen. The cache gateway is, for example, a logical uni Logical number (LUN), for example, block-based storage devices such as hard disks It can be exposed to processes running on devices in the Iant network. The process then , Start a data session with the LUN (eg SCSI session) and data command (For example, SCSI commands) can be sent to the cache gateway.
0159FIG. 24 shows for a cache gateway according to at least some embodiments. The overall architecture and its data I / O behavior are shown. Generally, cash gateway In the 800, when the write data is received from the customer process 830, the data is written. Only added to log 814; that data is later written by the upload process. Uploaded from 814 to remote data store 820. Books related to blocks Metadata for inclusive data (eg block position, block type, offset (eg) The singular or plural) and the length) may be added to the metadata store 806. at least In some embodiments, the metadata store 806 is a database, eg, Berk. It can be realized as an ely database (BDB). Cache gateway 800 At least some data (eg, frequently and / or recently used data) Can also be cached locally in the local cache 812, it has some reads But satisfied from local cache 812 instead of from remote data store 820 This can improve the response to the customer's read request. Local cache 812 , Can also be called read cache. Metadata store 806 caches locally The location and other information about the read data that was read may also be included in the local cache 812. To. Figure 24 shows one metadata store 806 read cache entry and write It shows an embodiment that includes both cache entries, but in some embodiments it is read. Metadata store 80 with separate take cache entry and write cache entry Can be maintained within 6. In at least some embodiments, the device from customer process 830 Is the data read request, if possible, write log 814 or local cache 812? Can be provided; otherwise, the requested data is from remote data store 830 Can be fetched. Fetch and (eg blockback) to satisfy read requests Local cache 812 or remote data buffered (to fa 804) Data from store 830 has an update in write log 814 for that data If that data is returned to customer process 830 to satisfy the read request, Can be updated with data from write log 814.
0160In at least some embodiments, write log 814 and data cache 812 Both can be implemented within a common local block-based data store 810. Bro The datastore 810 is implemented in volatile memory, non-volatile memory, or a combination thereof. It can be. Block data store 810 is implemented on top of cache gateway 800 The cache gateway 800 is mounted on the physical memory in the physical device to be installed. On external memory to the physical device (eg 1 assigned by the customer to gateway 800) It can be implemented on one or more storage devices), or a combination thereof.
0161Both write log data and cached read data are block data Toa 810 in block storage format, for example 4MB (4MB) Can be stored as a lock. Cached reads in block data store 810 The block can be considered a read cache and is a write log block in the block data store. The lock can be considered a write buffer. Metadata store 806 is block data Both read cache 812 blocks and write log 814 blocks in the tor 810 May include an entry to find. The block reads to satisfy the read request Can be read from cache 812 (or from write log 814) and blocks Upload process from write log 814 to remote data store 820 by load process Can be played. In at least some embodiments, write block write log 814 When uploading from, the uploaded data will be a new read block Can be added to read cache 812. Uploaded write log 814 blocks Can be marked as "free" in the block data store 810 and block the changes Metadata store 806 can be updated appropriately to reflect on data store 810 ..
0162In at least some embodiments, the write request is a relatively small part of the block. Only can be modified or changed. Therefore, in at least some embodiments, the block When uploading from write log 814, for example, as described above, data deduplication technique Only the changed parts can be uploaded to the remote data store 820 using the technique To. In addition, the write log 814 is one stored in different write log 814 blocks. Or it can contain multiple overlapping writes (ie, writes to the same logical block). When uploading write data from write log 814, two or more overlapping writes Can only be combined for upload. This join is outside the data store, for example, Can be executed within a block in block buffer 804; block itself in write log 814 The body does not change.
0163As mentioned above, in at least some embodiments, write blocks are written to log 81. When uploading from 4, the uploaded data will be a new read block Can be added to the read cache 812. At least in some cases, for example, write If the block contains a large number of changes and / or a large part of the write block is changed If so, the write block is simply a read cache 8 as a new read block. It is copied to 12 and the metadata store 806 is updated. However, as mentioned above, the book The include request may modify or change only a relatively small portion of the write log 814 block. So, at least in some cases, each corresponding block is the first remote day. It can be fetched from the tastore 820, and the fetched block is the read cache. Read the block to ensure that the entire block in the 812 is up-to-date. Updated with changes from read log 814 before adding to hash 812. As mentioned In addition, the write log 814 is a stack of two or more stored in different write log 814 blocks. It can contain companion writes (ie, writes to the same logical block), and therefore The blocks can be updated according to one or more write log 814 blocks. Small In at least some embodiments, the fetched block is read cache 812. Block buffer 8 for updates from write log 804 blocks before being added to Can be stored in 04.
0164In general, new writes are previously freed writes in block datastore 810. Although stored in log 814 blocks; block datastore 810 is full or One or more cached reads if detected as nearly full Blocks can be purged to make room for write data. Read Bro For other reasons, for example, to free up space for new read data. Note that it can be purged from block datastore 810. In various embodiments To purge read blocks from block datastore 810, up to different techniques Or policies can be used. For example, in some embodiments, the oldest read block Minimum frequency of use (LRU) policy to purge data from block datastore 810 -Can be applied.
0165In at least some embodiments, the cache gateway 800 is a remote day. It may provide an interface to two or more volumes 822 on the Tastore 820. Less Also in some embodiments, there is a separate write log 814 and read cache 812. , For each volume 822, can be maintained by cache gateway 800. In at least some embodiments, separate writes to two or more volumes 822 Log 814 and read cache 812 are real in the same block datastore 810 Can be revealed. However, in at least some embodiments, for different volumes 822 Write log 814 and read cache 812 to block datastore 810 Can be logically or physically separated. Moreover, in at least some embodiments, it is different. Multiple metadata stores 806 can be maintained for separate volume 822.
0166In Figure 24, the read cache 812 and write log 814 are blocked in the data store 8. Shown as logically separate within 10, but in at least some embodiments, Read and write log blocks for a given volume 822 are blocks Can be physically mixed within data store 810. For example, the first physical block is a read block It can be a lock, the second to fifth physical blocks can be write blocks, and the following two physics The block can be a read block, and so on.
0167As mentioned above, FIG. 24 shows a cache gate according to at least some embodiments. The overall architecture for the way and its data I / O behavior is shown. But the streak The page gateway can also be used as a shadowing gateway, for example, as shown in Figure 7. Can be set. FIG. 25 shows a shadowing game according to at least some embodiments. The overall architecture for the toway and its data I / O behavior is shown. Shadow in The gateway 801 is illustrated and described for the cache gateway 800 in FIG. It may include the same architecture, components, and data I / O behavior as described, but The monitoring gateway 801 is a metadata store for the read cache 812. Does not contain read cache 812 or entry in 806 and also cache gateway Except that the read-related operations described above are not performed on the way. Shadowing gate Write operations on the way, except that the write is not added to the read cache. It can be similar to that for a cache gateway. In addition, the customer process (up to singular) Or multiple) Read and write requests from 830 forwarded to local data store 840 Will be done. However, the write data from the write request will be sent to the remote data store 820. Doing. In at least some embodiments, the write data is block day. The write data in the write log 814 is added to the write log 814 in the datastore 810. , Periodically or irregularly, uploaded to remote data store 820, but it Holds a snapshot 824 of the primary data store on the local data store 840 To do.
0168In at least some embodiments, for example, as shown in FIG. 24, the cash gate Against the hay and, for example, against the shadowing gateway, as shown in Figure 25. Thus, the write log 814 and write operation can be optimized with respect to write performance. Less And in some embodiments, at least some I / O operations on the gateway 800 , The block data store 810 can be used as a sequential data store. Especially the write log The 814 can be treated as a sequential data structure, and the write operations for the write log 814 are sequential. It can be realized as the next write operation. In at least some embodiments, write log 81 4 can be treated as a one-dimensional data buffer implemented as a linear or circular queue. About cache gateway, downloaded from remote data store 820 Data was sent from customer process (s) 830 to gateway 800 It can be stored in a read cache 812 separate from the write data, and the write data is in the write log. Stored in 814. Both cache gateway and shadowing gateway For write requests, write requests from the customer process (s) 830 in any order. That is, write requests can be received (which can be unordered or non-sequential), Unordered write required received from customer process (s) 830 The write data indicated by the request can be of any size and in the data store of interest. Can be directed to any position or offset of. But unordered write required Any write data received from the customer process (s) 830 in the request is written It is continuously written and added to the built-in log 814. In at least some embodiments Additions can be made at the subblock level; that is, two or more inputs of write data. A chest of drawers can be added within the same block in the write log 814. For write log 814 Metadata for updates (for example, write data in a block of write log 814) Fsets and lengths, as well as offsets in the data store of interest) are metadata Stored in store 806.
0169FIG. 26 shows a write on a block data store according to at least some embodiments. It is a flow chart of the method for writing to a log. Write log 814 as a sequential data structure For example, as a one-dimensional queue, the I / O handler 802 can realize the customer process. Block data store of arbitrary write data received from (s) 830 It may be feasible to write sequentially to the 810. One or more, as shown in 850 Write request can be received from customer process 830. Write requests can be made in any order That is, write requests can be received in an unordered manner and can be received by the customer process (single or unordered). Multiple) The write data indicated by the write request received from the 830 can be of any size. It can be directed to any position or offset in the data store of interest. 8 Block any write data as shown in 52 Write log on datastore 810 8 Sequential writes can be performed to write to 14 consecutively. Block as shown in 854 The data in the sequential write to data store 810 is adjacent in the block data store 810. Position (eg sector) to implement, eg block data store 810 Can be written to adjacent locations (eg, sectors) on storage. Adjacent positions are Note that it can be in the same write log block, but not necessarily. I want to be. The use of sequential writes to storage is random on the underlying storage. The need to perform a ct seek can be reduced or eliminated. Random sector seek Has a negative effect on I / O operation. For example, disk I / O -Put requires random sector seek by using continuous write It can be increased by 10 to 100 times when compared with non-sequential and discontinuous writing. To 856 As shown, the metadata store 806 is intended to reflect writes to write log 814. , Can be updated appropriately. Metadata for writes, at least in some embodiments Can be added to metadata store 806 in succession, but that is because the metadata is metadata Data in write log 814 than if added more randomly to Taster 806 Read metadata store 806 by processes that need more efficient access to Can be made possible.
0170In at least some embodiments, all write log 814 data is blocked. It is not always possible to write to adjacent locations within store 810. For example, two There can be 812 read cache blocks between 814 blocks of write logs. Therefore, 8 In 54, the embodiment writes as much write log 814 data as possible to adjacent locations. Can try to get in, but marked as being used in some positions (eg blocks) If so, it may be necessary to skip that position. Metadata store 806 Write log 814 data, even if the data is not stored in adjacent blocks It will be updated appropriately so that you can find it.
0171As mentioned above, logically, arbitrary write data is added to the end of the write data. To achieve this, in at least some embodiments, the block buffer 804 , A block of the same size as used in write log 814 (eg 4MB block) Is secured at. It is added until the allocated buffer block is full. another A buffer block can be allocated to append new write data; full buffer The fablock asynchronously and continuously to the write log 814 on the block data store. Can be rushed. The full block in write log 814 is the upload interface Uploaded to remote data store 820 asynchronously and continuously by Ace Get; Blocks uploaded from Write Log 814 are marked as "empty" obtain.
0172In the cache gateway embodiment shown in FIG. 24, in order to maintain data integrity, The read data is before the gateway 800 returns the requested data to customer process 830. May need to be integrated with the write data. Figure 27 shows a small number of cache gateways. It is a flow chart of a method for satisfying a read request according to at least some embodiments. .. A read request is received from customer process 830, as shown in 860. At least yes In some embodiments, when a read request is received from customer process 830, the gateway The 800 examines the data range of the read in the metadata store 806 and overlaps the read range. Determine if there is data in the competing write log 814. 862 in Figure 27, heavy If matching data is found in write log 814 that completely covers the read range, 8 As shown in 64, the data from write log 814 directly satisfies the read request. Can be used. Otherwise, in 866 in Figure 27, the overlapping data has a read range. If found in the partially covered write log 814, then the data is as shown in 868. The read cache 812 may be checked for existence for the data range of. If the data is in read cache 812, one or more, as shown in 870 Data blocks (s) can be fetched from the read cache 812. So Otherwise, one or more blocks are remote data stores, as shown in 872. Can be fetched from 820. In some embodiments, the block has some reads From both the read cache and the remote data store 820 to satisfy the request Note that it can be fetched. In 874 of Figure 27, the fetched data block Can then be updated with modified data from write log 814. 87 in Figure 27 In 6, the modified data is requesting process 830 to satisfy the read request Can be returned to. In some embodiments, the updated bro is shown in 878 of Figure 27. Can be added to the read cache 812.
0173In some embodiments, the remote data store 820 to satisfy the read request? The block read from is added to the read cache 812 and is requesting a block. Can be updated from write log 814 before sending to process 830. Alternatively, Bro The cook is buffered in, for example, block buffer 804 and updated in that buffer. Can be done. The updated block then comes from buffer 804, the process requesting it. It can be sent to 830 and added from buffer 804 to read cache 814.
0174In some embodiments, the read cache used to satisfy the read request 8 The blocks in 12 are updated in place with the data from write log 814, and then To process 830 requesting from read cache 812 to satisfy read request Can be sent. Alternatively, the block is read from the read cache 812, for example. For example, it can be buffered in block buffer 804 and updated in that buffer. Further The new block is then sent from buffer 804 to the requesting process 830. Can be added from buffer 804 to read cache 814. Read in buffer Previous version blocks in the read cache 812 taken are marked as free It can be kicked and / or overwritten with a newly updated block.
0175In 866 of Figure 27, if there is no overlapping data in the write log 814, 8 of Figure 27 As shown in 80, the read cache 812 is whether the read request is the read cache 812. You can check if you are satisfied. In 880 in Figure 27, the read request is read If satisfied from Shu 812, read cache 81, as shown in 882 in Figure 27. Data from 2 can be returned to customer process 830 to satisfy the read request. Figure 2 At 7 880, if the read request is not satisfied from the read cache 812, Figure 27 One or more data blocks (s) are remote, as shown in 884. Can be fetched from data store 820. Fetched as shown in 886 of Figure 27 Data from the block can be returned to customer process 830 to satisfy the read request To. In some embodiments, to satisfy the read request, as shown in 888 of Figure 27. Blocks fetched from remote data store 820 to read cache 812 Can be added to.
0176In at least some embodiments, the gateway 800 is a customer acquired and remo A snapshot of the write log 814 uploaded to the datastore 820, For example, it can be requested through a console process provided by the service provider Can be done. In addition, or instead, Gateway 800 has regular or non-regular Automatically take a snapshot of write log 814 to remote data store 8 Can be uploaded to 20. Uploading a snapshot of write log 814 Can provide protection for data, for example, from hardware and software failures. .. In at least some embodiments, the snapshot is a snapshot at a point in time. And the modified data in the write log at the time the snapshot was requested. Only data is uploaded in the snapshot. In at least some embodiments Will be uploaded with modified data about the cache gateway embodiment If the data is downloaded from the remote data store 820 for future reading The locally stored read cache 812 is also uploaded so that it does not need to be It can be updated with at least some of the data being done. The changed data is remote data Data and metadata in write log 814 after being uploaded to store 820 The corresponding data in Tor 806 can be discarded (for example, marked as "empty") and its Space can be reused.
0177[Combine write data for uploading to remote data store] As mentioned earlier, write log blocks can be used on remote data strikes, either periodically or irregularly. Can be uploaded to a. In at least some embodiments, write log blocks Data deduplication techniques can be used when uploading. However, the data duplication described Exclusion techniques are in blocks (s) that are staged to be uploaded Works during the upload process for any data. Customer process (singular or also Any writes from (plural) are added to the write log in sequence, and the customer process (singular) Or more than one) can be written to the same position in the target data store multiple times, so write b A block or blocks can be in the same location on the target data store (eg, off). Can include more than one write directed to a set and / or range).
0178Therefore, at least some embodiments are for write data in write log blocks. And implement pre-upload coalescing technology Can be. With this technology, write log blocks that are staged for upload ( Or metadata for multiple blocks) in the same location in the target data store Whether there are two or more writes in the write log block (s) that are directed Can be inspected to determine if. If there are multiple writes for a given position, Suppressed previous writes (s) when constructing a buffer block to be loaded Can be done. In this way, the bro that is passed to the upload process for upload For example, the pre-upload join technology is applied according to the data deduplication technology. To a given position, perhaps not to write more than one to the same position, which could otherwise exist Can contain only one write (latest write) of.
0179[Illustrated system] In at least some embodiments, one or more storages described herein. A computer system that implements some or all of the gateway technology is shown in Figure 28. One or more computers accessible, such as computer system 3000 shown A general purpose computer system that contains or is configured to access a functional medium May include. In the illustrated embodiment, the computer system 3000 is input / output (I). / O) One or coupled to system memory 3020 via interface 3030 Includes multiple processors 3010. The computer system 3000 is an input / output interface. It further includes a network interface 3040 coupled to the device 3030.
0180In various embodiments, the computer system 3000 has one processor 3010. Including a single processor system, or some (eg 2, 4, 8, or another suitable Can be a multiprocessor system that includes processor 3010. Processor 30 10 can be any suitable processor capable of executing instructions. For example, various embodiments So the processor 3010 is x86, PowerPC, SPARC, or MIPS Any various instruction set architecture, such as ISA, or any other suitable ISA It can be a general purpose or embedded processor that implements ISA. Multiprocessor system In the system, each of the processors 3010 can generally implement the same ISA, but not necessarily. That is not needed.
0181System memory 3020 is accessed by processor (s) 3010 It may be configured to store possible instructions and data. In various embodiments, the system Memory 320 is a static random access memory (SRAM), synchronous die. Namic RAM (SDRAM), non-volatile / flash memory, or any other tie It can be implemented using any suitable memory technology, such as memory. Illustrated Embodiment Now, for storage gateway technology, such as the methods, technologies, and data mentioned above. , Program instructions and data that provide one or more desired functions. It is stored as code 3025 and data 3026 in Mori 3020.
0182In one embodiment, the input / output interface 3030 is a network interface 30. Processor 3010, system memory 302, including 40 or other peripheral interfaces Configured to coordinate I / O traffic between 0 and any peripherals in the device obtain. In some embodiments, the input / output interface 3030 is one component (eg, For example, the data signal from system memory 3020) is taken from another component (eg, a processor). Any required protocol, to convert to a format suitable for use by 3010) Timing, or other data conversions can be performed. In some embodiments, I / O in The surface 3030 is, for example, PCI (Peripheral Component). Interconnect) A variant of the bus standard or Universal Serial Bus (USB) Support for equipment installed through buses for various types of peripherals, such as standards Can include In some embodiments, the functionality of the I / O interface 3030 is like For example, it is split into two or more separate components, such as Northbridge and Southbridge. obtain. Also, in some embodiments, an interface to system memory 3020, etc. Some or all of the functions of the input / output interface 3030 are directly connected to the processor 3010. Can be contacted.
0183The network interface 3040 is a computer system 3000 and a network. A device or other device 3060 connected to multiple networks 3050 (eg, the present specification). With other computer systems or devices, as shown in the other figures in the document) It can be configured so that data can be exchanged between them. In various embodiments, network in The surface 3040 is, for example, the type of Ethernet (registered trademark) network. Supports communication over any suitable wired or wireless common data network Can be. In addition, the network interface 3040 also has an analog voice network. Fiber via telecommunications / telephone networks such as digital fiber communication networks -Via a storage area network such as a channel SAN, or any other suitable May support communication over open networks and / or protocols.
0184In some embodiments, the system memory 3020 is of storage gateway technology. Store the program instructions and data described above in connection with other figures for the implementation of the embodiment. It may be an embodiment of a computer accessible medium configured to be such. But others In an embodiment of, program instructions and / or data are of different types of computers. It can be received, transmitted, or stored on accessible media. Generally speaking, compute The accessible medium is via a magnetic or optical medium, such as the I / O interface 3030. And the disc or DVD / CD, etc., combined with the computer system 3000, etc. It may include a persistent storage medium or a memory medium. Persistent computer-accessible storage medium Is in some embodiments of computer system 3000, up to system memory 3020. RAM (eg SDRAM, DDR SD) that can be included as another type of memory Any volatile or non-volatile medium such as RAM, RDRAM, SRAM, etc.), ROM, etc. Can also be included. In addition, the computer accessible medium is network interface 3 Communication such as network and / or wireless links that can be achieved via 040 A transmission medium or signal, such as an electrical, electromagnetic, or digital signal transmitted through a medium. Can include issues.
0185[Conclusion] Various embodiments are implemented according to the aforementioned description of computer accessible media. Instructions and / or data reception, transmission or storage may be further included. Generally speaking Computer-accessible media include, for example, magnetic or optical media (eg, disks as well. DVD / CD-ROM), RAM (eg SDRAM, DDR, RDRAM, S RAM, etc.), volatile or non-volatile media such as ROM, storage media or memory Communicated via media and communication media such as networks and / or wireless links Can include transmission media or signals, such as electrical, electromagnetic, or digital signals.
0186The various methods shown in the figures and described herein represent exemplary embodiments of the methods. Those methods may be realized by software, hardware, or a combination thereof. The order of the methods can be changed, and various elements can be added, reordered, combined, omitted, modified, etc. , Can be.
0187Various modifications and changes may be made, as will be apparent to those skilled in the art who will benefit from this disclosure. To. It is intended to include all such modifications and changes, and therefore the above description is intended. , Rather than a restrictive meaning, it is considered an exemplary meaning.
0188Various embodiments can be described with the following appendices in mind: Appendix 1. Remote data by storage gateway on customer network To the service provider who provides the tor to the customer of the service provider, its storage game To start the process of registering a toway, The storage gateway makes the storage gateway a customer network Receive configuration information that specifies it to function as a shadowing gateway on the screen And the shadowing gateway is a local data strike in the customer network A. Shadow the data stored on it to a remote data store to create local data. Receiving configuration information that the store acts as the primary data store for that data When, Shadow-in storage gateway on customer network in response to configuration information Setting as a gateway and By storage gateway from one or more processes on the customer network To receive read and write requests destined for the primary data store, For read requests, passing the read request to the primary data store, About write request: Pass the write request to the primary data store and Update the snapshot of the primary data store on the remote data store with write data Send the write data indicated by the write request to the service provider for new And How to include.
0189Appendix 2. Create a snapshot of the primary data store that matches the primary data store To service at least some of the data stored on the local data store The storage gateway opens the bootstrap process to upload to the lobbyer The method according to Appendix 1, further comprising starting.
0190Appendix 3. Receiving read and write requests, read and write requests By passing to the primary data store and by requesting a write to the service provider It is possible to terminate the indicated write data by boots by the storage gateway. The method described in Appendix 2, which is executed at the same time as the trap process.
0191Appendix 4. The above-mentioned transmission of the write data indicated by the write request to the service provider. When the gateway process enters pass-through mode, where communication is interrupted for a period of time, When the gateway process exits pass-through mode, After exiting the pass-through mode, the primary data store matches the primary data store. The gateway process is on the local data store to create a napshot Uploading at least part of the stored data to your service provider The method according to Appendix 1, further comprising.
0192Appendix 5. One or more processes on the customer network are on the customer network Read for one or more data ports exposed by the storage gateway The method according to Appendix 1, which initiates a take request and a write request.
0193Appendix 6. Sending the write data to the service provider is a read request and And passing write requests to the local data store, as well as read and write requests. Returning the response from the local data store to the request to one or more processes The method described in Appendix 1, which is performed asynchronously.
0194Appendix 7. The local data store contains one or more storage devices, and the method described above is 1. Remo at least some data on at least one of one or more storage devices Further includes restoring from a snapshot of the primary data store on the data store , The method described in Appendix 1.
0195Appendix 8. Send the write data indicated by the write request to the service provider. To do Buffering write data into the write log and Write buffered by the storage gateway upload component Uploading only data to the corresponding upload component of the service provider And The method according to Appendix 1, including.
0196Appendix 9. The remote data store depends on the storage service of the service provider. Before the write data provided by the write request and indicated by the write request to the service provider You can send the service according to the interface to the storage service. The method according to Appendix 1, wherein the data is transmitted by the data.
0197Appendix 10. With at least one processor A memory that contains program instructions, and the program instructions are Local on the customer network from one or more processes on the customer network Receiving read and write requests destined for the data store Passing the read and write requests to the local data store, Local datastore maintained on the remote datastore by the service provider Write indicated by write request to update the snapshot of Sending data to the service provider At least one pro to achieve a gateway process that can work to do With memory, which can be executed by Sessa A device equipped with.
0198Appendix 11. A snapshot of the local data store that matches the local data store. The gateway process was stored on the local data store to create the data Can act to upload at least part of the data to the service provider , The device according to Appendix 10.
0199Appendix 12. One or more processes on the customer network are on the customer network For one or more data ports exposed by the storage gateway of The device of Appendix 10 that initiates read and write requests.
0200Appendix 13. The gateway process sends the write data to the service provider. Passing read and write requests to the local data store to send, And start responding to read and write requests from the local data store The device according to Appendix 10, which is capable of operating to be returned to the process and run asynchronously.
0201Appendix 14. The gateway process services the gateway process. The device according to Appendix 10, which can be further operated to initiate the process of registering with the data. ..
0202Appendix 15. The gateway process follows the configuration information received from the storage service. It can even operate to be configured as a shadowing gateway, Remote data from the data stored on the local data store by the doing gateway The device according to Appendix 14, which shadows the store.
0203Appendix 16. The gateway process has at least some of the data in the local data store. Data can be further actuated to restore from a snapshot of the local data store The device according to Appendix 10.
0204Appendix 17. Send the write data indicated by the write request to the service provider Because the gateway process, Buffering the write data in the write log and Upload the buffered write data from the write log to the service provider To load The device according to Appendix 10, which is further operable to do so.
0205Appendix 18. The upload is performed in accordance with data deduplication technology, The device according to Appendix 17.
0206Appendix 19. Remote data store depends on the storage service of the service provider And gated to send write data to its service provider The way process points the write data through the interface of the storage service. The device according to Appendix 10, which is further capable of operating to transmit in a defined format. ..
0207Appendix 20. A snapshot of the local data store is on the remote data store. Maintained in block format by the service provider and by write request The gateway process sends the indicated write data to the service provider. Upload a block of data modified by write data to the service provider The device according to Appendix 10, which is further operable to perform.
0208Appendix 21. Uploading the block is actually in accordance with data deduplication technology The device according to Appendix 20, which is carried out.
0209Appendix 22. Computer-executable program life that realizes the gateway process A persistent computer-accessible storage medium for storing orders and its gateway The process is: Customer network computer according to configuration information received from service provider It is to be set as a shadowing gateway on the screen, and it is a shadowing game. Tway uses a service provider to store data stored on a local data store To shadow to a remote data store that is maintained Can operate to be configured as a gateway and Shadowing data stored on the local data store to the remote data store To make the gateway process Directed from one or more processes on the customer network to the local data store To receive a write request Local data strike maintained by the service provider on the remote data store Indicated by the write request to update the snapshot with write data To upload the write data to the service provider Can work to do, Persistent computer-accessible storage medium.
0210Appendix 23. Following the above settings, local, matching the local data store The gateway process is local to take a snapshot of the data store Upload at least part of the data stored on the data store to your service provider Sustainable computer-accessible memory according to Appendix 22, which is operational to Medium.
0211Appendix 24. The gateway process makes a read request directed to the local data store. 22. The persistence computer described in Appendix 22, which is further operational to receive and process. Data accessible storage medium.
0212Appendix 25. One or more processes on the customer network are on the customer network For one or more data ports exposed by the storage gateway of The persistent computer access described in Appendix 24, which initiates read and write requests. Possible storage medium.
0213Appendix 26. The gateway process now passes write requests to the local data store. Further operational, the gateway process writes data to the service provider To upload, to pass a write request to a local data store, And can be operated to execute the read request asynchronously with the processing. Persistent computer-accessible storage medium as described in 24.
0214Appendix 27. Upload the write data indicated by the write request to the service provider To load, the gateway process, Buffering the write data in the write log and Is the buffered write data a write log according to data deduplication technology? To upload to a service provider Can work further to do, The persistent computer-accessible storage medium according to Appendix 22.
0215Appendix 28. Gateau to upload write data to the service provider The way process writes according to the service provider's web service interface Sustainability controller described in Appendix 22, which can be further actuated to upload data. Puta-accessible storage medium.
0216Appendix 29. Gateau to upload write data to the service provider The way process provides a block of data modified by the write data. Sustainable computer described in Appendix 22, which can be further actuated to upload to data. Data accessible storage medium.
0217Appendix 30. The upload is performed in accordance with data deduplication technology, Sustainable computer-accessible storage medium according to Appendix 29.
34 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17 Sheet 18 Sheet 19 Sheet 20 Sheet 21 Sheet 22 Sheet 23 Sheet 24 Sheet 25 Sheet 26 Sheet 27 Sheet 28 Sheet 29 Sheet 30 Sheet 31 Sheet 32 Sheet 33 Sheet 34
Every citation, both ways
| Document | Relation | Office |
|---|---|---|
| JP2002324000A | Cites | Japan |
17 members in 6 offices
Members17
| Document | Office | Kind | |
|---|---|---|---|
| CA2840596A1 | Canada | A1 | |
| US2013007219A1 | United States of America | A1 | |
| WO2013003713A2 | World Intellectual Property Organization (WIPO) | A2 | |
| WO2013003713A3 | World Intellectual Property Organization (WIPO) | A3 | |
| EP2727001A2 | European Patent Office (EPO) | A2 | |
| CN103930879A | China | A | |
| JP2014529111A | Japan | A | |
| EP2727001A4 | European Patent Office (EPO) | A4 | |
| US9294564B2 | United States of America | B2 | |
| JP2016129036A | Japan | A | |
| US2016205187A1 | United States of America | A1 | |
| JP6073878B2 | Japan | B2 | |
| CA2840596C | Canada | C | |
| JP6139718B2This record | Japan | B2 | |
| CN103930879B | China | B | |
| US10536520B2 | United States of America | B2 | |
| EP2727001B1 | European Patent Office (EPO) | B1 |
11 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD |
Numbers
- Publication
- 6139718
- Application
- 22137
Titles2
- Japanese
- シャドーイングストレージゲートウェイ
- English
- Shadowing storage gateway
Classification
- CPC, 9
- H04L67/1095
- G06F11/1464
- H04L67/1097
- G06F16/128
- G06F16/1724
- H04L67/56
- H04L67/568
- H04L67/02
- G06F9/44
- IPC, 3
- G06F12 00
- G06F3 06
- G06F13 10
