System and method for restricting data transfers and managing software components of distributed computers

The present invention relates to the management of a computer system. More specifically, the present invention relates to systems, methods, computer-readable media, and computer-readable memory for limiting data transfer and managing software components of distributed computers.

The Internet and its use have expanded significantly in recent years and are expected to continue. One of the most prominent uses of the Internet is the World Wide Web WWW, also known as the "Web", which is a collection of documents that users can see or provide. Yes (called a "web page"), which typically contains links to one or more other pages that the user can access. Many companies and individuals create a presence on the web, describe themselves or themselves on one or more web pages, describe their products and services, identify other target information, and products. It is typical to make it possible to purchase services and services.

Web pages are typically made available on the Web through one or more Web servers, a process also called "hosting" Web pages. These web pages may be freely available to anyone who requests them to view them (for example, company advertisements), or when access to the web pages is restricted. Yes (for example, a password may be required to access a web page). Given the large number of people requesting to view a web page (especially given that the web is globally accessible), a large number of servers are sufficient to host the web page. (For example, hosting the same web page on multiple servers will increase the number of people who can access the web page at the same time). In addition, because the Web is geographically dispersed and access is inconsistent, it is often desirable to distribute servers to different remote locations to minimize access times for people in different locations around the world. ing. In addition, people tend to view web pages all day long (again, especially given that the web is globally accessible), so the server that hosts the web page is 24 hours a day, 24 hours a day. It needs to be kept functional.

However, managing a large number of servers can be difficult. A highly reliable power supply is required to keep the server up and running. Physical security is needed to prevent theft and other malicious attempts to corrupt or sneak into the server. A reliable internet connection is required to ensure that access requests reach the server. The correct operating environment (eg temperature, humidity, etc.) is required to ensure that the server operates properly. Therefore, "co-location facilities" have been developed to help businesses deal with these problems.

Here, the joint location facility is a complex that can accommodate a plurality of servers. Common location facilities typically provide a reliable internet connection, a reliable power source, and an appropriate operating environment. Co-location facilities also typically include multiple secure areas (eg, cages) that allow different companies to keep their servers there. A collection of servers that a particular company keeps in a co-location facility is actually a "server cluster", even if only a single server is in any individual co-location facility. being called. In this case, the specific company will be in charge of managing the operations of the servers in its own server cluster.

<p> However, problems are also occurring in such joint location facilities. One issue is data security. Different companies (even competitors) can have server clusters in the same co-location facility. In such a situation, data received from the Internet (or sent from a server in a server cluster) for one company is routinely routed to another company's server located at that co-location facility. ) It is necessary to be careful not to be done.</p><p> Another issue is the issue of server management after being placed in a co-location facility. System administrators (system administrators) belonging to a company now contact the administrator of a co-location facility (typically by phone) in the event of a server failure (or other problem) and a particular server. It is possible to request the administrator to reset (typically pressing the hardware reset button on the server side or powering off the server and then powering it on). With this limited reset function, the management function that a company can obtain is very poor. Alternatively, system administrators belonging to the enterprise can go to the joint location facility on their own and witness the failed server. Unfortunately, having a system administrator go to a co-location facility and witness a server wastes a lot of time. Therefore, it would be convenient if there was an improved way to manage the server computer in the common location facility.</p><p> In addition, the number of individual user computers located around the world continues to grow, in the form of personal computers (PCs), personal digital assistant PDAs, pocket computers, and palm-sized (palm). -sized) Computers, handheld computers, digital cellular phones, etc. Managing software on these user computers can be a daunting and time-consuming task, and the users of these machines are often non-professional, making them particularly difficult. .. System administrators and technicians often go to the remote location where the user computer is located or walk-through management operations over the phone. It would be even more convenient to improve the management of remote computers at the user's location without the user's intervention.</p><p> The present invention has been made in view of such problems, and an object of the present invention is a system for limiting data transfer and managing software components of a distributed computer, which can improve the management of a computer system. And to provide a method.</p>

<p> The following describes limiting data transfer and managing software components in a cluster of server computers located in a co-location facility.</p><p> According to one aspect of the invention, the controller (named "BMonitor") is located on a computer (eg, each node located in a communal location facility). BMonitor includes multiple filters that identify where data can be sent and received, such as another node in a co-location facility or a client computer coupled to a computer over the Internet. It has become like. These filters can be modified by one or more management devices attached to the computer during the operation of the computer.</p><p> In another aspect, a controller named "BMonitor" (located on the computer) manages the software components that reside on the computer. When a request is received by BMonitor from an external source, the request is executed by BMonitor. This request can be made from the management console located in the same location as the computer or from the management console located away from the computer.</p><p> In another aspect, a controller named "BMonitor" (located on the computer) acts as a trusted third party that mediates the interaction between multiple management devices. I am doing. BMonitor is a multiple ownership domain Each domain is associated with a management device, and each domain has its own set of rights that indicate what type of management function can be instructed to run on BMonitor. .. Only one ownership domain is the top-level domain at any given time, and this top-level domain has a further extended set of rights than the lower-level domains. The top-level domain can create a new ownership domain that can be associated with other management devices, and can also remove the top-level domain and transfer the management rights of the corresponding management device to the lower-level ownership domain. It can be undone at any time by the corresponding management device. Every time the ownership domain, which is the top-level ownership domain, is changed, the computer's system memory can be erased so that sensitive information from one ownership domain is used by devices corresponding to other ownership domains. Is prevented.</p><p> In another aspect, BMonitor runs at a higher privilege level than other software engines running on the node, causing other software engines to interfere with the limits set by BMonitor. Is being prevented.</p><p> Hereinafter, the present invention will be described with reference to the accompanying drawings, but the present invention is not limited to the following description. Also, similar components and / or functions are shown using the same reference numerals throughout the drawing.</p>

<p> According to the present invention, according to the present invention, it is possible to efficiently manage a computer system by limiting data transfer and managing software components in a cluster of server computers located in a common location facility.</p>

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In each drawing, parts having the same function are designated by the same reference numerals, and duplication of description will be omitted.

FIG. 1 is a diagram showing a client / server network system and environment that can be used in some embodiments of the present invention. Generally, a system consists of one or more (n) client computers 1021 to 102n and a cluster of multiple server computers (server clusters) 1061-1, 1061-2, 106m-1, 106m-2, respectively. One or more (m) co-location facilities 1041 to 104 m, one or more management devices 110, and one or more co-location facilities (eg, not included in the co-location facility) Includes server 112 and. These servers, client computers, and management devices communicate with each other through the data communication network 108. The communication network of FIG. 1 includes a public network 108 such as the Internet. In addition to or as an alternative to the Internet, other types of communication networks can be used, including local area network LANs and wide area networks. WAN) etc. are included. The construction of the data communication network 108 can be performed in various ways, including wired and / or wireless communication media.

A wide range of communication protocols can be used for communication using network 108. In one implementation, client computers 1021 to 102n and server computers in clusters 1061-2, 106m-1, 106m-2 can communicate with each other using Hypertext Transfer Protocol HTTP. Web pages are hosted by server computers and written in markup languages such as HTML (Hypertext Markup Language) or XML (eXtensible Markup Language).

The management device 110 serves to manage the software components of one or more computing devices located away from the device 110. This management can also include limiting the transfer of data to and from the managed computing device. In the example shown in FIG. 1, the management device 110 remotely manages one or more of clients 1021 to 102n, server clusters 1061-1, 1061-2, 106m-1, 106m-2, or server 112. Is possible. A wide range of remotely manageable computing devices include personal computer PCs, networked PCs, multiprocessor systems, microprocessor-based or programmable home appliances, minicomputers, mainframe computers, and more. Gaming console, internet equipment, personal digital assistant Includes PDAs), pocket computers, palm-sized computers, handheld computers, digital cellular phones, and more. Remote management of computing devices is done by propagating commands to the devices over network 108, as described in detail below.

In the following description, embodiments of the present invention are described in the general sense of computer executable instructions executed by one or more conventional personal computers, such as program modules. In general, a program module contains routines, programs, objects, components, data structures, etc. that perform a particular task or implement a particular abstract data type. Further, as will be appreciated by those skilled in the art, various embodiments of the present invention can be implemented in other computer system configurations, including handheld devices, gaming consoles, and the Internet. Includes Internet appliances, multiprocessor systems, microprocessor-based or programmable home appliances, network PCs, minicomputers, mainframe computers, and more. In a distributed computer environment, program modules can be stored on both local and remote memory storage devices.

Alternatively, embodiments of the present invention can be implemented in hardware or in combination with hardware, software, and / or firmware. For example, the present invention makes it possible to implement all or part of it in one or more application specific integrated circuit ASICs or programmable logic device PLDs. ..

FIG. 2 is a schematic diagram showing an example of a computer 142 that can be used according to some embodiments of the present invention. Examples of the computer 142 shown are computers that function as client computers 1021 to 102n in FIG. 1, computers and nodes located in the joint location facilities 1041 to 104m in FIG. 1 and other locations (eg, below). There is a node 248) in Figure 5 described, or a local or remote management console, which are described in detail below.

The computer 142 includes a bus 148 that connects various system components, including one or more processors or processing units 144, system memory 146, and system memory 146 to processor 144. Bus 148 represents one or more of several types of bus structures, including memory buses or memory controllers, peripherals that employ any of the various bus architectures. Includes bus, high-speed graphics port, and processor or local bus. System memory includes read only memory ROM 150 and random access memory RAM 152. The basic input / output system BIOS 154 consists of basic routines that help transfer information between elements in computer 142, such as during startup, and are stored in ROM 150.

Computer 142 also reads and writes to and from a hard disk (not shown) and is connected to bus 148 via a hard disk drive interface 157 (eg, SCSI, ATA, or any other type of interface). , A magnetic disk drive 158 that reads and writes to and from the removable magnetic disk 160 and is connected to bus 148 via the magnetic disk drive interface 161 and a removable optical disk 164 such as a CD-ROM, DVD, or other optical media. It is equipped with an optical disk drive 162 that reads and writes to and from and is connected to bus 148 via the optical drive interface 165. These drives and their associated computer-readable media, as non-volatile storage, store computer-readable instructions, data structures, program modules and other data for the computer 142. In the example environment described here, a hard disk, a removable magnetic disk 160, and a removable optical disk 164 are adopted, but as those skilled in the art can understand, it is possible to store data accessible by a computer. Other types of computer-readable media that can be used can also be used in an exemplary operating environment, such as magnetic cassettes, flash memory cards, digital video disks, random access memory (RAM), read-only. There is memory (ROM) and so on.

Some program modules are hard disk, magnetic disk 160, optical disk 164, ROM 150 or RAM It can be stored in 152, including operating systems 170a, 170b, one or more application programs 172a, 172b, other program modules 174a, 174b, and program data 176a, 176b. It has been. The user can enter commands and information into the computer 142 through an input device, such as a keyboard 178 or a pointing device 180. Other input devices (not shown) include microphones, joysticks, gamepads, satellite dishes, scanners, and the like. The above and other input devices are connected to the processing unit 144 via interface 168, which is coupled to the system bus. Monitor 184 and other types of display devices are also connected to system bus 148 via an interface, such as video adapter 186. In addition to monitors, personal computers are typically equipped with other peripheral output devices (not shown), such as speakers and printers.

Computer 142 is optionally operating in a networking environment that uses a logical connection with one or more remote computers, such as remote computer 188. The remote computer 188 can be another personal computer, server, router, network PC, peer device or other common network node, and Figure 2 shows only the memory storage device 190. However, it typically contains many or all of the above-mentioned elements associated with the computer 142. The logical connections shown in Figure 2 include a local area network (LAN) 192 and a wide area network (WAN) 194. Such networking environments are widespread in offices, enterprise-wide computer networks, intranets, and the Internet. In an embodiment of the invention described herein, the remote computer 188 is a Microsoft Corporation (Redmond, Redmond, Inc.) You are running an Internet web browser program (which may optionally be built into your operating system), such as the "Internet Explorer" web browser manufactured and sold in Washington State.

When used in a LAN networking environment, computer 142 is connected to local network 192 through a network interface or adapter 196. When used in a WAN networking environment, the computer 142 is typically equipped with a modem 198 and other components to establish communication over a wide area network 194 such as the Internet. The modem 198 is available as an internal type or an external type, and both are connected to the system bus 148 via an interface (for example, a serial port interface 168). In a networking environment, the program modules described above or parts thereof in connection with the personal computer 142 can be stored in a remote memory storage device. As can be seen from the above, the illustrated network connections are exemplary, and other means of establishing communication links between computers can also be used.

Generally, the data processor of a computer 142 is programmed by instructions stored at different times in various computer-readable media of the computer. Programs and operating systems are typically distributed, for example, on floppy (registered trademark) disks or CD-ROMs. They are installed from there or loaded into the computer's secondary memory. At run time, these, at least in part, are the primary electronic memory of the computer. It is loaded in. According to the invention described herein, an instruction or program for performing the steps described below in association with a microprocessor or other data processor is a computer-readable storage medium of the above and other types. It is stored in. Further, according to the present invention, the computer itself is programmed according to the methods and methods described later. In addition, certain subcomponents of the computer are programmable to perform the functions and steps described below. According to the present invention, such subcomponents are programmed as described below. Further, according to the present invention described herein, the data structure is embodied on various types of memory media, as described below.

For convenience of explanation, other executable program components such as programs and operating systems are shown here as separate blocks, but as is well understood, such programs and components are timed. They are located in different storage components of the computer and are run by the computer's data processor.

FIG. 3 is a detailed block diagram showing an exemplary joint location facility. The joint location facility 104 includes a plurality of nodes 210-1 to 210-n (also referred to as server computers) as shown in the figure. The joint location facility 104 can accommodate any number of nodes, and can easily accommodate thousands of nodes.

Nodes are grouped into clusters called server clusters (or node clusters). To simplify the description and avoid drawing confusion, only one cluster 212 is shown in Figure 3. Each server cluster contains a node corresponding to a specific customer of the joint location facility 104. A node in one server cluster is physically isolated from a node in another server cluster. This physical isolation can be in a different form, such as a locked individual cage or individual room at the shared location facility 104. Physically isolating the server cluster ensures that customers at the joint location facility 104 are the only ones who have physical access to their nodes (and no other customers). Alternatively, server clusters can be logically isolated from each other rather than physically (eg, using the cluster boundaries described in detail below).

Landlord / tenant relationships (also known as lessor / lessee relationships) can also be configured on a node basis. The owner (and / or operator) of the joint location facility 104 owns (or has the right to) an individual node and can therefore be viewed as a "land road". The customer at the joint location facility 104 leases the node from Landlord and can therefore be seen as a "tenant". Landloads are generally indifferent to what type of data or program is stored on a node by a tenant, but they demarcate clusters so that nodes from different clusters communicate with each other. I try to prevent it. This is described in detail below. In addition, the use of a node provides the tenant with the assurance that the land load will not have access to sensitive information stored by the tenant, even if the node is leased only to the tenant.

Nodes in different clusters have the same transport medium (one or more) that allows access to network connections (s) 216 and, in some cases, application operations management console 242, even if they are physically isolated. Often physically attached to one or more) 211. This is described in detail below. This transport medium can be wired or wireless.

Since each node can be coupled to the shared transport medium 211, each node can be configured to limit which other nodes it can send and receive data to. If it is possible to include several different nodes in a tenant's server cluster, the tenant wants to be able to pass data between different clusters within that cluster for purposes such as processing and storage. In some cases. However, tenants generally do not want data to be passed to other nodes that do not belong to the server cluster. By configuring each node in the cluster to limit which other nodes it can send and receive data to, it is possible to set and apply server cluster boundaries. Setting and applying such server cluster boundaries prevents tenant data from being accidentally or illegally transferred to nodes that do not belong to the cluster.

These initial boundaries, set by the land load, prevent communication between nodes of different customers, ensuring that each customer's data is passed to that customer's other nodes. If the customer himself defines the sub-boundary separately in the cluster and sets the sub-cluster of the node, the data entering and exiting from the sub-boundary is prevented from being passed to and from other nodes in the cluster. Customers are free to add, modify, delete, and so on such subcluster boundaries, but only within the boundaries defined by the land load (that is, cluster boundaries). Limited to. Therefore, the customer cannot change the boundaries in such a way that the communication with the node is extended to another node that does not belong to the same cluster.

The joint location facility 104 provides a reliable power supply 214 and a reliable network connection (s) 216 (eg, network 108 in FIG. 1) for each of node nodes 210-1 to 210-n. are doing. Power supply 214 and network connection 216 are shared by all of node nodes 210-1 to 210-n, but otherwise to node nodes 210-1 to 210-n or node groups (eg clusters). It is also possible to provide power supply 214 and network connection 216. A wide range of conventional mechanisms for providing reliable power sources, all of which can be used in combination with backup generators, redundant generators, batteries, fuel cells, or other power storage mechanisms in the event of a power failure (power outage). It can be used to provide a reliable power supply 214, such as power supplied by a company. Similarly, a wide range of traditional mechanisms for providing reliable network connections are all redundant connection transport media, heterogeneous connection media, different access points (eg, different internet access points, different internet service providers). Can be used to provide network connection 216, such as Internet service provider ISP).

In some embodiments, node nodes 210-1 to 210-n are joined by the operator or owner of the joint location facility 104, along with the space and services of the facility 104 (eg, reliable power supply 214 and network connection 216). Leased or sold to the customer. In other embodiments, space and services at facility 104 are leased to the customer, while one or more nodes are provided by the customer.

Management of each node node 210-1 to 210-n is performed in a multi-layer system. FIG. 4 is a block diagram showing an exemplary multi-layer management architecture. This multi-tier architecture consists of three layers. That is, the cluster operation management layer 230, the application operation management layer 232, and the application development layer 234. Cluster Operations Management Layer 230 is implemented locally (for example, in a joint location facility) in the same location as the managed server (s) and is responsible for managing the server's hardware operations. There is. In the illustrated example, the cluster operations management layer 230 is irrelevant to which software components are running on node nodes 210-1 to 210-n, and is relevant to node nodes 210-1 to 210-n. All you have to do is continue n hardware operations and set boundaries between node clusters.

In contrast, application operations management layer 232 is located in a remote location (for example, separate from the co-location facility) where the managed server is located and is still attached to the server. It is implemented at the location of the client computer that is in communication. Application operations management layer 232 is responsible for managing server software operations and defining subboundaries within the server cluster. There are many ways to connect a client to a server, such as via the Internet or via a dedicated (eg dial-up) connection. The client can also remain attached to the server or can be attached indefinitely (eg, only when needed for administrative purposes).

Application Development Layer 234 is implemented on a different client computer that is located in a different location than the server (for example, a location different from the co-location facility) and is a software component or engine that runs on the server. Is in charge of developing. Alternatively, it is possible to develop additional software components and engines for the node by allowing remote clients to access the current software located on node nodes 210-1 to 210-n of the joint location facility 104. Is. The client on which the application development layer 234 is implemented is generally a different client from the client on which the application operation management layer 232 is implemented, but these two layers 232 and 234 are on the same client. It can also be implemented (at least partially).

Although only three layers are shown in Figure 4, apart from this, a multi-tier architecture can consist of different numbers of layers. For example, splitting the application operations management layer into two layers and giving each layer different (or overlapping) responsibilities results in a four-layer architecture. Management at these layers can be done from the same location (eg, sharing a single application management console) or from different locations (eg, two different operations management consoles).

Returning to FIG. 3, the joint location facility 104 includes a cluster operations management console for each server cluster. In the example of FIG. 3, the cluster operations management console 240 is associated with the cluster 212. The Cluster Operations Management Console 240 implements Cluster Operations Management Layer 230 (Figure 4) for Cluster 212 and is responsible for managing the hardware operations of the nodes in Cluster 212. The cluster operations management console 240 monitors the hardware in the cluster 212 and attempts to determine hardware failures. It is possible to monitor any hardware failure over a wide range, including processor failures, bus failures, memory failures, and so on. A way to monitor hardware operations is for the Cluster Operations Management Console 240 to send test messages or control signals to the node requesting the use of specific hardware to respond (no response or erroneous response is a failure). Produces test messages and control signals requesting the use of specific hardware and periodically sends them from the node to the Cluster Operations Management Console 240 (within a specific time period). Various methods are possible, such as (when no message or control signal is received, it indicates that a failure has occurred). Alternatively, the Cluster Operations Management Console 240 may try to determine only what type of hardware failure has occurred, rather than attempting to determine what type of hardware failure has occurred.

When a hardware failure is detected, the Cluster Operations Management Console 240 works to correct the failure. The actions taken by the Cluster Operations Management Console 240 may vary not only with the hardware, but also with the type of failure, and with different server clusters. Corrective actions include notifying the administrator (eg blinking lights, audio alarms, email messages, cellphone or pager calls, etc.) and attempting to physically correct the problem (eg rebooting the node). , Start another backup node in place of the failed node, etc.).

The cluster operations management console 240 also sets cluster boundaries within the joint location facility 104. Cluster boundaries set by console 240 prevent nodes 210-3 within one cluster (eg cluster 212) from communicating with another cluster (eg clusters that do not belong to cluster 212), on the other hand. At the same time, ensure that node 210-3 in the cluster does not interfere with its ability to communicate with other nodes in the cluster. According to these boundaries, the tenant can know that even if the network connection 216 is shared by the tenant, it cannot propagate its data to the nodes of other tenants located at facility 104. , The security of the tenant's data will be obtained.

In the illustrated example, each cluster in the joint location facility 104 includes a dedicated cluster operations management console. Alternatively, a single cluster operations management console can be associated with multiple server clusters to manage their hardware operations. According to another alternative method, it is possible to associate multiple cluster operations management consoles with a single server cluster to manage their hardware operations. These multiple consoles can manage a single server cluster in a shared manner, or one console can act as a backup for another console (for example, redundancy can improve reliability. , Maintenance etc. will be easier).

The application operations management console 242 is also coupled to the joint location facility 104 and is in communication. The application operations management console 242 is typically located in a location away from the co-location facility 104 (ie, typically in the customer's office rather than in the co-location facility 104). Although different application operations management consoles 242 can be associated with each server cluster in the joint location facility 104, another method is to associate multiple consoles 242 with a single server cluster or a single console with multiple server clusters. It is also possible to associate 242. Application Operations Management Console 242 implements Application Operations Management Layer 232 (Figure 4) for Cluster 212, which not only manages software operations within Cluster 212, but also secures subboundaries within Cluster 212. I am also in charge of that.

The application operations management console 242 monitors the software in cluster 212 and attempts to determine software failures. It is possible to monitor a wide range of software failures, such as "hanging" or unresponsive application processes or threads, application process or thread run-time errors, and so on. Software operations can be monitored in a variety of ways (similar to hardware operation monitoring described above), for example, a test message that the Application Operations Management Console 242 requires the use of a particular routine to respond. Or control signals to a specific process or thread running on the node (no response or false response indicates a failure). Alternatively, generate a message or control signal requesting the use of a specific software routine and periodically send it to the application operations management console 242 from a process or thread running on the node (identify such message or control signal). If you do not receive it within the time period, it indicates that a failure has occurred). Alternatively, the application operations management console 242 may attempt to determine only what type of software failure has occurred, rather than attempting to determine what type of software failure has occurred.

When a software failure is detected, the application operations management console 242 works to correct the failure. The actions taken by the Application Operations Management Console 242 can vary not only with the software, but also with the type of failure, and with different server clusters. Corrective actions include notifying the administrator (eg blinking lights, audio alarms, email messages, cellphone or pager calls, etc.), attempting to correct the problem (eg rebooting the node, software). There is a reload of the component or engine image, rerun after terminating the process, etc.).

Therefore, the management of nodes 210-1 to 210-n is distributed to a plurality of managers regardless of the number of other nodes (if any) located in the same location as the node. This multi-tiered management separates hardware operations management from application operations management, allowing management responsibilities for nodes to be shared between two different consoles, each under the control of a different entity. become.

FIG. 5 is a detailed block diagram showing an exemplary node managed remotely, according to some embodiments of the present invention. Node 248 can be a joint location facility node 210-1 to 210-n or a separate device (eg, client 102-1 to 102-n or server 112 in Figure 1). .. Node 248 contains a monitor 250 called "BMonitor" and multiple software components or engines 252 and is coupled to a mass storage device 262 (which may include a mass storage device). In the illustrated example, node 248 is a computing device with one or more processors that support multiple privilege levels (eg, rings on x86 architecture processors). In the illustrated example, these privilege levels are called rings, but different implementations that employ different processor architectures may use different terms. According to this multiple ring, a set of prioritized levels Is provided so that the software runs at that priority level. The priority level is 4 levels (ring 0, 1, 2, It is often 3). Ring 0 is typically called the most privileged ring. A software process that runs in ring 0 typically has more functions (eg, instructions) that it can access than a process that runs in a less privileged ring. In addition, the processor running in a particular ring cannot change the code or data in the higher priority ring. In the illustrated example, BMonitor The 250 runs on ring 0, while the engine 252 runs on ring 1 (or ring 2 and / or 3). Therefore, the code or data in BMonitor250 (running in ring 0) cannot be modified directly by engine 252 (running in ring 1). Rather, such changes need to be made by the engine 252 requesting the BMonitor250 to make the changes (eg, by sending a message to the BMonitor250 or by calling a function of the BMonitor250). Implementing BMonitor250 in ring 0 protects BMonitor250 from uncontrolled or malicious engines 252 that attempt to bypass the restrictions imposed by BMonitor250.

Alternatively, the BMonitor250 can be implemented in other ways that protect the BMonitor250 from uncontrolled or malicious engines 252. For example, node 248 can include multiple processors, one for running engine 252 and one for running BMonitor250. Having only BMonitor250 run on a processor other than the one on which Engine 252 is running makes it possible to effectively shield BMonitor250 from Engine 252.

BMonitor250 is the basic control module for node 248. That is, it controls both the network interface card and the memory manager (which may optionally be built-in). Controlling the network interface card (which may be separate from the BMonitor250 or built into the network interface card) allows the BMonitor250 to send and receive data to and from node 248. Can be controlled. Controlling the memory manager allows BMonitor250 to control the allocation of memory to engine 252 running on node 248, thus preventing uncontrolled or malicious engines from interfering with BMonitor250 operations. ..

Even though various aspects of node 248 may be under the control of BMonitor250 (eg, a network interface card), BMonitor250 does at least some of its functionality with the engine 252 running on node 248. I am making it available. The BMonitor250 is an interface that allows the engine 252 to request access to features from it, such as sending data to another node 248 or the Internet (eg, through controller 254, described in detail below). .. These requests can take many forms, such as sending a message, calling a function, and so on.

The BMonitor250 includes a controller 254, a network interface 256, one or more filters 258, one or more keys 259, and a BMonitor250 Control Protocol BMCP module 260. The network interface 256 is an interface connecting the node 248 and the network (for example, the network 108 in FIG. 1). Can (or cannot) send or receive data to or from any other node (and / or other source) or target (eg, coupled to Internet 108 in Figure 1) located at the co-location facility? ) Is determined by the filter 258. To determine these nodes or other sources / targets, a network address (for example, an Internet Protocol IP address), some kind of globally unique identifier. It can be done in a variety of ways, such as (ID), a locally unique identifier (eg, a numbering scheme owned or unique to the joint location facility 104).

Filter 258 can completely restrict access to the node (eg, cannot send or receive data to or from the node) or partially restrict access to the node. Partial access restrictions can take many forms. You can limit a node, for example, you can receive data from a node but not send it to a node (or vice versa). As another example, a node can be restricted so that only certain types of data (communication conforming to certain protocols, such as HTTP) can be sent and / or received to and from the node. There are many ways to implement filtering based on a particular type of data, such as putting the data in a packet and propagating the data along with header information that indicates the type of data contained in the packet. ..

The filter 258 can be added by one or more management devices 110 in FIG. 1 or by either the application operations management console 242 or the cluster operations management console 240 in FIG. In the example shown, the filter added by the Cluster Operations Management Console 240 (to set cluster boundaries) restricts full access to a node (for example, all access to another node). Filters added by the Application Operations Management Console 242 (to set subboundaries within the cluster) or by Management Device 110 restrict either full or partial access to the node. It is possible.

Controller 254 also imposes some restrictions on which filters can be added to filter 258. In the multi-tier management architecture shown in Figures 3 and 4, controller 254 allows the cluster operations management console 240 to add any filter it needs (which defines the boundaries of the cluster). Controller 254, on the other hand, limits the application operations management console 242 to adding filters that are at least as restrictive as the filters added by console 240. If console 242 attempts to add a filter that is less restrictive than the filter added by console 240 (in which case the subbound will cross the cluster boundary), controller 254 will add that filter. (Alternatively, you can modify the filter so that it doesn't become less restrictive). By providing such a restriction, the controller 254 can prevent the sub-boundary set at the application operations management level from crossing the cluster boundary set at the cluster operations management level.

Controller 254 operates to use one or more filters 258 to limit data packets sent from and / or received by node 248. All data destined for one engine 252, or data sent from engine 252 to another node, is passed through the network interface 256 and filter 258. The controller 254 applies a filter 258 to the data, and the target of the data (typically specified in the header part of the packet containing the data) is allowed (typically specified in the filter 258). Compare with and / or restricted) nodes (and / or network addresses). If the filter 258 indicates that the target of the data is allowed, the controller 254 allows the data to be passed to the target (sent to or out of node 248). On the other hand, if the filter 258 indicates that the target of the data is unacceptable, the controller 254 prohibits the data from being passed to the target. The controller 254 may return a notification that the data cannot be passed to the target to the source (source) of the data, or it may simply ignore or discard the data.

Applying filter 258 to the data by controller 254 allows you to place limits on the boundaries of the server cluster (Figure 3). Filter 258 can be programmed using the node addresses of all nodes in the server cluster (eg, cluster 212) (eg, by the application operations management console 242 in Figure 3). In this way, controller 254 prevents data received from nodes that do not belong to the server cluster from being passed to engine 252, as well as data that is sent to a node other than the node in the server cluster. Prevents being sent. Similarly, the data received from the Internet 108 (Figure 1) can identify the target node 248 (eg, by IP address), so the controller 254 on a node other than the target node will pass the data to the engine 252. Will be prevented. In addition, the filter 258 can be easily modified by the Cluster Operations Management Console 240, so the server cluster boundaries can be easily modified to accept changes in the server cluster (eg, nodes to server clusters). Adding and / or removing a node from the server cluster).

The BMCP module 260 implements the Distributed Host Control Protocol DHCP, so the BMonitor250 (and therefore node 248) gets an IP address from a DHCP server (for example, the Cluster Operations Management Console 240 in Figure 3). It makes it possible. During the initialization process of node 248, the BMCP module 260 requests an IP address from the DHCP server, and in response to this request, the DHCP server gives the IP address to module 260. More information about DHCP is available from Microsoft Corporation (Redmond, WA).

Software Engine 252 includes any of a wide range of traditional software components. Examples of engine 252 include an operating system (eg, Windows® NT), a load balancing server component (eg, balancing the processing load of multiple nodes 248), and a caching server component (eg, another). Cache data and / or instructions received from or over the Internet on node 248, storage manager components (for example, manage storage of data received from or over the Internet on another node 248), etc. There is. In some implementations, each of the engines 252 is a protocol-based engine, so even if the engine 252 and BMonitor250 are not written in the same programming language, they communicate with the BMonitor250 and other engines 252 through messages and / or function calls. ..

Controller 254 is used in conjunction with loader 264 to control the execution of engine 252. This control can take many forms, such as starting or starting engine 252, ending engine 252, reloading the image of engine 252 from a storage device, debugging engine 252 execution, and so on. .. Controller 254 receives instructions from the application operations management console 242 of FIG. 3 regarding which of these control actions to take and when to take those control actions. When invoking the execution of engine 252 (including restarting an engine whose execution was recently aborted), controller 254 communicates with loader 264 to store the image of engine 252 in a storage device (eg, device). Load from 262, ROM, etc.) into the memory (eg RAM) of node 248. The loader 264 operates as before, copying the image of the engine from the storage device to memory and initializing the required operating system parameters so that the engine 252 runs. Therefore, control of engine 252 is actually managed by a remote device rather than locally at the same location as managed node 248.

The controller 254 is also an interface from which the application operations management console 242 of FIG. 3 or the management device 110 of FIG. 1 can specify which filters to add (and / or remove) from the filter set 258.

The controller 254 also has an interface from which the cluster operations management console 240 of FIG. 3 can pass commands to the controller 254. Various types such as rebooting a node, shutting down a node, putting a node in a low power state (for example, suspend or standby state), changing cluster boundaries, changing encryption keys (if any), and so on. It is possible to transmit hardware operation-centric commands from the cluster operation management console 240 to the controller 254.

Controller 254 also provides encryption support for BMonitor250 (which is optional), so you can securely store your data on mass storage devices 262 (eg, magnetic disks, optical disks, etc.), and It allows secure communication between node 248 and an operations management console (eg console 240 or 242 in Figure 3) or other management device (eg management device 110 in Figure 1). Controller 254 stores multiple encryption keys, including symmetric keys (private keys used in private key cryptography) and public / private key pairs (for public key cryptography). It is possible to include a variety of heterogeneous keys that are used in encrypting and / or decrypting the data, such as.

BMonitor250 utilizes public key cryptography to be secure between node 248 and the management console (eg consoles 240 and 242 in Figure 3) or management device (eg management device 110 in Figure 1). Communication is performed. This public-key cryptography is based on a key pair that includes both a public key and a private key, and an encryption algorithm. Cryptographic algorithms encrypt data based on a public key so that it cannot be effectively decrypted without a private key. Therefore, since the communication from the public key holder is encrypted using the public key, only the private key holder can decrypt the communication. Public key cryptography is a well-known RSA (Rivest, Shamir, Adelman) A variety of cryptographic techniques can be used. As an introduction to the basics of cryptography, Bruce Schneir, Applied Cryptography: Protocols, Algorithms, and Source Code in C, John Wiley &amp; Sons, first edition There is copyright ownership 1994 (or 2nd edition copyright ownership 1996).

BMonitor250 has been initialized to include public / private key pairs for both landlords and tenants. These key pairs can be generated by BMonitor250, but otherwise they can be generated by other components and stored in BMonitor250 (in this case, assuming that the other components do not corrupt the key pair information). Trusted). In this specification, U is used to indicate the public key and R is used to indicate the private key. The public / private key pair for Landlord is (U_L, R_L), And the public / private key pair for the tenant is (U)_T, R_T). BMonitor250 has public key U_LAnd U_TIs made available to Landlord, but the private key R_LAnd R_T Keeps it secret. In the illustrated example, the BMonitor250 is the private key R._LAnd R_T Not only the landlord, but also the tenant, the information encrypted using the public key (for example, through the cluster operations management console 240 and the application operations management console 242, respectively), is not exposed. It is guaranteed not to be decrypted by any other entity except BMonitor250.

Public key U on land road_LAnd U_TGiven, Landlord assigns nodes 210-1 to 210-n to a particular tenant and gives that tenant the public key U._TCan be given. This public key U_TWith, the tenant can only decrypt BMonitor250 (private key R)_TYou can encrypt the communication to BMonitor250). A careful initial step for tenants, though not always necessary, is that the BMonitor250 has a new public / private key pair (U)._T, R_T) Is now required to be generated from BMonitor250. In response to this request, the key generator (not shown) of controller 254 or BMonitor250 generates a new public / private key pair in one of a variety of well-known methods, and the new key pair is used as a tenant key pair. Store as and new public key U_TTo the tenant. When you generate a new key pair, the tenant tells any other entity, including the land load, the tenant's public key U._TIs guaranteed to go unnoticed. In addition, tenants are also able to generate new key pairs at different times.

Having a public / private key pair when the BMonitor250 stores the private key and informs the tenant of the public key allows the tenant to securely convey the information to the BMonitor250. To ensure that the information is securely communicated from BMonitor250 to the tenant, another public / private key pair is generated by the tenant and the public key portion is communicated to BMonitor250. Therefore, any communication from the BMonitor250 to the tenant can be encrypted using this public key portion and can only be decrypted by the holder of the corresponding private key (ie, only by the tenant).

The BMonitor250 also holds the disk key as one of the keys 259, which is generated using one or more symmetric keys (where symmetric keys are secrets). The private key used for key encryption). The disk key, which is also a symmetric key, is used by the BMonitor 250 to store information on the mass storage device 262. Since the BMonitor250 keeps the disk key secure, it is only used to encrypt the data that the node stores on the high-capacity storage device 262 and decrypt the data that the node retrieves from the high-capacity storage device 262. (Therefore, you don't have to give your disk key to any other entity, including landlords and tenants).

The disk key ensures that the data stored on the high-capacity storage device 262 can only be decrypted by the node 248 that encrypted it, and not by any other node or device. Thus, for example, if the high-capacity storage device 262 is removed and an attempt is made to read the data on the device 262, the attempt will fail. The BMonitor250 uses a disk key to encrypt the data stored on the high-capacity storage device 262, regardless of the source of the data. For example, data sources include client devices used by tenant customers (eg, clients 1021 to 102n in Figure 1), management devices (eg, device 110 in Figure 1 or console 240 or 242 in Figure 3), and so on. There is.

In one implementation, the disk key is generated by combining the storage keys that correspond to each management device. There are many ways to combine storage keys, and in one implementation one of the keys is used to encrypt the other key and the resulting value is encrypted with the other key of the key. It is combined in this way.

In addition, BMonitor250 acts as a trusted third party that mediates interactions between multiple mutually distrustful management agents that share the management of Node 248. For example, the landlord and tenant of node 248 typically do not completely trust each other. Therefore, BMonitor250 acts as a trusted third party to ensure that the information made available to BMonitor250 by a particular entity or agent is accessible only to that entity or agent, and not to other entities or agents. We try to trust the lessor and lessee of the lender (for example, confidential information given by the lender is not accessible to the borrower and vice versa). BMonitor250 is a layered ownership domain DO The set of is used to facilitate this trust. The ownership domain is the basic unit of authentication and rights in BMonitor250, and each management entity or agent (eg, lender and borrower) is associated with a separate ownership domain (note that each management entity has more than one). If you have a management device, you can exercise management responsibility from it).

FIG. 6 is a block diagram showing a set of exemplary ownership domains according to some embodiments of the present invention. Multiple (x) ownership domains 280, 282, and 284 are organized as a ownership domain stack 286. Each ownership domain 280-284 is associated with a particular management level and one or more management devices (eg, device 110 in Figure 1, consoles 240 and 242 in Figure 3, and so on). Base or root Ownership domain 280 corresponds to the actual owner of the node, as in the land load mentioned above. The following sub-ownership domain 282 corresponds to an entity that leases hardware from the hardware owner (eg, the tenant mentioned above). A management device within a particular owner domain can set up another ownership domain for another management device that is higher on the ownership domain stack 286. For example, an entity whose nodes are leased can set up a different ownership domain for another entity (for example, to set up a node cluster that implements a database cluster).

When a new ownership domain is created, it is pushed to the top of the ownership domain stack 286. It remains the top-level ownership domain until you create another new ownership domain or the right is revoked. Ownership domain rights can be revoked by a device in one of the lower level ownership domains on the ownership domain stack 286, and when revoked, that ownership domain becomes the other higher level ownership. If there is a domain, it will be popped (removed) from stack 286 along with it. For example, if the owner of node 248 (ownership domain 280) revokes the rights of ownership domain 282, ownership domains 282 and 284 will be popped from ownership domain stack 286.

Each ownership domain has a corresponding set of rights. In the illustrated example, the top-level ownership domain has one set of rights, which includes the following rights: That is, (1) the right to push a new ownership domain onto the ownership domain stack, (2) the right to access any system memory within a node, (3) either within or combined to a node. The right to access that mass storage device, (4) the right to modify (add, remove, or modify) packet filters placed on the node, (5) the software engine on the node (eg, engine 252 in Figure 5). ) Right to start running, (6) right to stop running the software engine on the node, including resetting the node, (7) right to debug the software engine on the node, (8) own The right to change the authentication credential (for example, its public key or ID), (9) the right to modify its own storage key, (10) The right to subscribe to events such as engine events, machine events, and / or packet filter events (for example, notifying the management console or other device when one of these events occurs). In addition, each of the lower level ownership domains has a separate set of rights, which includes the following rights: That is, (1) the right to pop an existing ownership domain, (2) the right to modify (add, remove, or change) packet filters placed on the node, (3) own authentication credential (eg, public). The right to change the key or ID), (4) the right to subscribe to machine events and / or packet filter events. Alternatively, you can remove some of the above rights from the set (for example, in some circumstances you may not need the right to debug the software engine on the node), or include other rights in the set. It is also possible (for example, a top-level node can include the right to pop off itself from the ownership domain stack).

Ownership domains can be added to or removed from the ownership domain stack any number of times during the operation period. Which ownership domain is removed and / or added depends on the activity being performed. As an example, if the owner of node 248 (corresponding to root ownership domain 280) wants to perform some operation on node 248, all higher level ownership domains 282 ~ 284 are revoked. , The desired operation is performed (ownership domain 280 becomes the top level domain, so an extended rights set is available) and new ownership domains can be created and added to the ownership domain stack 286. (For example, a Management Agent that was previously associated with a top-level ownership domain will be reverted to its previous position).

The BMonitor250 checks which rights the ownership domain has each time a request is received from an entity that corresponds to one of the ownership domains (eg, the management console controlled by that entity). If the ownership domain has the required rights to fulfill the request, the BMonitor 250 will execute the request. However, if the ownership domain does not have the required set of rights, the request will not be executed (for example, you can return a notification to the requester that the request cannot be executed, or you can just ignore the request).

In the illustrated example, each ownership domain is an identifier Includes (ID), public key, and storage key. The identifier is used as a unique identifier for the ownership domain, the public key is used to send secure communications to the management device corresponding to that ownership domain, and the storage key is stored on the mass storage device. Used (at least in part) to encrypt information. Including an additional private key for each ownership domain allows the management device corresponding to the ownership domain to send secure communications to BMonitor. When the root ownership domain 280 is created, it is initialized to have its ID and public key (for example, by BMonitor250). The root ownership domain 280 can be initialized to include the storage key (and private key), or it can be added later (for example, let BMonitor250 generate it, or BMonitor250 from the management console). Tell, etc.). Similarly, each time a new ownership domain is created, the ownership domain that creates the new ownership domain transmits the ID and public key to the new ownership domain, BMonitor250. The storage key (and private key) can be created for the new ownership domain during or after the ownership domain is created.

BMonitor250 authenticates the management device corresponding to each of the ownership domains. BMonitor will not accept any commands from the management device until the management device is authenticated, and can authenticate the sensitive information (encryption key) of a particular ownership domain as corresponding to that ownership domain. Publish only to managed devices. This authentication process can be performed any number of times during the operation of the node, so the management device for one or more ownership domains will change over time. There are many possible ways to authenticate a management device. In one implementation, when a management device requests a connection with BMonitor250 and claims that it corresponds to a particular ownership domain, BMonitor250 will generate a token (eg, a random number) of the ownership domain. Encrypt the token with the public key and send the encrypted token to the requesting management device. Upon receiving the encrypted token, the management device uses its private key to decrypt the token and returns the decrypted token to the BMonitor250. If the returned token matches the token generated by BMonitor250, the authenticity of the management device is verified (because only the management device with the corresponding private key can decrypt the token). A similar process could be used for the BMonitor 250 to authenticate itself to the management device.

Once authenticated, the management device can propagate the request to the BMonitor250 and have it execute one of the requests (provided that it has the right to do so). A wise thing for the admin console, though not always necessary, is to change its public / private key pair the first time you authenticate yourself to the BMonitor250.

When a new ownership domain is created, the management device that creates the new ownership domain can optionally terminate the existing engine 252 and erase the system memory and bulk storage device. This enhances the security level in addition to encryption, preventing one management device from accessing information stored on the hardware by another. .. In addition, each time the ownership domain is popped off the stack, BMonitor250 shuts down the existing engine 252, erases system memory, and also erases the storage key for that ownership domain. Therefore, the information stored by that ownership domain is not accessed by the rest of the ownership domains. That is, since the memory has been erased, the data is not in the memory and the information on the mass storage device cannot be decrypted without the storage key. Alternatively, the BMonitor250 can also erase high capacity storage devices. However, if you erase the key and leave the data encrypted, the BMonitor250 can recover the data when the popped ownership domain is recreated (using the same storage key). become.

FIG. 7 is a flow chart showing the overall operation of the BMonitor 250 according to some embodiments of the present invention. First, the BMonitor250 monitors the incoming input (block 290). The sources from which these inputs are sent are another node 248, a client computer via network connection 216 (Figure 3), Client Operations Management Console 240, Application Operations Management Console 242, Engine 252, and Management Device 110 (Figure 1). ), And so on.

If the request received is a control request (for example, from one of the consoles 240 or 242 in Figure 3 or the management device in Figure 1), whether the requesting device has the required rights to the request. Is checked (based on top-level ownership domain) (block 292). If the requesting device does not have the required rights, BMonitor250 returns to input monitoring (block 290) without executing the request. However, if the requesting device has the required rights, the request is executed (block 294) and the BMonitor 250 continues to monitor the input received (block 290). However, if the request received is a data request (for example, inbound from a client computer via another node 248 or network connection 216, outbound from engine 252, etc.), BMonitor250 accepts the request or otherwise rejects it. Then continue monitoring the input being sent (block 290). Whether the BMonitor250 accepts the request depends on the filter 258 (Fig. 5), as described above.

FIG. 8 is a flowchart illustrating an exemplary process when processing an outbound data request according to some embodiments of the present invention. The process in Figure 8 is implemented in BMonitor250 in Figure 5 and can be run in software. The process of FIG. 8 is described in relation to the components shown in FIGS. 1, 3 and 5 as follows.

First, an outbound data request is received (step 300). Controller 254 compares the request with the outbound request restrictions (step 302). This comparison is the information that corresponds to the data (eg, the information in the header of the packet that contains the data or the information that is inherent in the data, for example, how the data request is passed to the BMonitor250 (eg, multiple function calls). This is done by accessing information such as which one was used) and comparing it with the outbound requirement constraints maintained by filter 258. This comparison allows the BMonitor250 to determine if it is allowed to pass outbound data requests to the target (step 304). For example, if filter 258 indicates to which target data cannot be sent, outbound data requests can only be passed to the target if the target identifier (ID) is not specified by filter 258.

If the outbound data request is allowed to be passed to the target, BMonitor250 sends the request to the target (step 306). For example, the BMonitor250 can send a request to the target via transport medium 211 (sometimes via network connection 216) or to network 108 via another connection. However, if the outbound data request is not allowed to be passed to the target, the BMonitor 250 rejects the request (step 308). The BMonitor250 can optionally send a notification that the request has been rejected to the source of the request, or simply drop the request.

FIG. 9 is a flowchart illustrating an exemplary process when processing an inbound data request according to some embodiments of the present invention. The process in Figure 9 is implemented in BMonitor250 in Figure 5 and can be run in software. The process of FIG. 9 will be described in relation to the components shown in FIG. 5 as follows.

First, an inbound data request is received (step 310). Controller 254 compares the request with the inbound request constraint (step 312). This comparison is made by accessing the information corresponding to the data and comparing it with the inbound requirement constraints maintained by filter 258. By making this comparison, the BMonitor250 can determine which of the software engines 252 is allowed to receive the data request (step 314). For example, if the filter 258 indicates from which source the data can be received, the engine 252 is only allowed to receive the data request when the source of the data is specified by the filter 258.

If the inbound data request is allowed to be received, the BMonitor250 forwards the request to the target engine 252 (step 316). However, if the inbound data request is not allowed to be received from the source, BMonitor250 rejects the request (step 318). The BMonitor250 can optionally send a notification that the request has been rejected to the source of the request, or simply drop the request.

<u style="single">Conclusion</u> Although the above description uses terminology specific to structural features and / or method steps, it is understandable that the invention as defined in the claims is the specific invention described above. It is not limited to features and steps. Rather, the particular features and steps are disclosed as exemplary embodiments for carrying out the present invention.

<figref num="1">FIG. 5 illustrates a client / server network system and environment that can be used in some embodiments of the present invention.</figref><figref num="2">FIG. 5 is a schematic diagram showing an example of a computer that can be used according to some embodiments of the present invention.</figref><figref num="3">It is a detailed block diagram which shows the exemplary joint location facility.</figref><figref num="4">It is a block diagram which shows the example multi-layer management architecture.</figref><figref num="5">It is a detailed block diagram which shows the example node by some Embodiment of this invention.</figref><figref num="6">FIG. 6 is a block diagram showing an exemplary ownership domain set according to some embodiments of the present invention.</figref><figref num="7">It is a flow diagram which shows the overall operation of BMonitor by some embodiments of this invention.</figref><figref num="8">It is a flowchart which shows the exemplary process for processing the outbound data request by some embodiments of this invention.</figref><figref num="9">FIG. 5 is a flow chart illustrating an exemplary process for processing an inbound data request according to some embodiments of the present invention.</figref>

Code description

1021 ~ 102n client 1041 ~ 104m Joint location facility 1061-1, 1061-2, 106m-1, 106m-1 server cluster 108 internet 110 Management device 112 server 210-1 ~ 210-n nodes 230 Cluster Operations Management Layer 232 Application Operations Management 234 Application Development Layer 240 Cluster Operations Management Console 242 Application Operations Management Console 248 nodes 250 B Monitor 252 engine 256 network interface 258 filter 259 key 260 BMCP module 262 Mass storage device 264 loader 280, 282, 284 Ownership domain 286 Ownership domain stack