Reliable multicast for cluster communications

Reliable multi-directional propagation for cluster communications

Related patent applications

This patent application is related to the following U.S. patent application "'.ClusterDestinationAddressTable IPRoutingforClt,sters", file number RO998-088, patent number 09/173,090, the date of the request 1998/10/15.

Background of the invention

Scope of invention

The present invention relates generally to cluster computers, and more particularly to communication architectures used on clusters of computer systems.

Background technique

In this electronic age, society's dependence on computer systems includes several kinds of information. According to various combinations of hardware (such as half-bone, circuit board, etc.) and software (such as computer programs), the design of computer systems has also changed a lot. Today, Many computer systems are designed to form a network with other computer systems through which a single computer system can access information stored on and processed by other computer systems, so that the network allows many computer systems to access many electronic resources.

Wired connections between computer systems and the use of common communication protocols can be made into networks. The choices are based on the following factors, including: the number of network computer systems, the distance between computer systems, and the computer system. The purpose of information exchange. If only a few computer systems are connected to the network at very close distances, the protocol can be extremely simple. However, if the number of computer systems increases significantly and the distance between computer systems is large, the communication protocol becomes complicated.

The complexity of communication protocols also varies according to the type of information exchange. For example, some agreements emphasize the accuracy of transmitting large amounts of information, while others emphasize the speed of information transmission. The communication requirements of applications executed on the computer system network determine the translation. The type of agreement, an example of a computer application that requires instant and reliable information transfer is a cluster management application.

The purpose of the computer system network cluster is to provide continuous available resources and shared load. From the perspective of the computer system user, the cluster of the computer system is a computer system, but in fact, it is a computer system network that supports each other, if it is in a computer system. If an overload or failure occurs in the cluster, the cluster cluster management application reassigns the processing of the failed computer system to another computer system in the cluster so that the user does not see an interruption in the available resources.

Clustering is achieved through the cluster management application executing on each computer system in the cluster, and these applications continue to interoperate with the cluster network to control cluster actions, such as computer systems in the cluster. 1211 - Continue to monitor the same cluster Each of the other computer systems is in place to ensure that each computer system is in action and performs its assigned processing. Cluster messages are also used to disseminate updates, such as the primary and backup work of those computer systems in the cluster. Solid support clusters require a lot of information, and the protocols used in cluster messages must support high-speed, instant, and reliable messaging.

Unfortunately, existing protocols have many limitations in providing instant and reliable information transfer for cluster communication, such as the general design of a network for use in a single area (also known as a local area network (LAN)), using existing agreed computer systems. The cluster is limited to the network in the area. The main limitation of this computer system cluster is that the cluster configuration is limited to individual languages. In addition, the protocols that support WAN, such as TCPIIP, do not have the reliability, speed and speed required for consolidation, so they cannot Effectively provide group communication between computer systems in a cluster.

As the resources available in the computer system network increase, the demand for using these network resources continues to increase, and Tj-seeking the cluster as a tool, the continuous availability of these network resources is relatively increased, if not reliable and fast. And the rapid way to communicate between the complex cone clusters of the computer system can not fully realize the continuous utilization of network resources.

Summary of invention

A cluster communication system is disclosed in accordance with the present invention to support reliable and rapid cluster communication. The cluster communication system of the preferred embodiment can be used to provide this reliable and rapid cluster communication for deployment of the cluster beyond the single area network {LAN). The cluster communication system provides reliable and rapid cluster communication by facilitating the multi-directional propagation of messages between clustered systems. In particular, the preferred embodiment provides for the establishment of a cluster between potentially multicastable messages. The preferred embodiment provides this multi-directional propagation while providing the required mechanisms to ensure sequential messaging between systems. The preferred embodiment extends this rapid and reliable cluster communication by providing point-to-point communication over relay nodes on systems other than the same language, and the relay node can then multi-directionally propagate to other elements of its linguistic multi-directional propagation group. . The preferred embodiment provides a cluster communication system that uses potentially reliable multi-directional propagation for rapid cluster communication, whereby the cluster is used to extend beyond a single regional network.

The preferred embodiment provides this rapid cluster communication over a single language by establishing and extending over known internet protocols and user data protocols, which provides highly reliable and fast interoperability between other IP systems. great.

The above and other features and advantages of the present invention will be apparent from the following description of the preferred embodiments of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The preferred embodiments of the present invention will be described with reference to the accompanying drawings in which

Real nL the best mode of the invention

The present invention relates to cluster communication. A preferred embodiment of the present invention uses object-oriented techniques to establish and extend Internet Protocol (IP) and User Data Protocol (UDP) to provide reliable and rapid cluster communication for cluster configuration. Extending beyond a single area network (LAN), in particular, a preferred cluster communication system provides reliable and rapid cluster communication by facilitating multi-directional propagation of information between clustered systems. For those unfamiliar with object-oriented programs, clustered user data protocols, and Internet Protocols, the following overview provides many basic concepts and terms to help the reader understand the preferred embodiment of the present invention, and is familiar with clustering, objects. Those who direct the program and the Internet transfer agreement can skip the introduction and jump directly to the detailed description section of this manual.

1. Introduction

Object oriented technology

An object-oriented program is a program implementation method in which a program is divided into a plurality of object sets that cooperate with each other, each of which represents an instance of a certain type, and its type is a member of a type hierarchy that is combined by a inheritance relationship, and an object-oriented program The difference from the standard program is that it uses objects instead of the first basic building block to generate computer programs. The difference is based on the fact that the object-oriented programming technology is designed to be completely different from the programming skills.

The focus of procedural design is on the whole process to solve the problem. However, the focus of object design is to treat the problem as a group of autonomous entities that can work together to propose a solution. The autonomous entity of object-oriented technology is of course object, object-oriented. The technology and program technology are obviously different. The problem is that they are divided into groups of cooperative objects rather than into nested computer programs or programs.

Thus, a pure object-oriented program consists of code entities called objects, each of which is an identifiable package of code and data so that at least one service can be provided at the request of the client. In theory, the object has two parts, that is, an external object interface and an internal object example. In particular, the object interface functionally encapsulates all the objects, so that other objects must communicate with the object through the object interface, and the object is uniquely captured. Processing or other operations are via the methods defined on the object. This protects the internal data portion of the object from external damage and, in addition, because the external object cannot enter the internal instance, the internal instance can be changed without affecting other features of the program.

Accordingly, the object system isolates the service requester (the container object) from the service provider (the server object) by a well-defined encapsulation interface. In the classical object model, the container object transmits the request message to the server object. To perform any necessary or desired functions, the message can identify a particular method for execution by the server object, and also provide any necessary parameters, the server object receives and translates the message, and can then decide what service to perform.

Today, there are many computer languages that support object-oriented programs such as Smalltalk, ObjectPascal, C++, and Java Zheng, which are *1-T. of these languages, which can support object-oriented programs to some extent.

Cluster

A cluster is a computer (or group of computers) that is connected together to allow computers to share work and backup each other, so a cluster allows the computer system to continue even if at least one of its computers fails in the cluster. Detecting and providing services" For computer users, a computer system cluster is a computer system. The cluster is transparent to the users of the computer cluster. It does not need to know whether it is using a computer system or multiple computer systems. Conversely, users of computer clusters need to know that they use the resources they need, such as databases, printers, files, etc. By clustering the computer systems together, the necessary resources can be continued.

There are many advantages to clustering computer systems. First and foremost, clustering provides high availability by allowing a group of centralized computer systems to back up each other as a backup, and second, by allowing additional computers to be added when needed. Systems, clusters can increase adaptability to improve processing capabilities, and third, workload can be balanced between computer systems in a cluster.

The computer system that constitutes the cluster is also called a node. The technical noun node refers to a processor, a communication controller, or a terminal. However, for the purpose of the cluster, the node refers to one of the computer systems in the cluster, usually in the cluster. The node is assigned a primary and backup responsibility to support the cluster. The assigned responsibility is a function or multi-function, such as providing access to data, executing a computer application, or providing access to hardware resources (such as printers, scanners, or faxes). machine). The nodes in the cluster communicate to ensure that all nodes have a role, so that the cluster software of each node is active and can actively monitor the status to switch from primary to backup. These basic cluster functions are called heartbeats, which are low between nodes. The order message ensures that each is actionable and normal.

The primary node cannot perform its assigned clustering function because they are offline for management and maintenance reasons, or the primary node fails to perform its function, in either case when the node's primary responsibility for assignment is an assigned function and it cannot be executed The cluster management application must ensure that the cluster consumer can still use resources that are assigned to the node and cannot be executed. When a node does not report, the other nodes receive a warning and query the situation of the stationary node. The nodes work together to obtain the common view, that is, when the stationary node is removed, and the primary responsibility is transmitted to the backup node, according to which Cluster users can still use the resources they need, even when the primary responsibility of the computer system is to provide these resources that are not yet available.

The cluster management application and the communication structure between all nodes enables a cluster to be operated by a single computer system from the perspective of the user. The cluster manufacturing application uses a common communication protocol to transfer many small messages between nodes, such as transmitting messages. To each node to inform them about other nodes in the cluster, send a message to each node to keep them up-to-date, that is, those nodes have primary and backup responsibilities for these functions, and the message is sent to each node so that all nodes agree When a node is unable to perform its assigned responsibility, what action should be taken, each node in the cluster must receive these cluster messages in the correct order to ensure proper clustering.

Because cluster management requires a small amount of information to be transmitted quickly, and the communication protocol used by the cluster message must support instant and reliable information transfer, the preferred embodiment of the present invention provides a communication server that uses UDP multi-directional propagation and cooperates with IP to facilitate rapid and reliable communication. Cluster message.

Networking agreement and user agreement

Applications, interfaces, protocols, and hardware used to facilitate computer networks are typically illustrated using the Open Systems Connectivity (OSI) rate reference model, which describes the members of a seven-layer network system, and each layer has its In its own special function group, each computer on the network is equipped with these seven layers of functions, so when the message is transmitted over the network, the data is transmitted down to the layers of the transmitting end, and when the message arrives at the other end, the other data is Transfer up to the layers in the receiving computer. The actual programs and hardware with these layers are usually computer operating systems, applications, transport and network protocols, and hardware and software combinations to place signals on the transmission medium.

The Internet Protocol (1P) is a common protocol in the network layer of the OSI reference model. Internet Protocol defines the method of transferring data from one computer to another on the Internet. The Internet Protocol has become many others. The protocol chosen for regional and wide area networks, although the Internet Protocol developed allows cooperative computers to share resources on the Internet, it can still be used to share resources on less complex networks, including small areas (also Known as a computer in a local area network (LAN), a computer in a different location (also known as a wide area network (WAN)), and an interconnected network (also known as a network)

The IP layer provides the basic service of transmitting messages to their destinations. Messages can pass through many different networks before reaching their final destination. However, users can fully control IP messaging as long as the user can provide the destination Internet address. site.

IP transmission is based on a numeric Internet gateway, which consists of four digits separated by periods, such as 9.1.1.1. The Internet address can be further divided into a network address and a host address. In the previous example, the address is The network part is 9.1.1 and the host address is I. How many Internet addresses are network addresses and how much is the host address depends on the network. The host roughly refers to the hardware on the network. And transmitting IP messages. As for clustering, the host refers to one of the computer systems (also called nodes) in the cluster.

User Data Protocol (UDP) and Transmission Control Protocol (TCP) are common transport layers for managing data end-to-end control. These layers work together to transfer data to and from the network layer. For example, the TCP protocol process divides the message into parts called packets on the transmitting end, and then combines the packets into the receiving end to form a received message. When the message is to be transmitted, the TCP protocol divides the message into packets, and transmits the packets to the network. The layer (usually IP) is transmitted for transmission. At the receiving end, the IP layer receives the packets and transmits the packets to the receiving TCP layer (recombined here).

UDP provides similar interaction with the network layer, that is, transmitting data to and receiving data from the network layer. However, UDP does not provide services for dividing messages into packets and recombining them at the other end. Clearly, UDP does not provide The ordering of the packets when the data arrives, which means that the application using UDP must ensure that the entire message has arrived and the order is correct.

TCP is called a connection-oriented protocol, which means that the external connection is maintained until the message to be exchanged has been exchanged. TCP is called a reliable protocol because it has the functions required for reliable communication, such as TCP including ensuring transmission, and providing retransmission when needed. In terms of the function of the zither, the UDP protocol is designed to be high performance, but there is no function required to ensure reliable transmission.

For example, UDP provides a technique called multi-directional propagation, which is the communication between a single transmitter and multiple recipients on the network. This is a more rapid message between multiple computers, for example, without providing multi-directional In the transmitted TCPIIP network, separate messages must be transmitted to the receiving computers. Therefore, in order for each message data to flow from the application to the transport layer and the network layer on the transmitting computer, a separate message is transmitted to each receiver to degrade the transmission. The performance of the system, in contrast, a computer system that provides multi-directional propagation (-IraUDPIIP network), the transmission of information is transmitted only once through the transmission layer and the network layer of the transmission system. Therefore, if different messages are transmitted to each computer, Use multi-directional propagation to more quickly reach the same message that delivers the same message to many different computers

2. Detailed description

In accordance with the present invention, a cluster communication system is provided to support reliable and rapid cluster communication, and a preferred embodiment of the cluster communication system can be used to provide this reliable and rapid cluster communication for the cluster configuration to extend beyond a single area network (LAN). The cluster communication system provides reliable and rapid cluster communication by facilitating the multi-directional propagation of messages between clustered systems. In particular, the preferred embodiment provides for the establishment of a cluster of clusters between the provision of multi-directional propagation messages. The preferred embodiment provides this multi-directional propagation while providing the required mechanisms to ensure sequential message transfer between systems. The preferred embodiment extends this rapid and reliable cluster communication by providing additional point-to-point communication between systems in non-identical languages. The preferred embodiment therefore provides a cluster communication system for reliable cluster communication using reliable multi-directional propagation, whereby the cluster is used to extend beyond a single area network

Referring to FIG. 1, a computer system 100 according to a preferred embodiment of the present invention is an AS/400 medium-sized computer system. However, those skilled in the art can understand that the method and apparatus of the present invention can be equally applied to any computer system, regardless of whether the computer system is For complex multi-user computing devices or single user messages such as personal computers or workstations. The computer system 100 details how the preferred embodiment configures a group of centralized nodes, and the other nodes in the cluster on the network 195 preferably include: processor 110 main memory 120 memory controller 130, auxiliary storage The interface 140, the terminal interface 150, and the network interface 190 are all connected via the system bus bar 160. It is noted that various modifications or additions to the computer system 100 of FIG. 1 may be made within the scope of the present invention, such as adding cache memory or other peripheral devices. FIG. 1 illustrates only some of the features of computer system 100.

The processor 110 performs the computing and control functions of the computer system 100 and includes a returning central processing unit (CPU). The processor 110 includes a single integrated circuit (such as a microprocessor) or includes any suitable number of integrated circuits. A&i or a mated circuit board is provided to perform the functions of the processor, and the processor 110 preferably executes the computer program in the main memory 120 as needed.

Auxiliary storage interface 140 yuan allows the computer system 100 to store and retrieve information on auxiliary storage devices, such as magnetic disks (such as hard disks or floppy disks) or optical storage devices (such as CD-ROM). A suitable storage device is directly stored. The storage device (DASD) 170, the DASD 170 of FIG. 1 is a floppy disk drive that can read programs and data from the floppy disk 180. It is important to note that although the present invention is described in a full-function computer system (the following will continue to be explained), A person skilled in the art will appreciate that the mechanisms of the present invention can be distributed into a program product in a variety of forms, and that the present invention is applicable regardless of the particular type of signal-containing media that actually performs the distribution. Examples of signal-containing media include 'recordable media. Such as floppy disks (such as disk 180) & CD-ROM, and transmission media such as digital and analog communication links, including wireless communication links.

The memory controller 130 is responsible for moving the request information from the main memory 120 (or via the auxiliary storage interface 140 to the processor 110) by using a processor (not shown) separate from the processor 110. Although the memory controller 130 in the figure is a separate entity for illustrative purposes, those skilled in the art will appreciate that in fact some of the functions provided by the memory controller 130 are actually located in the processor 110, the main memory 120 and/or the auxiliary. The storage interface 140 belongs to the circuit.

The terminal interface 150 yuan allows system administrators and computer programmers to communicate with the computer system loo via a programmable workstation, although the system 100 of FIG. 1 includes only a single main processor 110 and a single system bus 160. Suitable for computer systems with multiple processors and multiple system busses. Similarly, although the system bus 60 of the preferred embodiment is a typical hardware multi-drop bus, any connection device can be used as long as it can support two-way communication in a computer-related environment.

The network interface 190 supports the transfer of information between the computer system 100 and the remote computer system in the network 195. In the preferred embodiment, at least one node can be similarly set up on the network 195 to cooperate with the computer system 100 as a cluster. The network interface 190 preferably includes at least one network interface card 193. Each network interface card 193 is usually an expansion card, which can be easily added to a computer system such as the computer system 100. The network interface card 193 includes peripheral component connections (PCI). ) Expansion cards, Industry Standard Architecture (ISA) expansion cards, exclusive interface cards, and any of the custom and future interface cards. Those skilled in the art will appreciate that the functionality of the network interface 190 can be directly implemented as part of the main memory and processor 110. Network 195 represents any kind of custom network, including the Internet, corporate intranet, regional network (LAN), wide area network (WAN) or any custom or future development hardware and software configuration, irrl computer system Can communicate with each other.

The main memory 120 preferably includes: at least one application 121, the cluster management application 122 operating system 123, the cluster cup server 124, and the cluster communication server 125. All of the programs in the memory 120 are used in the broadest sense, and This includes any and all forms of computer programs, including source code intermediate code, machine code and any other representation of a computer program.

In the preferred embodiment, the application 121 can include any program to use the cluster to provide greater reliability and adaptability, so the application 121 generally includes all of the programs, wherein the computer system 100 is the primary or backup node for these applications. Examples of programs include web servers, file servers, and database servers.

The cluster management application 122 provides the required mechanisms to generate a management cluster. This includes management of the management requests to manage the computer cluster. For example, this preferably includes mechanisms for generating clusters, adding and removing cluster node kites.

In the preferred embodiment, cluster topology server 124 cluster communication service 125 and network message server 126 are integrated with operating system 123 to provide basic functionality to facilitate communication between nodes in the cluster, of course, You can understand that these functions can also be provided as additional applications for other operating systems. For example, you can add functions to the following operating systems, -k "1BM OS/2, OS1400, RS16000, Microsoft Windows NT, Novell" for NetWare Linux and other Unix Various products

In a preferred embodiment, the detailed topology server 124 provides the functionality required to set up and execute at least one multi-directional propagation group in the cluster, the noun multi-directional propagation group used in the application being a node that has been combined Meta-licensing is transmitted to all members of the group by means of multi-directional propagation messages, as described below. In a typical cluster configuration, the primary node of a particular application and all backup nodes are members of a multi-directional propagation group, which allows Quick and reliable messaging between all nodes sharing shared responsibility"

In the preferred embodiment, the cluster communication server 125 provides a network to facilitate the transmission and confirmation of the reception of the multi-directional broadcast message. In addition, the cluster communication server 125 also includes the function of transmitting the point-to-point message to each node in the cluster as needed, cluster communication. Server 125 also provides the ability to ensure multi-directional propagation and sequential delivery of peer-to-peer messages.

In the preferred embodiment, the network message server 126 includes a protocol group to transmit and receive multi-directional propagation and peer-to-peer messages as directed by the cluster communication server 125. To facilitate this, the communication message server 126 preferably includes a user profile 127. And the Internet Protocol 128 User Profile 127 adds the standard header format to the message for transmission to other computers in the network 195. As described below, the preferred embodiment of the present invention uses and extends the multi-directional propagation of UDPIIP. Features to facilitate reliable and fast cluster communication.

The master memory 120 does not have to include all parts of all mechanisms at all times, for example, a portion of the application 121, the cluster management application 122, and the operating system 123 can load instruction cache memory (not shown) for execution by the processor 110. Other files can be stored in a magnetic (or optical) storage device LL (not shown), and although the computer programs in the figure are located in the same memory location, it is to be understood that the main memory 120 can be composed of different memory locations. The memory used herein is any storage location in the virtual memory space of the referred system 100.

It is also understood that computer system 100 is typical of nodes in a cluster, and that the nodes in the cluster preferably include cluster cup server 124 and cluster communication server 125 to facilitate reliable multi-directional propagation of information between nodes in the cluster.

Referring now to Figure 2, a typical cluster of computer system 2004 is schematically illustrated to illustrate how the preferred embodiment of the present invention facilitates rapid and reliable messaging between nodes in a cluster. In particular, the preferred embodiment can be used to illustrate how the cluster can be expanded beyond a single language. The typical cluster 200 includes nine nodes, where node AD is in subnet 1, node EG is in subnet 2, and node H&I is in child. Network 3. Each subnet's node clusters are networked together and have a common network address (such as a common IP header). The three subnets are connected by a network. This network is preferably connected by a standard IP. And related hardware (such as routers, transmission media, etc.), such as the Internet, the entire industry internal network, or connected via a dedicated WAN connection.

In the preferred embodiment, the nodes of each subnet are referred to as relay nodes. As described below, the relay node is a node that transmits the multidirectional propagation cluster message from other subnetworks, and the relay node continues. Multi-directionally propagates messages to other nodes in the sub-network, for example, node A is called a relay node of sub-network 1, node E is called a relay node of sub-network 2, and node H is called sub-network 3. Relay node.

At least one multi-directional propagation group is established on a typical cluster 200 of the preferred embodiment. As described above, the multi-directional propagation group is a group of nodes in a cluster that has been designated to communicate using the reliable multi-directional propagation techniques provided by the preferred embodiment. , usually define groups of nodes, primary and backup, work together to support a particular application

For example, a multi-directional propagation group can be established including nodes A, C&D on the sub-network I, which in turn is used to provide a network server, where node A is the primary node and node C&D is the backup node. The second multi-directional propagation group can include a node B&D on the sub-network 1, and a node E, G&F on the sub-network 2. This multi-directional propagation group can then be used to provide a database in which node E is the master node and nodes B, D, F&G are the backup nodes. Those skilled in the art will appreciate that these are just two of many different types of applications and groups that are supported by the use of preferred embodiments of the present invention.

By dawn. The preferred embodiments of this group are advantageous for reliable multi-directional propagation between nodes in a defined group, while still allowing clusters to exist outside the limits of a single language, which allows cluster applications to use reliable multi-directional propagation messages. To transfer cluster messages.

Referring to Figure 3, to illustrate the cluster management application 122 of the preferred embodiment in detail, as described above, the cluster management application 122 provides the required mechanisms to generate cluster management, using the cluster management application 122 for a demanded call using a defined interface. The appropriate method on the cluster management application 122, the cluster management application preferably includes: generating the cluster createCluster () method, destroying the cluster destroyCluster () method, solving the cluster resolveCluster () method, adding the section Y.} addNode () method, removing the node removeNode The () method starts the node startNode() method and ends the node endNode() method. These methods provide the mechanisms needed to create a cluster, add nodes to the cluster, remove nodes from the cluster, and so on.

Using the generate cluster createCluster() method to generate a cluster, and as the input name of the cluster to be generated, it is also desirable to generate a cluster topology ClusterTopology object to provide cluster topology service 124 functionality and to generate a cluster communication object. Provides cluster communication server 125 functionality. Similarly, destroy the cluster destroyCluster{) method as a parameter for the cluster name, and in addition to any consistent objects generated to implement the cluster. Resolving the cluster The resolveCluster() method refers to a method for extracting metrics from a cluster topology server for use by a particular cluster of cluster names. Adding a node addNode() method joins a particular node to a particular cluster. This method also assigns a specific IP address to a particular node. The remove node removeNode(} method removes a specific node from the cluster. The start node startNode{} method specifically resets the nodes that have joined the cluster and starts with a action cluster node. Similarly, the end node() method is terminated. A node that has joined the cluster fails.

Referring to FIG. 4 in detail to illustrate the cluster topology server 124 in the preferred embodiment, the cluster topology server 124 provides the required functionality to set up and implement at least one multi-directional propagation group in the cluster, again, the multi-directional propagation group is herein Medium refers to nodes that have been joined together to allow reliable multi-directional propagation of messages to members of the group. Once a multi-directional propagation group is set up, multi-directional propagation can be used to achieve communication between nodes in the cluster, and multi-directional propagation allows all nodes to transmit quickly and reliably to share common responsibility.

The cluster topology server 124 includes the method of generating the createMCGroup(} method, adding the addMCGroupMembers() method, removing the removeMCGroupMembers() method, deleting the deleteMCGroup() method, resolving the resolveCCServices() method, starting the node startNode() method, and ending the node endNode ( Method. createMCGroup() uses a specific name to generate a new multi-directional propagation group addMCGroupMembers{) method to add a new specific node to a specific multi-directional propagation group. This method also assigns a specific node to receive messages for transmission to its sub-network. This has a very specific multi-directional propagation address. Similarly, the removeMCGroupMembers() method removes a specific node from a specific multi-directional propagation group. The deleteMCGroup() method deletes a specific multi-directional propagation group, and the resolveCCServices() method takes the indicator. Send back the cluster communication service object for this node.

Referring to FIG. 5 in detail to illustrate the cluster communication server 125 of the preferred embodiment, the cluster communication server 125 provides a network to facilitate transmission and confirmation of receipt of previously defined multi-directional broadcast groups, and can transmit and acknowledge receipt of peer-to-peer points as needed. Message to each node in the cluster. The object-oriented method is used to implement the cluster communication server 125 of the preferred embodiment, and preferably includes: a cluster communication service object; a CCOrderedConnection object of each message queue in the cluster, and a CCMessag device of each current sub-message; each destination subnet The packet header of the road receives the CCMessage object of each action; the user data message buffer: CCGroupMa object: CCServiceGroup object and CCGroup object of each defined multi-directional propagation group "CCNodeMap object: CCServiceNode object and CCNode of each node in the cluster Object; CCServiceNode object and CCNode object of each subnet in the cluster; CCsubnetMap object; CCServiceSubnet object and CCSubnet object of each subnet in the cluster; and CCScam object.

It is noted that at the beginning of the preferred embodiment, the cluster communication server 125 resides at each node of the cluster, and the function of the cluster communication server 125 is to transmit messages to other nodes and receive messages from other nodes. In its transport function, the cluster communication server 125 is used to transmit messages to other nodes, including multi-directionally propagating messages to the preset group. "And in its delivery function, the cluster communication server 125 tracks the receiving approvals from all nodes receiving the message. And ensuring that no additional messages are sent until the previous message has been received. On its receiving function, the cluster communication server 125 receives the messages from other nodes on the cluster and transmits the messages to their final destination. Moreover, in terms of its receiving function, It tracks the receipt of the message to ensure that the message is not unintentionally transmitted to the destination twice, and transmits the endorsement message to the node that sent the message."

The cluster communication service object provides the basic functionality of the cluster communication server 125 and includes methods for transmitting peer-to-peer messages to specific nodes and transmitting multi-directional propagation messages to previously defined multi-directional propagation groups. The cluster communication service objects also include methods to ensure multi-directional propagation. Messages and peer-to-peer messages are transmitted in a specific order.

In particular, the sendMsgtoNode() method transmits a point-to-point message to the specified node of the call. Similarly, the sendMsgtoGroup(} method transmits the multi-directional propagation method to a specific group.

The requestOrderedConnection() method allows a peer-to-peer message of a member of the group to be sorted relative to the multi-directional message in a specific message. As described below, each multi-way group has a CCOrderedConnection object. Implementing the group's ordered multi-directional propagation message ', 1, in addition, it is also preferable to use another CCOrderedConnection object to implement a peer-to-peer message queue, usually these messages are transmitted to their nodes in a light-on-first-out manner. And group, requestOrderedConnection () method to send the indicator back to the specific delivery CCOrderedConnection object, you can use the group name to specify the CCOrderedConnection object, in this case the returned indicator is the CCOrderedConnection object of a specific group or the node name, in this case sent back The indicator is the CCOrderedConnection object of a particular node. "This indicator can then be used as a parameter to send a message method to allow a special message to be sent to a specific queue. This can be used to insert any specific CCOrderedConnection object with nine point-to-point messages. *1iuAA,; To 11 indicators point to a special 11 multi-directional propagation group C The COrderedConnection object, that is, a peer-to-peer message can be transmitted with respect to the special item order of the multi-directional message to be transmitted to the group through the message, so that the method facilitates the deduction of the message in the mixed message type.

The sendMsgtoNode() method sends a message to a specific node, especially the sendMsgtoNode() method; the destination node ID1 points to the indicator that conveys the message, points to the indicator of the object, and if the message cannot be transmitted or the transmission is confirmed, the notification is notified. The object, and the indicator pointing to the flag object, is used to feed the delivery option. These transmission options generally include the following items, such as reliable message delivery, unreliable message delivery, notification of failure, notification of successful delivery, and the like. Reliable messaging is transmitted using the CCOrderedConnection object to ensure sequential, confirmed delivery, and reliable messaging. Unreliable messages are bypassed and sent directly to CCNodd without requesting the recipient's approval. Unreliable messaging is used for things that don't have to be confirmed, hehe. Cluster heartbeat message, so do not use the bandwidth in the transmission queue, notification of failure and notification of transmission parameters (after exhausting all data), if the message fails, decide whether to notify the transmitter, or whether it should only be transmitted Notify the sender.

Similarly, the sendMsgtoGroup() method sends a message to a specific group. This message uses: the destination node ID, the indicator that points to the message to be sent, the indicator that points to the object, and if the message cannot be transmitted or the message is acknowledged, the object to be notified is notified. And indicators for flagged objects, for sending transmission options. These transmission options generally include the following items: deductive reliable message transmission, unreliable message transmission, notification of failure, and notification of successful delivery. "Re-reliable message transmission using CCOrderedConnection The object transfer port 11 ensures the sequential, confirmed transmission, and reliable message transmission. The unreliable message is bypassed and transmitted directly to the CCNode object without requesting the recipient's approval. Again, the failure notification And the notification of the transmission parameter (after exhausting all the data), if the message fails, it is determined whether the transmitter is to be notified or whether the transmitter should only be notified at the time of transmission.

The Shutdown{) method is used to suspend communication when needed. The preferred embodiment uses the UDPIIP protocol as described above to facilitate communication between nodes in the cluster. This requires UDP to be enabled when generated, and the Shutdown() method is used to set the cluster communication. The status of the service object, remove any remaining messages from the queue, and turn off any open UDP ports used by this cluster.

The CCGroupMap object includes mappings of all multi-directional propagation groups defined in the cluster. In particular, the CCGroupMap object includes a form of all multi-directional propagation group IDs in the cluster, and an indicator points to the CCServiceGroup object of each multi-directional propagation group. As described above, each of the defined multi-directional propagation groups preferably generates CCServiceGroup objects and CCGroup objects, and the CCServiceGroup objects include those specific to a particular cluster communication service object. In particular, the CCServiceGroup object preferably includes a subnet transmission table. List the various subnets and the nodes in the corresponding group on these subnets. This table is used to determine which subnets need to transmit a message 511 to a particular group. The CCServiceGroup* component preferably also includes the last message sequence number of each group to track the sequence number used to transmit the last message to each group, and a sub-network sub-stand counter array to track each message. FJ has fully recognized those CCSubnet objects when starting The counter becomes zero when the CCSubnet object that sent the message has caused the value of the CCGroup object subnet wait counter to decrease.

The CCNodeMap object also includes mappings for all nodes in the cluster. In particular, the CCGroupMap object includes a form of all node IDs in the cluster, and an indicator points to each of the multi-directional CCServiceNode objects. As mentioned above, each node in the cluster preferably generates CCServiceNode objects and CCNode objects, including the normal nodes. Preferably, the CCServiceNode object includes the last approved message array, and each data stores the sequence number of the last message, and the acknowledgement has been received by the corresponding node to which each message is received. As described below, the array can be used to determine which nodes have not been accepted. To the message.

Preferably, the CCNodd component includes an address of the node, preferably including a predetermined point-to-point address of the node, and any other indicator pointing to the indicator address of the node, which also includes the preset multi-directional propagation bit of the node. site.

Preferably, the CCServiceNode object also includes an array for subsequent reception, wherein each data corresponds to a sequence number to be received next, and is transmitted by a CCServiceNode object of each message. The CCServiceNode object will only transmit messages with this sequence number, or a higher sequence number. This ensures that the CCServiceNode object does not transmit a message to the destination CCServiceNode object it has received and transmitted to receive and transmit a higher sequence number message to account for the fact that the intervening message is peer-to-peer rather than transmitted to the receiving node. The risk of unintentional loss of the message will not be allowed because the message will be transmitted again until all destination nodes have received it, or until it is determined that the destination node cannot be used because it has been removed.

It is pointed out that the CCNode object and the CCServiceNode object can facilitate the receiving function of the cluster communication server 125 to transmit messages to the node, in particular, the CCNode object of the transmitting node and the CCServiceNode object (ie, the node transmitting the subject message) are received to be sent to the resident. The node's message, and the message is sent to its destination (deceleration engine), and the request is issued upon request. The CCServiceNode object includes the next to be received, and the numbers in the array 1J include the next sequence number, that is, the node expects to receive from the corresponding message }1! of the transmitting node. As mentioned above, this is used to ensure that the CCServiceNode object does not inadvertently retransmit the message to the destination, such as a female. When retransmitting to complete the transfer to other nodes.

It should also be noted that the cluster communication server 125 of each node preferably itself includes a CCNode object and a CCServiceNode object, ie, a regional node. These objects are used to facilitate the receiving function of the cluster communication server 125 to send back a message, ie, the message is sent back to itself.

The CCSubnetMap object also includes mapping of all subnets to include cluster nodes. In particular, the CCSubnetMap object includes a form with all subnet IDs of the cluster nodes, and an indicator points to the corresponding CCServiceSubnet object as described above, with cluster nodes. Each subnet finally generates CCServiceSubnet objects and CCSubnet objects. The actions of these objects depend on the type of the corresponding subnet. For example, if the corresponding subnet is a regional language, the CCSubnet object uses UDP multidirectional propagation to propagate to multiple directions. Group, in another example, if the corresponding subnet is a non-regional language including multiple members of a multi-directional bridging group, CCSubnet uses UDP peer-to-peer messages to transmit messages to a relay node, and the relay node then uses UDP multi-directional propagation. Multi-directional propagation to node members in the group. The loop subnet also exists in its form, in which the message is transmitted back to the transmitting node, which can be achieved by transmitting only the appropriate indicator to the node object, and the point-to-point subnet facilitates the point-to-point transmission of the message to a particular node.

Preferably, each CCSubnet object includes a preset address, which can be used if no transmission is specified. The preset address is determined according to the type of the corresponding subnet. In particular, in the preferred embodiment, there are three types of pre-configuration. Set the address, that is, multi-directional propagation, point-to-point and loop. Regarding the point-to-point subnet, the preset address is only the IP address of the destination node. As for the loop subnet, the preset address may be its own IP address, or a special preset address is used to indicate that it is to be transmitted to Its own message. As for the multi-directional propagation sub-network, it is preferable to establish a pre-defined preset UDP multi-directional propagation address for addressing the multi-directional propagation group, using a highly identifiable transmission to the address. To the propagation, all nodes joining the multi-directional propagation group are aligned with the IP push to receive the message conveying the address of the Internet. Note that if necessary, all groups can be assigned to a single preset UDP multi-propagation address, and the nodes of the regional sub-network that are not in the group then receive all multi-directional propagation messages, but only the transmitted group is its member. This message will only be processed. By setting the survival time 1P parameter to one, these messages can be prevented from being unintentionally scattered outside the regional subnet. The known router reduces the time-to-live parameter by one to relay a message, and if the lifetime is zero, the router This message will not be relayed. Therefore, by setting the survival time to 1, all the multi-directional propagation messages can be transmitted to the same preset UDP multi-directional propagation address without causing excessive transmission of messages. As described above, separate point-to-point messages are transmitted to relay nodes on non-regional languages that include members of a particular group and thus ensure that all members also receive the message.

Each CCServiceSubnet object preferably includes an approval counter r bad11, the array, the IJ has an acknowledgement counter for each message queue, and the CCNode object receives the approval message and the recognition counter is decremented by one to allow the CCSubnet object to determine when all nodes are determined. Recognize the receipt of the message. The CCServiceSubnet object also preferably includes the remaining retry arrays, and the array '1J leaves the retrying members to make it possible for each message to be queued before the message transmission is determined to be a failure. After failing several times, try to send the message to other available addresses, such as the node's default point-to-point address.

Hey. As described above, the CCOrderedConnection object implements a sequential message '''in the preferred implementation*J, providing a CCOrderedConnection object for use by each defined multi-directional propagation group, and preferably, at least one other CCOrderedConnection object is also used to implement the peer-to-peer message. Message J

Using the CCMessage object for transmitting messages, the CCMessage object preferably includes pointers to the message data stored in the user profile message buffer. The CCMessage object is an object that queues the CCOrderedConnection message '1l. In order to reliably transmit the message, the CCMessag device is maintained until it is assumed that all nodes and groups have received the message to accept the message. At this point, the }%CCMessage object is sent to the available object area until it is set for new messages. The CCMessage object preferably identifies whether it is a single node or multi-directional propagation group, source and destination ID, message ID, whether the message is reliable or Unreliable transmission, whether the message is a ringer or cluster structure, related message queues, and notification objects are available to facilitate the call when the message is completed. This information can be used to create a packet header to receive the CCMessage object.

Establishing and connecting the packet header to the CCMessage object of each action, in particular, in the preferred embodiment, establishing and connecting the packet header to the CCMessage object of each subnet, and the corresponding message is transmitted to the subnet, for example, if The message is sent to the nodes of the subnet 1, 2, J, and the packet is placed on the corresponding CCMessage object of each destination subnet. Preferably, each packet header includes a source and destination ID, a source and destination address of the node or group, and an indicator pointing to the flag object to specify the request service, the message sequence number, and the sequential connection number of the specified message queue. The header also preferably includes a relay bit to indicate whether the receiving node should relay the message to other nodes in its language, or it is just a general peer-to-peer message.

The CCScam object facilitates the transfer of messages between the CCSubnet object and the UDPIIP protocol below, so the CCScam object preferably includes a method of transmitting messages, which can be called by the CCSubnet object when the message is transmitted. This preferably includes a method of transmitting a reliable message to set the message flag to request approval and send it back to the sender. Also included is a method of transmitting an unreliable message that does not request approval but transmits any previously submitted approval request. CCScam preferably includes a method of transmitting packet header information. This message only transmits the packet header without transmitting the CCMessage object. This method is used to recognize the message sent back to the transmitting node, and the bandwidth is reserved. When calling these transmission methods, CCScam is The message is sent to the UDPIIP layer below for delivery to a specific node.

The CCScam object also preferably includes a mechanism that can divide the large message into a number of small packets when needed, and then receive the CCSubne, and the packets can be combined in the correct order, especially for the large message via the network. The bandwidth is transmitted and transmitted to improve the reliability of message transmission.

Referring to Figure 6 to illustrate a method 500 for transmitting a multi-directionally propagated message in accordance with a preferred embodiment of the sendMsgtoGroup() method on a cluster structure call cluster communication service object, in the first step 502, the cluster structure is required to transport the cluster When the message is sent to other nodes in the cluster, the call and sendMsgtoNode() methods are called, so this requires a variety of messages and content that can be used to specify the multi-directional broadcast group for transmitting messages, and includes indicators to point to buffers including message data. .

The next step W504 is to arrange the transmission message into the CCOrderedCOnnection object Store) storing the appropriate message l}l1, and set the CCMessage object to include the message. As mentioned above, it is preferable to have a CCOrderedConnection object message '11 for each multi-directional propagation group, and at least a point-to-point message. The sendMsgtoGroup() method adds a specific message to the message FJ of a particular multi-directional propagation group, generates a CCMessage object to include the message material, preferably by translating from the available CCMessage object area, and for this purpose.

The next step 506 is to obtain the CCGroup object indicator of the specific multi-directional propagation group from the CCGroupMap object. As described above, the CCGroupMap object preferably includes all previously generated multi-directional propagation group forms and indicators pointing to the CCGroup&CCServiceGroup object to which it belongs. With this indicator, the next step 508 is to obtain a form of the subnet that includes the nodes in the particular multi-directional propagation group. As described above, when the multi-directional propagation group is generated, the sub-network form is stored in the CCGroup object, and further, the indicator pointing to each CCSubnet object corresponds to the sub-network including the node in the multi-directional propagation group, and is stored in the CCGroup object. in

The next step 510 calls the appropriate transmission method on each CCSubnet object to correspond to the sub-network including the multi-directional propagation node, which is an indicator provided by the CCSubnet group, and the CCSubnet object calls the corresponding CCServiceSubnet object transmission method, the next step is 512 The CCServiceSubnet object adds the current sequence number to the message method and increments the sequence number to send the message, which ensures that the next message receives a larger sequence number than the current message.

The next step 514 causes the CCServiceSubnet object to transmit a message to the CCScam object by calling the appropriate method on the CCScam object and sending the appropriate address to the CCScam object. "The transmitted address is based on the transmitted subnet type and the corresponding multidirectional The propagation, relay, loop, and different addresses of the peer-to-peer message. As for the multi-directional communication message, the CCServiceSubnet object transmits the preset UDP multi-directional propagation address and the current serial number. As for the relay and the point-to-point CCSubnet object, the destination node is transmitted. IP address and corresponding message }1l0. Current serial number, as for the loop, the address is a preset loop address to indicate that the message is only sent to itself under the entire UDPIIP stack.

The next step 516 is to have CCScam use the multi-directional broadcast message address to transmit the message to the node of the regional language by transmitting the message to UDP, and to enable CCScam to transmit the peer-to-peer message using the appropriate address of the relay node in the non-regional language, UDP. The layer then uses UDP multi-directional propagation as needed to format the message and transmit it to the IP to transmit it to the destination node, to reach the CCNode object via IP&UDP for the destination node's transit node.

The destination node of the relay node checks the relay bit of the file header and knows that it is a relay message, and then calls its CCScam object to multi-directionally propagate the message to other members of the group in its language. The relay node is unreliable. The message transmission is only required to send the acknowledgement back to the transmitting node. As described above, the unreliable message transmits the message without the approval of the requesting relay node, but includes the approval request of the original transmitting node.

By means of the transmission of the message, the preferred embodiment then accepts the approval of all destination nodes before transmitting the secondary message of the object queue, whereby the preferred embodiment ensures reliable transmission and additional assurance of receipt. The message has been sorted in each message queue. This provides the required reliability and sequencing to increase multi-directional propagation quickly to effectively support cluster communication while providing resiliency to extend the cluster beyond a single language, therefore, reference is made to method 600 of Figure 7 to illustrate the acknowledgement message to ensure that the message is transmitted. Correct order

The first step 602 causes all nodes to receive a message to increment the desired sequence number of the message. As described above, each node includes its own node object to include an array of desired numbers, and the numbers in the array include the next sequence number. It is the node object that is expected to be obtained from the corresponding message queue. The node object only receives messages that are equal to or greater than the expected sequence number of the transmitted message queue.

Next step 604. All nodes today receive a message to transmit the acknowledgement message back to the transmitting node, which can be executed by calling the transfer authorization method node on the CCSubnet, and the CCSubnet then calls the transport packet header on the CCScam object. As described above, the method of transmitting the packet header is transmitted to the original transmitting node in the form of a packet header message, and the message is transmitted to the CCNode object in a point-to-point manner for the receiving node of the original transmitting node.

The next step 606 causes the CCNode object of the transmitting node to update the last message approval number of the corresponding message queue when receiving the approval, and the CCNode object also increments the approval counter of the CCSubnet object corresponding to the receiving node, and continues to all CCNodd. After receiving the approval and reducing the recognition counter to 0, if the approval counter of a message queue has not been reduced to 0 after a preset time, the next step 608 causes the CCSubnet object to retransmit the message to all nodes that are not recognized for reception. The CCServiceSubnet object polls the node objects of the nodes in the group to determine which nodes do not recognize the last message. The CCServiceSubnet object then transmits the message to any unrecognized node by calling the appropriate method on the CCScam object. The call can be a peer-to-peer message. , or if many nodes do not recognize receipt, a new multi-directional propagation can be performed.

When the acknowledgement counter of a message queue has been reduced to 0, this means that all nodes have accepted the receipt, and the next step 610 causes the CCSubnet object to decrement the subnet counter to one, that is, the CCGroup object that originally transmitted the message. When the CCSubnet object that originally transmitted the message has caused the CCGroup object subnet waiting counter to decrease by one, the counter becomes 0. For the CCGroup object, this means that the message has been successfully transmitted to all nodes of the multi-directional propagation group, and the corresponding message can be transmitted. The next message of the column.

This step 614 releases the CCMessage object corresponding to the transmitted message, which sends the message back to the available CCMessage object area, and then calls the appropriate method on the cluster communication service object to transmit the CCOrderedConnection message to the next message, which makes the process Returning to step 504 of method 500

A cluster communication system is disclosed in accordance with the present invention to support reliable and rapid cluster communication. The cluster communication system of the preferred embodiment can be used to provide this reliable and rapid cluster communication for the cluster configuration to extend beyond a single area network (LAN). "The cluster communication system is provided by multi-directional communication between cluster-friendly systems. Reliable and rapid cluster communication. In particular, the preferred embodiment provides for the establishment of a cluster between the provision of multi-directional communication messages. The preferred embodiment provides this multi-directional propagation while providing the required mechanisms to ensure inter-system routing. The preferred embodiment extends this fast and reliable cluster communication by providing additional point-to-point communication between systems on non-identical LANs. Thus, the preferred embodiment provides for the use of reliable multi-directional propagation for rapid cluster communication. The cluster communication system is thus used by the cluster to extend beyond the single area network.

While the invention has been particularly shown and described with reference to the preferred embodiments of the present invention, it is understood that various changes in form and detail may be made without departing from the spirit and scope of the invention.

Figure 1 is a block diagram of an apparatus in accordance with a preferred embodiment of the present invention;

Figure 2 is a schematic diagram illustrating a cluster;

3 is a schematic diagram of a cluster management application in accordance with a preferred embodiment of the present invention;

4 is a schematic diagram of a cluster topology server in accordance with a preferred embodiment of the present invention;

Figure 5 is a diagram of a cluster communication server in accordance with a preferred embodiment of the present invention;

Figure 6 is a flow chart illustrating a method of transmitting a message in accordance with a preferred embodiment of the present invention;

Figure 7 is a flow chart illustrating an authentication and clustering message method in accordance with a preferred embodiment of the present invention.