Migrating middlebox state for distributed middleboxes

Many companies today have large and sophisticated networks that include switches, hubs, routers, middleboxes (eg firewalls, load balancers), source network address translators. (Transformers), etc.), servers, workstations and other network devices, which support a variety of connections, applications and systems. More advanced computer networking demands a paradigm for network control. This computer networking includes virtual machine migration, dynamic workloads, multi-tenancy, and customer-specific quality of service and security configuration. Networks are traditionally managed through low-level configurations of individual network components. Network configurations often depend on the networks underneath, for example blocking a user's access by an access control list ("ACL") entry requires that they know the user's current IP address. To do. More complex tasks require broader network knowledge, and passing guest user port 80 traffic through an HTTP proxy keeps track of the current network topology and the location of each guest. Demand to grasp. This process is more difficult than sharing network switching elements across multiple users.

In response, there is an expanding movement towards a new network control paradigm called Software Defined Networking (SDN). In this SDN paradigm, a network controller running on one or more servers in the network controls, maintains, and implements the control logic. This control logic manages the transfer behavior of the shared network switching element on a user-by-user basis. Making network management decisions often requires knowledge of network conditions. To facilitate making management decisions, the network controller provides an application programming interface that creates and maintains a network state display, and allows management applications to access the net arc state display.

Some of the key objectives of maintaining large networks (including both data centers and corporate networks) are scalability, mobility, and multitenancy. Many methods that address one of these goals interfere with at least one of the other goals. For example, one can easily provide network mobility for virtual machines within an L2 domain, but the L2 domain cannot be resized to a large size. Also, maintaining a large degree of user isolation complicates mobility. As such, there is a need for improved solutions that can meet the goals of scalability, mobility, and multitenancy.

In some embodiments, the user includes one or more middleboxes (eg, firewall, load balancer, source network address translator, intrusion detection system (IDS), wide area network optimizer, etc.). Provide a system that makes it possible to identify a logical network. The system logically distributes logical transfer elements (eg, logical switches, logical routers, etc.) across several management switching elements running on several physical machines that also manage virtual machines in the logical network. Realize the network. In implementing such a logical network, the systems of some embodiments implement different middleboxes in different ways. For example, the system implements a first middlebox in a distributed manner (eg, using a middlebox implemented across several managed middlebox elements running on a physical machine in parallel with the management switching elements). And can also implement a second middlebox in a centralized way (eg, as a single electronic device or virtual machine, as a cluster). In some embodiments, the determination of whether to achieve a particular middlebox in a distributed or centralized manner is based on state sharing requirements between different middlebox elements when the middlebox is distributed. ..

In some embodiments, the spectrum for possible implementations of the logical middlebox on the physical network ranges from a fully distributed middlebox to a fully centralized middlebox. In addition, a single type of middlebox can be implemented in both centralized and distributed forms, including within the same management logical network. For example, a user wants a first firewall to filter all incoming traffic from the external network and a second firewall to filter traffic between different subnets of the logical network. In some cases. In some cases, the best solution can implement the firewall as a single electronic device that forwards all incoming traffic, while the physical machines that manage the virtual machines of the logical network. It is possible to realize a second firewall in a distributed format that covers all of the above.

Once in this spectrum, it is a fully distributed middlebox architecture. In this case, the middlebox is implemented across several nodes (physical host machines). Each of the physical host machines, in some embodiments, manages at least one virtual machine in a logical network that includes a logical middlebox. In addition, the management switching element operates to realize the logical transfer element of the logical network in each of the host machines. Because a particular physical host machine can manage a set of virtual machines within multiple logical networks (eg, belonging to different tenants), the distributed middlebox and management switching elements running on the host are different logical networks. It can be virtualized to realize the middlebox group and the logical transfer element group.

In some embodiments, the middlebox can be achieved in a distributed form when a minimal shared state (or a fully unshared state) is required between the middlebox instances. At least some types of middleboxes are stateful, in which case they are between machines (eg, between two virtual machines in the network, between virtual machines in the network and external machines, etc.). Establish a state for the connection at. In some embodiments, the middlebox establishes a state for each transport layer connection (eg, TCP connection, UDP connection). For distribution in some embodiments, a middlebox element running on a particular host machine creates a state for transport connections that pass through it, but other middleboxes running on other host machines. There is no need to share state with the element. If you want to apply state only to virtual machines managed on a particular host machine and you do not need to perform any analysis with the state information that the middlebox has established for other virtual machines. The middlebox can be distributed. Examples of such middleboxes include source network address translation (S-NAT), destination network address translation (D-NAT), and firewalls.

In addition, some embodiments allow distribution of the middlebox with minimal levels of state sharing. For example, a load balancer queries a group of machines to balance the traffic, determine the current level of traffic sent to each of the groups of machines, and distribute it to other load balancers. However, each load balancing element can execute a load balancing algorithm to execute queries at regular intervals. This is because each other load balancing factor and state information each time a packet is forwarded to one of the virtual machines, or each time a transport (eg TCP, UDP, etc.) connection is established with one of the virtual machines. Is not something to share.

The other side of the spectrum is a fully centralized middlebox implementation. In such a centralized implementation, the management switching element within the host sends all traffic to the middlebox for processing to the same middlebox electronics. This single middlebox is a separate physical machine or a separate virtual machine that runs within a physical network within your own host machine (or within the same host as one of multiple virtual machines in your network). Can be. If the management switching element identifies that the packet should be sent to the middlebox, the switching element sends the packet over the physical network (eg, through a tunnel) to the middlebox. The middlebox processes the packet and sends the packet (actually a new packet) to another management switching element for processing (eg, a pool node).

Some embodiments use such a centralized middlebox when the distributed middlebox requires packet sharing at data plane speeds. That is, for each traffic packet processed by the Middlebox element, this Middlebox element would have to update all other Middlebox instances with the state changes resulting from packet processing. That is, each traffic packet passing through the Middlebox will spike in additional traffic to update all of the other Middlebox instances. Examples of such middleboxes include IDS, and WAN optimizers. For example, in order to properly monitor intrusions, IDS operations need to know almost every connection in the network. Thus, if the IDSs are distributed, new state updates will need to be sent for each packet processed by the distributed IDS elements.

As a third option, some embodiments use a cluster architecture for some middleboxes that resembles a fully centralized architecture, and the cluster operates as a centralized resource pool rather than a single physical machine. Middlebox clusters (eg, IDS box clusters) are useful when the networks (or networks) that use the middlebox are larger, and a single electronics has enough resources to handle larger deployments. It may not have (for example, memory, processing power, etc.). However, if the cluster is a middlebox that requires all knowledge of state information, this state information will be shared among the various machines in the cluster. In some embodiments, the middlebox cluster does not require state updates on a packet-by-packet basis, rather than on a per-transport connection (or less frequent per-packet frequency, some updates per connection) unit. Is a better option than a single electronic device. To perform the required high-speed state sharing, some embodiments link with a middlebox machine in the cluster via a separate dedicated high-speed connection for sharing state information.

The above summary is provided as a brief introduction to some embodiments of the present invention. It does not mean that all of the invention-specific matters disclosed in the present specification are introductions or summaries. The detailed description that follows and the drawings that are referenced in the detailed description further illustrate the embodiments described in the abstract and other embodiments. Therefore, a complete reference to the abstract, detailed description and drawings is required to understand all the embodiments described herein. Also, the particulars to be claimed are not limited by the abstract, detailed description and exemplary details in the drawings, but rather are defined by the accompanying claims. This is because the particular matter requested can be implemented in other particular forms without departing from the spirit of that particular matter.

The novel features of the present invention will be described in the appended claims. However, for purposes of illustration, some embodiments of the invention are illustrated in the figures below.<figref num="1">It is a diagram conceptually showing the spectrum for the implementation of a logical middlebox on a physical network, ranging from a fully distributed middlebox to a fully centralized middlebox.</figref><figref num="2">It is a figure which conceptually shows the logical network topology of some embodiments.</figref><figref num="3">It is a figure which conceptually shows the implementation of the distributed middlebox of some embodiments.</figref><figref num="4">It is a figure which conceptually shows the fully centralized implementation of the middlebox of some embodiments.</figref><figref num="5">It is a figure which conceptually shows the implementation of the middlebox as a cluster of resources of some embodiments.</figref><figref num="6">It is a figure which conceptually shows the network which realizes the logical network including the intrusion detection system of some embodiments.</figref><figref num="7">It is a diagram conceptually showing the architecture of the host machine of some embodiments including both the distributed software middlebox element and the software switching element.</figref><figref num="8">It is a figure which shows the network control system of some embodiments for configuring the management switching element and the distributed middlebox element in order to implement a logical network.</figref><figref num="9">It is a figure which conceptually shows the propagation of data through the network control system of some embodiments.</figref><figref num="10">It is a figure which shows the example architecture of the network controller of some embodiments.</figref><figref num="11">It is a figure which conceptually shows a complicated logical network topology which intervenes several middleboxes.</figref><figref num="12">It is a diagram conceptually showing one specific physical implementation of the network of FIG. 11 in a host virtualization environment.</figref><figref num="13">It is a figure which conceptually shows the electronic system which realizes some embodiments of this invention.</figref>

The following detailed description of the invention describes and describes some details, examples and embodiments of the invention. However, those skilled in the art will appreciate that the invention is not limited to the embodiments described and that the invention may be realized without some specific details and examples discussed. It will be obvious and obvious.

Some embodiments provide a system that allows a user to identify a logical network. This logical network contains one or more middleboxes (eg, firewalls, load balancers, network address translators, intrusion detection systems (IDS), wide area network (WAN) optimizers, etc.). This system realizes a logical network by distributing logical transfer elements (for example, logical switches, logical routers, etc.) across several management switching elements. This management switching element runs on several physical machines that are also the host virtual machines for that logical network.

In the realization of such a logical network, the systems of some embodiments realize different middleboxes in different ways. For example, the system implements the first middlebox in a distributed format (eg, using a middlebox implemented across several management middleboxes running on a physical machine in parallel with a management switching element). A second middlebox can be implemented in a centralized format (eg, as a single electronic device or as a virtual machine, as a cluster). In some embodiments, the determination of whether to achieve a particular middlebox in a distributed or centralized format is based on state sharing requirements with different middleboxes when the middlebox is distributed. ing.

FIG. 1 is a diagram conceptually showing a spectrum 100 for the implementation of a logical middlebox on a physical network, ranging from a fully distributed middlebox to a fully centralized middlebox. As mentioned above, different middleboxes can be realized at different points along this spectrum. In addition, a single-type middlebox can be implemented in both centralized and distributed formats, including within the same management logical network. For example, a user may want a first firewall to filter all traffic coming from an external network and a second firewall to filter traffic between different subnetworks of a logical network. .. In some cases, the best solution is to implement the first firewall as a single electronic device to which all external traffic is forwarded, while the physical realization of a virtual machine in a logical network. Achieving a second firewall in a distributed format across all of the machines.

At the left end of spectrum 100, the fully distributed middlebox architecture 105 of some embodiments is shown. As shown in the figure, the middlebox is implemented across several nodes (physical host machines). In some embodiments, each of the physical host machines manages at least one virtual machine within a logical network that includes a logical middlebox. In addition, the management switching elements operate on each of the host machines to implement the logical transfer elements of the logical network (eg, logical routers, logical switches). A particular physical host machine can manage virtual machines within two or more logical networks (eg, belonging to different tenants), so both the distributed middlebox and the management switching elements running on the host Can be virtualized to implement middlebox and logical transfer elements from different logical networks. In some embodiments, the middlebox is implemented as a module or set of modules that acts as a hypervisor for the host machine.

As shown in the figure, Middlebox can be achieved in a distributed format when a minimal shared state (or not fully shared state) is required between Middlebox instances. At least some types of middleboxes are stateful, in which case they are between machines (eg, between two virtual machines in the network, between virtual machines in the network and external machines, etc.). Establish a state for the connection at. In some embodiments, the middlebox establishes a state for each transport layer connection (eg, TCP connection, UDP connection). For distribution in some embodiments, a middlebox element running on a particular host machine creates a state for transport connections that pass through it, but other middleboxes running on other host machines. There is no need to share state with the element. If you want to apply state only to virtual machines managed on a particular host machine and you do not need to perform any analysis with the state information that the middlebox has established for other virtual machines. The middlebox can be distributed. Examples of such middleboxes include source network address translation (S-NAT), destination network address translation (D-NAT), and firewalls.

In addition, some embodiments allow distribution of the middlebox with minimal levels of state sharing. For example, a load balancer queries a group of machines to balance the traffic, determine the current level of traffic sent to each of the groups of machines, and distribute it to other load balancers. However, each load balancing element can execute a load balancing algorithm to execute queries at regular intervals. This is because each other load balancing factor and state information each time a packet is forwarded to one of the virtual machines, or each time a transport (eg TCP, UDP, etc.) connection is established with one of the virtual machines. Is not something to share.

The other side of spectrum 100 is a fully centralized middlebox implementation 110. In this implementation, the management switching element in the host sends all traffic to the middlebox for processing to the same middlebox electronics. This single middlebox is a separate physical machine or a separate virtual machine that runs within a physical network within your own host machine (or within the same host as one of multiple virtual machines in your network). Can be. If the management switching element identifies that the packet should be sent to the middlebox, the switching element sends the packet over the physical network (eg, through a tunnel) to the middlebox. The middlebox processes the packet and sends the packet (actually, a new packet) to another management switching element for processing.

In some embodiments, the management switching element to which the new packet is transmitted is the pool node. Pool nodes are for handling traffic that, in some embodiments, edge switching elements (ie, they are located on the host machine and are directly connected to the virtual machine) cannot handle it. , A particular type of management switching element that is located inside the network (ie, not directly attached to any virtual machine at the edge of the network). In another embodiment, instead of sending the packet to the pool node, the middlebox sends the traffic to the management switching element built directly on the middlebox electronics.

Some embodiments use such a centralized middlebox when the distributed middlebox requires packet sharing at data plane speeds. That is, for each traffic packet processed by the Middlebox element, this Middlebox element would have to update all other Middlebox instances with the state changes resulting from packet processing. That is, each traffic packet passing through the Middlebox will spike in additional traffic to update all of the other Middlebox instances. Examples of such middleboxes include IDS, and WAN optimizers. For example, in order to properly monitor intrusions, IDS operations need to know almost all network connections. Thus, if the IDSs are distributed, new state updates will need to be sent for each packet processed by the distributed IDS elements.

The Middlebox cluster architecture 115 is similar to the fully centralized architecture 110, except that it operates as a centralized resource pool rather than a single physical machine. As shown in the figure, middlebox clusters (eg, IDS box clusters) are useful when the networks (or networks) that use the middlebox are larger, and a single electronics is a larger deployment. You may not have enough resources (eg, memory, processing power, etc.) to handle. However, if the cluster is a middlebox that requires all knowledge of state information, this state information will be shared among the various machines in the cluster. In some embodiments, the middlebox cluster does not require state updates on a packet-by-packet basis, rather than on a per-transport connection (or less frequent per-packet frequency, some updates per connection) unit. Is a better option than a single electronic device. To perform the required high-speed state sharing, some embodiments link with a middlebox machine in the cluster via a separate dedicated high-speed connection for sharing state information.

The above description provides examples of logical middleboxes for various implementations within the network of some embodiments. Some of the more detailed embodiments will be described below. Section I describes several different embodiments of the Middlebox architecture. Section II describes distributed middlebox implementations for some embodiments. Section III then describes a network control system of several embodiments for configuring the network to implement a logical network containing one or more middleboxes. Section IV provides an example of a network with several different middleboxes. Section V then describes an electronic system in which some embodiments of the present invention are realized.

I. Various middlebox architectures As mentioned above, some embodiments implement different middleboxes using different architectures within the management network. Even for the same logical network topology, some middleboxes are implemented in a distributed format (using middlebox elements that run on each host running the network's virtual machine), while other middleboxes. Is implemented in a centralized manner (using a single electronic device or a cluster of management switching elements on the host).

FIG. 2 is a diagram conceptually showing the logical network topology 200 of some embodiments. Network Topology 200 is a simplified network for illustration purposes. This network contains two logical L2 switches 205 and 210, which are connected to the logical L3 router 215. The logical switch 205 is connected to the virtual machines 220 and 225, while the logical switch 210 is connected to the virtual machines 230 and 235. The logical router 215 is also connected to the external network 250.

In addition, the middlebox 240 is connected to the logical router 215. Those skilled in the art will recognize that the network topology 200 merely represents one particular logical network topology in which the middlebox can be incorporated. In various embodiments, the middlebox may be located directly between two other components, or it monitors and handles all traffic entering and exiting the gateway and logical router (eg, logical network). It may be located directly between (for) and in other locations within a more complex network.

In the architecture shown in Figure 2, the Middlebox 240 is not located in the direct traffic flow from one domain to another, or between the outside world and the domain. Therefore, for the logical router 215, which determines which packet should be sent to the processing middlebox, the packet is sent to the middlebox until a routing policy is identified (by a user such as a network administrator). There is nothing. Some embodiments may use policy routing rules that transfer data based on data that exceeds the destination address (eg, destination IP or MAC address). For example, the user, for example, through the network controller application programming interface (API), all packets containing the source IP address in the logical subnet switched (switched) by the logical switch 205, or are switched by the logical switch 210. It can be identified that all packets entering the network from the external network 250 destined for the (switchable) logical subnet should be destined for the processing middlebox 240.

Different middleboxes can perform different functions within the network. For example, the firewall parses a data packet to determine if it should be allowed to pass (ie, similar to an ACL flow entry). The firewall remembers a set of rules (eg, entered by the user). This set of rules tells the firewall whether to drop the packet (ie, drop it) or allow the packet to pass (or, in some cases, drop the packet and give an error response to the source. (Reject the packet by replying) Judge. In some embodiments, the firewall is a stateful firewall that keeps track of transport (eg, at least one of TCP and UDP) connections and is stored for faster packet processing decisions. Use state information.

Source Network Address Translation (S-NAT) changes the source IP address of a packet in the packet header. For example, using S-NAT to hide the IP addresses of several different machines with different IP addresses from the destination machine by changing the source of packets from different machines to one IP address. Can be done. Destination Network Address Translation (D-NAT) also modifies the destination IP address of a packet to hide the actual IP address from the source machine. Load balancing takes the form of D-NAT, which uses various algorithms (eg, round robin, random allocation, etc.) to balance traffic across several destination machines. The load balancer receives a packet for a particular IP address exposed to the source machine, modifies the destination IP address of the packet, and matches it against a particular one of the destination machines selected by the load balancing algorithm.

An intrusion detection system (IDS), in some embodiments, is a passive middlebox that monitors a logical network for malicious behavior or policy violations. IDS can verify transport connections (eg TCP connections, UDP connections, etc.) to determine if an attack on the network is occurring.

A WAN optimizer is a middlebox device for improving the efficiency of data transfer across a WAN (eg, facilitating the flow of data across a WAN). Examples of WAN optimization techniques include data deduplication, data compression, latency optimization, cache and proxy processing, forward error correction, protocol spoofing, traffic shaping, Includes equalizing, connection limits, simple rate limits, etc. While the above content is some list of several different middleboxes, it is noted that some embodiments can include various different middleboxes that can be realized in distributed or centralized form. , Those skilled in the art will recognize.

Depending on the type of middlebox, and in some cases depending on the type of implementation required by the user, the middlebox as shown in Figure 2 is implemented in either centralized or distributed form. It will be. FIG. 3 conceptually illustrates the distributed implementation 300 of some embodiments. In particular, FIG. 3 shows several nodes, including a first host machine 305, a second host machine 310, a third host machine 315, and an Nth host machine 320. There is. Each of the first three nodes manages several virtual machines on network 200, virtual machine 220 is managed on the first host machine 305, and virtual machines 225 and 235 are managed on the second host machine 310. The virtual machine 230 is managed on the third host machine 315.

In addition, each of the host machines contains a management switching element (MSE). The management switching element of some embodiments is a software transfer element that implements a logical transfer element for one or more logical networks. For example, the MSE in hosts 305-320 contains a flow entry in the forwarding table that implements the logical forwarding element of network 200. Specifically, the MSE on the host machine implements the logical router 215 in addition to the logical switches 205 and 210. On the other hand, only some embodiments implement a logical switch on a particular node when at least one virtual machine connected to the logical switch is located on that particular node (ie, MSE on host 305). Only implement logical switch 205 and logical router 215 within). Since the Nth host 320 does not contain any virtual machine from network 200, the MSE residing on the host does not implement any logical transfer element from network 200.

Implementation 300 of some embodiments also includes a pool node 340 that connects to a host machine. In some embodiments, the host-resident MSE performs the first hop process. That is, these MSEs are the first forwarding element that the packet arrives at after being sent from the virtual machine, and also attempts to perform all of the logical switching and logical routing in this first hop. However, in some cases, a particular MSE may not remember a flow entry that contains all of the logical transfer information for the network, so it may not be possible to know what to do with a particular packet. In some such embodiments, the MSE sends the packet to pool node 340 for further processing. This pool node is an internal management switching element that, in some embodiments, stores a flow entry that covers a larger portion of the logical network than the edge software switching element.

Similar to the distribution of logical switching elements across hosts on which network 200 virtual machines reside, the middlebox 240 is distributed across the middlebox elements on these hosts 305-315. In some embodiments, the middlebox module (or set of modules) resides on the host machine (eg, operates within the host's hypervisor). When a user sets up a logical network (eg, Network 200), its input includes configuration from the middlebox. For example, for firewalls, the user will enter a set of rules for packet filtering (eg, based on IP address, TCP connection, etc.). In some embodiments, the network control system used to provide the management switching element to implement the logical transfer element is also used to provide the various middlebox elements that operate on the host machine. be able to. When a user inputs a middle box configuration to a controller in a network control system, this controller identifies a particular set of nodes for which the middle box configuration should be implemented and delivers that configuration to these nodes. (For example, through a set of controllers).

If one of the virtual machines sends a packet (eg, to another of the virtual machines, to an external address, etc.), the packet first proceeds to the locally managed switch element for processing. The MSE can use its own stored flow entries to make forwarding decisions to send the packet to the middlebox, and in some embodiments the packet is sent to the local middlebox on the same host. Send to element. In some embodiments, the middlebox element and MSE negotiate a software port for sending packets with minimal delay. After the middlebox processes the packet, some embodiments send the packet back to the MSE through this same port. In some embodiments, this packet is transmitted from the middlebox to the MSE as a new packet. Therefore, MSE requests new processing. However, in some situations the packet will not be returned. For example, if the middlebox is a firewall, the middlebox blocks or drops packets. In addition, some embodiments of the middlebox are passive, and packet duplications are sent in sequence to the middlebox to keep track of statistics, but not back to the switching element.

Figure 3 shows only one logical network implemented across hosts 305-320, while some embodiments span several logical networks across a set of hosts (eg, different tenants). To) is realized. In this way, the middlebox element on a particular host can actually store the configuration for several different middleboxes belonging to several different logical networks. For example, firewall elements may be virtualized to implement two (or more) different firewalls. They will effectively act as two separate middlebox instances, in which the middlebox elements will be sliced (split) into several (same type) "virtual" middleboxes. In addition, when the MSE on the host sends a packet to the middlebox, some embodiments add (prepend) a slice identifier (or tag) to the packet to provide some virtual middleboxes. Identify from which virtual middlebox the packet is being sent. When multiple middleboxes are implemented on the same middlebox element for a single logical network (eg, two different load balancers), the slice identifier is a particular middlebox rather than the logical network to which the packet belongs. It is necessary to identify the slice of the box. Different embodiments can use different slice identifiers for the middlebox.

Examples of middleboxes that can be dispersed in some embodiments include firewalls, S-NATs, and load balancers. In each of these cases, the middlebox plays an active role in packet processing (ie, the S-NAT and load balancer modify the packet's source and destination addresses, respectively, while The firewall decides whether to allow or drop the packet). However, each of these middlebox elements on a particular node can function on its own without requesting information from the corresponding middlebox elements on other nodes. Even distributed load balancer elements span different virtual machines, assuming that no virtual machine is likely to be overloaded unless other load balancer elements use the same algorithm. It is possible to load balance the incoming traffic individually. Nevertheless, in some embodiments, the load balancer element will share state at several levels (eg, after querying the destination virtual machine for usage and health statistics).

As mentioned above, FIG. 3 conceptually illustrates a distributed implementation of some embodiments for the middlebox 240 of the logical network 200. On the other hand, FIG. 4 conceptually shows a fully centralized implementation 400 of the middlebox 240. This implementation includes several nodes 405-420 that manage the virtual machines 220-235, as in the distribution example in Figure 3. These virtual machines are managed here by a first virtual machine 220 managed by node 405, a second virtual machine 225 and a fourth virtual machine 235 managed by node 410, and node 415. It is arranged with the third virtual machine 230. Similarly, the management switching element located at nodes 405-415 implements logical transfer elements 205-215. As in the distributed middlebox example, the management switching element performs a first hop operation on packets originating from the virtual machine.

However, in this example, the middlebox 240 is not distributed across hosts 405-415. Instead, the middlebox is realized as a single machine outside the host. In different embodiments, for different types of middlebox, this single middlebox can be a physically single electronic device (eg, a separate physical device), or a single virtual machine (actually, a single virtual machine). It can run on one host machine in a group of host machines, or on another host machine). For example, some embodiments may provide a first type of middlebox (eg, WAN optimizer) as a virtual machine, while a second type of middlebox (eg, eg, WAN optimizer) as a single electronic device. IDS) can be provided. In addition, some embodiments may also provide a set of options for a single type of middlebox.

Similar to the distributed Middlebox, in some embodiments, the network control system is used to provide a centralized Middlebox electronic device or virtual machine. Rather than a controller that receives configuration information and identifies some nodes to which the configuration information should be distributed, some embodiments of the controller identify the electronics in which the middle box is realized. Deliver the configuration to the electronics (eg, through an intermediate controller that manages the electronics). In some embodiments, some physical electronics are present in the physical network, and the controller selects one of these electronics to implement the middlebox. When the middlebox is implemented as a virtual machine, some embodiments select a host node for the virtual machine and deliver the configuration to that host node. In either case, the network control system identifies the connection or connection between the middlebox and the various management switching elements on the host. In some embodiments, the middlebox electronics support one or more types of tunneling, and the flow entry delivered to the management switching element uses tunnel encapsulation to send the packet to the middlebox. Contains a set of entries to identify.

If the flow entry in the management switching element specifies that it will send traffic to the middlebox, the management switching element also uses this tunnel information to encapsulate the packet, non-host packets of its own. Send to the middlebox through the tunnel. Like the distributed middlebox, some centralized middleboxes are active middleboxes. That is, the middlebox returns the packet to the network after executing the processing of its own middlebox. In some embodiments, such a middlebox is configured to always send a packet (as a new packet) to a pool node (eg, of some pool nodes, always the same pool node). It is set. In Figure 4, the centralized middlebox 425 sends all of its outgoing traffic to pool node 430. The pool node, which also implements the logical transfer elements 205-215, forwards the packet to the appropriate destination machine.

The distributed middlebox element may be virtualized to run middlebox instances for several different logical networks, as well as the centralized middlebox 425. Physically identical electronic devices (or virtual machines) can be used by several different logical networks. In some embodiments, slicing techniques similar to those used in distributed architectures are used. That is, the management switching element adds a tag to indicate the logical network (or specific logical middlebox within the logical network) to which the packet is sent, and the middlebox 425 uses this tag to send the packet. Identifies which middlebox instance should be used for processing. In some embodiments, the centralized middlebox electronics include several ports, each of which is assigned a different virtual middlebox instance. In such an embodiment, the slicing technique is not used and instead the incoming port is used to identify the correct virtual middlebox.

Whereas the middlebox 425 is a single resource, in some embodiments the middlebox is implemented as a centralized cluster of resources, as shown in Implementation 500 of FIG. This example is identical to that shown in Figure 4, except for a single Middlebox device, where the network contains a Middlebox cluster 505 with three Middlebox resources 510-520. In some embodiments, each of the Middlebox resources 510-520 is a separate device or virtual machine (ie, the equivalent of Middlebox 425).

In different embodiments, different architectures can be used within the Middlebox cluster. Some embodiments include an entry point (eg, a single physical device) into a cluster that load balances packets across a resource pool. Other embodiments have different host machines that are directly connected to different resources in the cluster. For example, network 500 may be set up with a first host machine 405 connected to resource 510, a second host machine 410 connected to resource 515, and a third host machine 415 connected to resource 520. Can be done. Another embodiment uses a master backup setup in which the cluster has two devices. All host machines are connected to the master, which shares state data with backup resources while performing middlebox processing.

As described with reference to FIG. 2, some embodiments use a centralized middlebox implementation when state sharing is required on a packet-by-packet basis. That is, for a given middlebox, middlebox processing requires all knowledge of the packets processed by the middlebox. For distributed middleboxes, this would lead to a surge in state updates sent over the network connected to the middlebox elements. However, if the middlebox is a single electronic device, this single electronic device will always store all of its states.

For middlebox clusters like cluster 505, this requirement means that state must be shared among middlebox resources in the cluster at high speed. Some embodiments use a dedicated connection between Middlebox resources to share this state information. That is, a particular port on each Middlebox device is dedicated only to state sharing between several devices in the cluster. Sometimes the middlebox cluster will be only two machines, or just a few machines, and they will work close together to make a dedicated connection more feasible. For situations with three or more middleboxes, some embodiments use a mesh network. In this network, each of the middlebox resources broadcasts state updates over the network to all other middlebox resources. Another embodiment uses a star network. In this network, a middlebox resource sends its state updates to a central resource, which merges the updates and sends them to other resources. While middlebox clusters require this additional infrastructure compared to centralized, this cluster has the advantage of being able to handle larger deployments when a larger number of packets are being processed. doing.

As mentioned above, the WAN optimizer and intrusion detection system are examples of middleboxes, and some embodiments are implemented as centralized middleboxes. This is because of the state sharing requirements. WAN optimizers use, for example, various optimization techniques to improve the efficiency of data transfer over the WAN. Performing these optimization techniques requires all access to traffic sent across the WAN, which means that centralized implementations are more optimal. The WAN optimizer can also be used to cache content sent across the WAN, which caches are stored together rather than being distributed across several hosts. If so, it only works to achieve that goal.

The intrusion detection system is a passive system (ie, does not drop or modify packets) and monitors the total number of connections, the addresses in these connections, the number of packets for each connection, and so on. In order to detect intrusions, IDS searches for patterns by connections, heuristics, etc., whereas IDS processing must be aware of all monitored traffic. If one distribution element has information about the first connection and the second distribution element has information about the second connection, then each element properly evaluates the network against intrusions. I don't have enough information to do this.

FIG. 6 conceptually illustrates a network 600 that implements a logical network that includes an intrusion detection system 625 in some embodiments. The logical network contains four virtual machines managed across three nodes 605, 610 and 620. The logical topology of the logical network in this case is the same as that shown in Figure 2 (with IDS as a middlebox). However, the aspects of the IDS implementation described herein apply equally to other network topologies. Network 600 contains a fifth node 615, which does not manage any virtual machine in a particular logical network (but manages at least one virtual machine from another network).

Unlike the figure above, FIG. 6 shows a connection with an arrow pointing the direction of packet forwarding between machines in the system. Thus, for example, all of hosts 605-620 can send packets to each other. That is, the virtual machine at host 605 sends the packet to the virtual machine at host 620 (via the management switching element at host 605 and through the management switching element at host 620), and the management machine at host 620 sends the packet. Send to the virtual machine on host 605. In addition, all of the host's MSEs use pool nodes to handle packets for which the Edge MSE cannot make forwarding decisions.

Each of the host machines 605, 610 and 620 manages virtual machines from a particular logical network and sends packets to the IDS. In some embodiments, all traffic on the logical network is sent to the IDS. However, these arrows are unidirectional. This is because the intrusion detection system of some embodiments is a passive middlebox. Some embodiments send duplicate packets to the IDS box 625 instead of forwarding traffic through the middlebox. IDS receives these duplicate packets (ie, packets for each transmitted through at least one of the networks between hosts and external networks) and performs its intrusion detection analysis. Since the intrusion detection system 625 does not output any traffic packets, there is no required connection between IDS 625 and pool node 630 in this figure.

II. Distributed middlebox implementation As mentioned above, some embodiments implement one or more different middleboxes in a distributed form with middlebox elements that work with some or all of the host machines. For the implementation of the centralized middlebox in some embodiments, the virtual machine of the logical network and the management switching element are arranged on the host machine group. This section describes distributed middlebox implementations of some embodiments within the host machine.

FIG. 7 conceptually illustrates an architectural diagram of a host machine 700 of several embodiments, including both a distributed software middlebox element and a software switching element. The distributed software middlebox element may be a network address translation element, a firewall element, a load balancing element, or any other middlebox implemented in a distributed format.

In this example, the middlebox element contains three components on the host machine: the middlebox daemon 790, which runs in the user space of the host machine 700, and the middlebox, which runs in the kernel of the host machine 700. There is kernel module 795. This figure shows the distributed middlebox elements as two components for illustration purposes, but the middlebox daemon 790 and the middlebox kernel module 795 aggregate the middlebox elements running on the host machine 700. Is forming. The software switching element (in this example, the open virtual switch ("OVS")) contains three components: the OVS kernel module 745 running in the kernel of the host machine 700, the OVS daemon 765, and the OVS database (in this example). There is a DB) daemon 767, the latter two running in the user space of the host machine.

As shown in FIG. 7, host 700 includes hardware 705, kernel 720, user space 721, and VM785-795. Hardware 705 includes typical computer hardware, including processing units, volatile memory (eg, random access memory (RAM)), non-volatile memory (eg, hard disk drives, flash memory, optics). There are disks, etc.), network adapters, video adapters, or any other type of computer hardware. As illustrated, hardware 705 includes NICs 710 and 715, which, in some embodiments, are typical network interface controllers for connecting computer devices to a network.

As shown in FIG. 7, the host machine 700 includes kernel 720 and user space 721. In some embodiments, the kernel is the most basic component of an operating system, which operates on separate memory spaces and also manages system resources (eg, between hardware and software resources). (Communication) plays a role. On the other hand, the user space is a memory space in which all user mode applications can operate.

Kernel 720 in some embodiments is a software abstraction layer, which runs above hardware 705 and under any operating system. In some embodiments, kernel 720 performs virtualization functions (eg, to virtualize hardware 705 for some virtual machines running on the host machine). And kernel 720 is, in some embodiments, part of the hypervisor. Kernel 720 handles a variety of administrative tasks. This includes memory management, processor scheduling, or any other operation to control the execution of VM735s and 738s running on the host machine.

As shown, kernel 720 includes device drivers 725 and 730 for NIC 710 and 715, respectively. Device drivers 725 and 730 allow the operating system (eg, on a virtual machine) to interact with the hardware of the host 700. In this example, device driver 725 allows interaction with NIC71, while driver 730 allows interaction with NIC715. Kernel 720 may include other device drivers (not shown) to allow the virtual machine to interact with other hardware (not shown) in the host 700.

Virtual machines 735 and 738 are independent virtual machines running on the host machine 700 (eg, user virtual machines as shown in Figure 3-Figure 6) using resources virtualized by kernel 720. .. In this way, the VM runs on any number of different operating systems. Examples of such operating systems include Solaris, FreeBSD, or any type of Unix-based operating system. Other examples include Windows®-based operating systems.

As shown, user space 721 of the host machine 700 includes daemon 790, OVS daemon 765, and OVS DB daemon 767. Other applications (not shown) may also be included in user space 721, including daemons for additional distributed middleboxes (eg, firewalls, load balancers, network address translators, etc.). OVS Daemon 765 is an application that runs in user space 721. The OVS daemon 765 of some embodiments communicates with the network controller 780 to receive instructions for processing and forwarding packets transmitted to and from the virtual machines 735 and 738, as described further below. .. The OVS daemon 765 in some embodiments communicates with the network controller 780 through an OpenFlow protocol, while in other embodiments it uses a different communication protocol for transferring physical control plane data. In addition, in some embodiments, the OVS daemon 765 has the network controller 780 OVS configuration information. After sending the DB daemon, get the configuration information from the OVS DB daemon 767.

In some embodiments, the OVS DB daemon 767 also runs in user space 721. The OVS DB daemon 767 of some embodiments communicates with the network controller 780 to configure an OVS switching element (eg, at least one of the OVS daemon 765 and the OVS kernel module 745). For example, the OVS DB daemon 767 receives configuration information from the network controller 780 and stores the configuration information in a set of databases. In some embodiments, the OVS DB daemon 767 communicates with the network controller 780 via the database communication protocol. In some cases, the OVS DB daemon 767 receives a request for configuration information from the OVS daemon 765. In these cases, the OVS DB daemon 767 gets the requested configuration information (for example, from a set of databases) and sends the configuration information to the OVS daemon 765.

The OVS daemon 765 includes an OpenFlow protocol module 770 and a flow processor 775. The OpenFlow protocol module 770 communicates with the network controller 780 to receive configuration information (eg, flow entry) for configuring software switching elements from the network controller 780. When module 770 receives configuration information from network controller 780, it translates that configuration information into information that can be interpreted by the flow processor 775.

The flow processor 775 manages the rules for processing and routing packets. For example, the flow processor 775 stores a set of rules received from the OpenFlow protocol module 770 (eg, in a storage medium such as a disk drive). In some embodiments, the rules are stored as a set of flow tables, each containing a set of flow entries. Flow processor 775 handles packets for integrated bridge 750 (discussed below) that do not have matching rules. In such a case, the flow processor 775 collates the rule stored in itself with the packet. If the packet matches a rule, the flow processor 775 sends the matched rule and its packet to the integrated bridge 750 for processing by the integrated bridge 750. In this way, when the integrated bridge 750 receives a similar packet that matches the generated rule, the packet will be matched against the exact matching rule generated in the integrated bridge 750, so that the flow processor 775 Eliminates the need to process packets.

In some embodiments, the flow processor 775 may not have a rule that matches the packet. In such cases, the flow processor 775 of some embodiments sends the packet to another management switching element (eg, a pool node) for handling the packet that cannot be processed by the edge switching element. However, in other cases, the flow processor 775 can receive from the network controller 780 a catchall rule that discards the packet if there is no rule in the flow processor 775 that does not match the packet.

As shown in Figure 7, kernel 720 contains the hibervisor network stack 740 and the OVS kernel module 745. The hypervisor network stack 740 is, in some embodiments, an Internet Protocol (IP) network stack 740. The hypervisor network stack 740 processes and routes IP packets. This IP packet is received from the OVS kernel module 745 and the PIF bridges 755 and 760. When processing packets destined for an external network host to host 700, the hypervisor network stack 740 determines which physical interface (PIF) bridges 755 and 760 the packet should be sent to. judge.

The OVS kernel module 745 processes and routes network data (eg, packets) between a VM running on host 700 and a network host outside of host 700 (eg, network received via NICs 710 and 715). data). In some embodiments, the OVS kernel module 745 implements a transfer table of physical control planes to one or more logical networks. To facilitate processing and routing network data, the OVS kernel module 745 communicates with the OVS daemon 765 (for example, to receive flow entries from the OVS daemon 765). In some embodiments, the OVS kernel module 745 includes a bridge interface (not shown) that sends packets to the hypervisor network stack 740 to the OVS kernel module 745 and also delivers packets to the OVS. Allows reception from kernel module 745.

Figure 7 shows the integrated bridge 750 and the OVS kernel module 745, including the PIF bridges 755 and 760. In some embodiments, the OVS kernel module 745 includes a PIF bridge for each NIC in hardware 705. In other embodiments, the PIF bridge within the OVS kernel module 745 can interact with multiple NICs in hardware 705. PIF bridges 755 and 760 route network data between the hypervisor network stack 740 and network hosts outside host 700 (ie, network data received via NICs 710 and 715).

The integrated bridge 750 processes and routes packets received from the network stack 740, VM735 and 738 (eg, via VIF), and PIF bridges 755 and 760. In some embodiments, the integrated bridge 750 stores a subset of the rules stored in the flow processor 775 (and / or a set of rules derived from the rules stored in the flow processor 775), which is integrated. The bridge 750 is currently using or most recently used to process and forward packets.

In some embodiments, the flow processor 775 in some embodiments is responsible for managing the rules group at the integrated bridge 750. In some embodiments, the integrated bridge 750 remembers only the active rule. The flow processor 775 monitors the rules stored in the integrated bridge 750 and sets active rules that have not been accessed for a given amount of time (eg, 1 second, 3 seconds, 5 seconds, 10 seconds, etc.). delete. In this method, the flow processor 775 manages the integrated bridge 750 so that the integrated bridge 750 remembers the rules in use or recently used.

Although Figure 7 shows one integrated bridge, the OVS kernel module 745 can include multiple integrated bridges. For example, in some embodiments, the OVS kernel module 745 includes a separate integrated bridge for each logical switching element, the logical switching element being implemented across the management network to which the software switching element belongs. Will be done. That is, the OVS kernel module 745 has a corresponding integrated bridge for each logical switching element implemented across the management network.

The above description relates to the transfer function of the management software switching element of some embodiments. While the software switching elements include a user space component that implements the control plane (OVS daemon 765) and a kernel component that implements the data plane (OVS kernel module 745), some embodiments have distributed middle box elements. Includes a control plane component (middle box daemon 790) that runs in user space and a data plane component (middle box kernel module 795) that runs in the kernel.

As shown, the middlebox daemon 790 includes a middlebox configuration receiver 791 and a middlebox configuration compiler 792. The middlebox configuration receiver 791 communicates with the network controller 780 to receive slice processing information in addition to the configuration for the middlebox. In some embodiments, the middlebox configuration is a set of records that describe the packet processing rules for the middlebox (for example, in the same format as the flow entry records received by the OVS daemon). For example, a firewall configuration contains a set of packet processing rules that describe when packets should be dropped, allowed, etc. (similar to ACL entries, but TCP connections as a factor for making decisions. Includes state). The source network address translation configuration contains a set of hidden IP addresses for the virtual machine that should be mapped to the IP addresses exposed by the transducer. In some embodiments, the load balancer configuration network-maps the exposed IP address to several different hidden virtual machine addresses, and TCP, which is new to any of the machines. Includes a load balancing (scheduling) algorithm to determine if a connection should be sent.

As mentioned above, the slicing information assigns an identifier to a particular middlebox instance to be executed by the distributed middlebox element. In some embodiments, this identifier is bound to a particular logical middlebox in a particular tenant's logical network. That is, if a particular logical network contains several different middleboxes with different processing rules, the middlebox daemon 790 will create several different middlebox instances. Each of these instances is identified by a different slice identifier on the packet sent to the middlebox. In addition, in some embodiments, the middlebox daemon 790 assigns a specific internal identifier to each of these instances, which the middlebox uses in its internal processing (eg, active TCP connection). To keep track of).

The middlebox daemon 790 also includes the middlebox configuration compiler 792. In some embodiments, the middlebox configuration compiler 792 receives middlebox configurations (eg, packet processing, modification or parsing rules) for a particular middlebox in the first language, which they receive in the middlebox. Compile to a set of rules for a second language that is optimal for your internal processing. The middlebox compiler 792 sends the compiled packet processing rules to the middlebox processor 796 of the middlebox kernel module 795.

The middlebox kernel module 795 processes packets sent to and / or sent from the VM running on the host 700 to determine whether to allow the packets to pass, discard the packets, and so on. As shown, the middlebox kernel 795 includes a middlebox processor 795 to perform these functions. The middlebox processor 795 receives the converted middlebox rules for a particular middlebox instance from the middlebox configuration compiler 792. In some embodiments, these transformed middlebox rules identify the packet processing pipeline within the middlebox.

To receive packets from the management switching element, the middlebox processor 796 of some embodiments connects with an abstract software port on the integrated bridge 750 of the OVS kernel module. Through this port on the integrated bridge, the management switching element sends packets to and receives packets from the middlebox after being processed by the middlebox (unless the middlebox discards packets). As mentioned above, these packets contain a slice identifier tag used by the Middlebox processor 796 to determine the set of compiled packet processing rules to apply to the packet.

The architectural diagram of the distributed middlebox and software switching elements shown in Figure 7 is an example of a configuration. Those skilled in the art will recognize that other configurations are possible. For example, in some embodiments, the middlebox processor that applies the compiled packet processing rules is located in user space 721 instead of kernel 720. In such an embodiment, the kernel is exposed to network interfaces 710 and 715 for full control by user space, allowing the middlebox processor to function in user space without compromising speed as compared to the kernel. Can be executed.

III. Network control system Section I above describes different middlebox implementation architectures from fully distributed to fully centralized. As mentioned above, in some embodiments, these middleboxes are provided via a network control system, which is used to provide a management switching element that implements the logical transfer element of the network. In some embodiments, the network control system is a set of hierarchical network controllers.

FIG. 8 shows a network control system 800 of several embodiments for configuring a management switching element and a distributed middlebox element to realize a logical network. As shown, the network control system 800 includes an input conversion controller 805, a logical controller 810, physical controllers 815 and 820, hosts 825-840, and a centralized middle box 845. As shown, the host 830-865 includes both a management switching element and a middlebox element, which may be implemented as shown in FIG. Those skilled in the art will appreciate that many other different combinations of various controllers and hosts are feasible for network control systems.

In some embodiments, each of the controllers in the network control system has the ability to function as at least one of an input conversion controller, a logical controller and a physical controller. Optionally, in some embodiments, a given controller can only have the ability to act as a specific one (eg, a physical controller) of various types of controllers. In addition, various combinations of controllers can be operated within the same physical machine. For example, the input conversion controller 805 and the logical controller 810 can be run on the same computer device that interacts with the user.

Also, each of the controllers shown in FIG. 8 (and subsequent FIG. 9) is shown as a single controller. However, each of these controllers may actually be a controller cluster operating in a distributed format that performs the processing of a logical controller, a physical controller, or an input conversion controller.

The input conversion controller 805 of some embodiments includes an input conversion application that transforms the network configuration information received from the user. For example, the user can identify a network topology as shown in Figure 2, which contains specific information about which machines are attributed to the logical domain. It efficiently identifies a set of logical data paths, or a set of logical transfer elements. For each of the logical switches, the user identifies a machine (ie, a logical port assigned to the logical switch) that connects to the logical switch. In some embodiments, the user also identifies an IP address for the machine. The input conversion controller 805 converts the input network topology into logical control plane data that describes the network topology. For example, the entry describes that a particular MAC address A is located on a particular logical port X on a particular logical switch.

In some embodiments, each logical network is managed by a particular logical controller (eg, logical controller 810). The logical controller 810 of some embodiments converts the logical control plane data into logical transfer plane data and the logical transfer plane data into universal control plane data. Logical transfer plane data, in some embodiments, consists of flow entries described at the logical level. For MAC address A on logical port X, the logical forwarding plane data can include a flow entry that identifies the packet to be forwarded to port X if the destination of the packet matches MAC A. ..

The universal physical control plane data of some embodiments is when the control system of some embodiments contains a large number (eg, thousands) of management switching elements to implement a set of logical data paths. Even is a data plane that allows the control system to scale. The universal physical control plane brings together the common features of different management switching elements to represent the physical control plane data without considering the differences in the management switching elements and the location details of the management switching elements.

As mentioned above, the logical controller 510 of some embodiments translates the logical control plane data into logical transfer plane data (eg, logical flow entry) and then converts the logical transfer plane data into universal control plane data. To do. In some embodiments, the logical controller application stack includes a control application for performing the first transformation and a virtualization application for performing the second transformation. Both of these applications, in some embodiments, use a rules engine for mapping a set of first tables to a set of second tables. That is, different data planes are represented as tables (eg, nLog tables), and controller applications use the table mapping engine to translate between data planes.

Each of the physical controllers 815 and 820 is a master of one or more management switching elements (eg, placed within a host machine). In this example, each of the two physical controllers is the master of the two management switching elements. The physical controller 815 is also the master of the centralized middlebox 845. In some embodiments, the physical controller receives universal physical control plane information for the logical network and transforms that data into customized physical control plane information for a particular physical management switch managed by the physical controller. In other embodiments, the physical controller passes the appropriate universal physical control plane data to the management switch, which includes the ability to perform the transformation itself (eg, in the form of a chassis controller running on the host machine). I'm out.

The conversion from the universal physical control plane to the customized physical control plane involves customizing the various data in the flow entry. For the above example, the universal physical control plane will contain several flow entries. The first entry describes a packet if it matches a particular set of logical data paths (for example, based on a packet received on a particular logical entry port) and the destination address matches MAC A. Describes forwarding to logical port X. In some embodiments, this flow entry is identical within the universal physical control plane and the customized physical control plane. Additional flows include a physical entry port (eg, the virtual interface of the host machine) and a logical entry port X (MAC). Generated to match (for packets received from A) and, in addition, to match logical port X with a specific exit port on the physical management switch. However, these physical entry ports and physical exit ports have been identified for host machines that include management switching elements. Thus, the universal control plane entry contains the abstract physical port, while the customized physical control plane entry contains the intervening actual physical port.

In some embodiments, the network control system disseminates data related to the middlebox of the logical network. In addition to the middlebox configuration data, the network control system disseminates data related to at least one of sending and receiving packets to and from the middlebox on the management switch and sending and receiving packets to and from the management switch on the middlebox.

As shown in FIG. 8, the same network control system, in some embodiments, delivers data to both the distributed and centralized middleboxes. Some physical controllers are used to disseminate distributed middlebox configurations, while some embodiments assign specific physical controllers to centralized middlebox electronics. In this example, the physical controller 815 is assigned to disseminate the configuration of the centralized middlebox 845, while the configuration for the distributed middlebox is disseminated through both the physical controllers 815 and 820.

The flow entries propagated through the network control system to the management switch to incorporate the middlebox will include a set of entries for sending the appropriate packets to the appropriate middlebox (eg, a particular middlebox). Flow entries that identify packets with a source IP address within a particular subnet that are forwarded to). In addition, the flow entries for the management switch will need to identify how to send such packets to the middlebox. That is, once the first entry identifies the logical exit port of a logical router that is bound to a particular middlebox, additional entries are required to connect that logical exit port to the middlebox.

For the centralized Middlebox 845, these additional entries will match the logical exit port of the logical router with a specific physical port of the host machine (eg, a physical network interface). Through this particular physical port, the host machine connects to the middlebox. In addition, the entries include encapsulation information for sending packets to the centralized middlebox electronics through the tunnel between the host machine and the middlebox.

For a distributed middlebox, the packet does not actually have to leave the host machine to reach the middlebox. However, the management switch element must nevertheless contain a set of flow entries for sending packets to the middlebox on the host machine. These flow entries again contain an entry for mapping the management switch element to the port that connects to the middlebox to the logical exit port of the logical router. However, in this case, the middlebox connects to a software abstract port within the management switch element rather than the physical (or virtual) interface of the host machine. That is, the port is "created in the management switching element to which the middlebox element is connected. The flow entries in the management switching element send packets to this port to forward to the middlebox in the host machine. To do.

For both distributed and centralized middleboxes, in some embodiments, the management switching element adds slice processing information to the packet. Basically, this slicing information is a tag that indicates to which of the (potential) instances operated by the middlebox the packet should be sent. In this way, when the middlebox receives a packet, the tag allows the middlebox to perform actions on the packet using the appropriate set of rules for processing, parsing, modifying, etc. of the packet. To do. In some embodiments, instead of adding slicing information to the packet, a different port group of management switching elements is defined for each middlebox instance, and that port group is actually used to firewall. Slice the traffic directed to (in a distributed configuration) or connect to different ports on the centralized electronics to distinguish between instances (in a centralized configuration).

The above content describes the propagation of transfer data to the management switching element. In addition, some embodiments use a network control system to propagate configuration data to the middlebox. FIG. 9 conceptually illustrates the propagation of data through the network control system of some embodiments. The left side of the figure shows the data flow to the management switching elements that realize the logical network, while the right side of the figure shows the propagation of middlebox configuration data and the propagation of network connection and slice processing data to the middlebox. ing.

On the left side, the input conversion controller 805 receives the network configuration via the API, which is converted to logical control plane data. This network configuration data contains a logical topology as shown in Figure 2. In addition, the network configuration data of some embodiments includes a set of routing policies that identify packets sent to the middlebox. If the middlebox is located on a logical line between two logical forwarding elements (for example, between a logical router and a logical switch), all packets sent over that logical line are automatically middle. It will be transferred to the box. However, for out-of-band middleboxes such as Network Architecture 200, the logical router will only send packets to the middlebox if a particular set of policies is specified by the user. ..

While routers and switches will normally forward packets according to the packet's destination address (eg, MAC address or IP address), policy routing is the other information stored by the packet (eg, source address, source). Allows forwarding decisions to be made based on (such as the combination of address and destination address). For example, a user may identify that all packets that have a source IP address within a particular subnet or a destination IP address that does not match a particular set of subnets should be forwarded to the middlebox. it can.

As shown, the logical control plane data is transformed into logical transfer plane data by the logical controller 810 (specifically, by the logical controller control application), followed by universal physical control (by the logical controller virtual application). Converted to plain data. In some embodiments, these transformations generate flow entries (in the logical transfer plane) and add matching information (in the universal physical control plane) through a set of logical data paths. The universal physical control plane also provides additional flow entries for mapping generic physical entry ports to logical entry ports (ie, generic abstract ports that are not dedicated to any particular physical host machine). It also contains an additional set of flow entries for mapping logical exit ports to general purpose physical exit ports. For example, for mapping to a centralized middlebox, the flow entries in the universal physical control plane make a forwarding decision to send a packet to the logical port that connects to the middlebox if the routing policy is met, and the middlebox. Contains mappings to logical ports to general-purpose physical ports of host machines that connect to.

As shown, physical controller 815 (one of several physical controllers) transforms universal physical control plane data into customized physical control plane data for the particular management switching elements 830-840 it manages. This transformation includes alternative specific data (eg, specific physical ports) for general purpose abstractions in universal physical control plane data. For example, in the example above, the port integration entry is configured to identify the appropriate physical layer port for a particular Middlebox configuration. This port can be a virtual NIC when the firewall acts as a virtual machine on the host machine, or manages when the firewall acts as a process (eg, daemon) in the hypervisor on the virtual machine. It can be the software abstract port described above within the switching element. In some embodiments, in the latter situation, this port is an IPC channel or TUN / TAP device interface. In some embodiments, the management switching element includes one specific abstract port for the fire module and sends this information to the physical controller for customizing the physical control plane flow. On the other hand, for the flow entries for sending packets to the centralized middlebox 845, the insert port is the actual physical port of the particular host machine on which the management switching element operates.

In addition, in some embodiments, the physical controller adds a set of flow entries to the middlebox that identifies specific slice processing information. For example, for a particular management switching element, the flow entry identifies adding a particular tag (eg, a VLAN tag or similar tag) to the packet before sending it to a particular firewall. can do. This slicing information allows the middlebox to receive the packet and identify which of its several independent instances should process the packet.

Management switching element 825 (one of several MSE groups managed by physical controller 815) performs the conversion of customized control plane data to physical transfer plane data. This physical transfer plane data is, in some embodiments, a set of flow entries stored within the switching element (physical router or switch or software switching element) for a received packet to which the switching element actually matches.

The right side of Figure 9 shows two sets of data propagated to the middlebox (centralized or distributed middlebox) rather than the management switching element. These first sets of data are actual middlebox configuration data that contains various rules that specify the behavior of a particular logical middlebox. This data is received by the input conversion controller 805 or a different input interface via an API specific to the middlebox implementation. In some embodiments, different middlebox implementations will have different interfaces presented to the user (ie, the user will have to enter information in different formats for different specific middleboxes). become). As shown, the user enters a Middlebox configuration, which is converted to Middlebox configuration data by the Middlebox API.

In some embodiments, the middlebox configuration data is a set of records, where each record identifies a particular rule. These records, in some embodiments, are in a format similar to the flow entry propagated to the management switching element. In practice, some embodiments use the same application on the controller to propagate a firewall configuration record for a flow entry and also use the same table mapping language (eg, nLog) for that record. Propagate.

In some embodiments, the middlebox configuration data is not transformed by the logical or physical controller, while in other embodiments, at least one of the logical and physical controllers is in the middlebox configuration data record. Perform at least minimal conversion. Many middlebox packet processing, modification and parsing rules work with the packet's IP address (or TCP connection state), and packets sent to the middlebox are exposed to this (ie, logical port information). The middlebox configuration does not require a conversion from a logical data plane to a physical data plane, as it will have information (not encapsulated within). In this way, the same middlebox configuration data is passed from the input conversion controller 805 (or other interface) to at least one of the logical controller 810 and the physical controller 815.

In some embodiments, the logical controller 810 stores a description of the logical network and a description of the physical implementation of the physical network. The logical controller receives one or more middlebox configuration records for the distributed middlebox, and which of the various nodes (ie, the host machine) needs to receive configuration information. To identify. In some embodiments, the entire middlebox configuration is delivered to the middlebox elements in all of the host machines, so that the logical controller has at least one virtual machine whose packets require the use of the middlebox. Identifies all of the host machines where is resident. This may be all virtual machines in the network (eg, for the middlebox shown in Figure 2), or it may be a subset of the virtual machines in the network (eg, specifics in the network). If the firewall applies only to traffic in your domain). Some embodiments make decisions about which host machine sends configuration data on a record-by-record basis. That is, each particular rule can be applied only to a subset of virtual machines, and only to hosts running those virtual machines that need to receive records.

Once the logical controller identifies the particular node that receives the record, the logical controller identifies the particular physical controller that manages these particular nodes. As mentioned above, each host machine has an assigned master physical controller. Thus, if the logical controller identifies only the first and second hosts as destinations for configuration data, the physical controllers for those hosts will be identified to receive data from the logical controller. (And no other physical controller will receive this data). For a centralized middlebox, the logical controller only needs to identify the (single) physical controller that manages the electronics that implement the middlebox.

To supply middle-box configuration data to the host, the logical controller of some embodiments pushes the data to the physical controller (using an export module that accesses the output of the table mapping engine in the logical controller). To do. In another embodiment, the physical controller requests configuration data (eg, in response to a signal indicating that the configuration data is available) from the export module of the logical controller.

The physical controllers pass the same amount of data as they pass the physical control plane data to the middlebox elements on the host machine they manage. In some embodiments, the middlebox configuration data and the physical control plane data are sent to the same database running on the host machine, and the management switching element and middlebox module get the appropriate information from the database. Similarly, for the centralized middlebox 845, the physical controller 815 passes the middlebox configuration data to the middlebox electronic device (eg, to a database that stores the configuration data).

In some embodiments, the middlebox transforms the configuration data. The middlebox configuration data will be received in a specific language for expressing rules such as packet processing, parsing, and modification. The middlebox of some embodiments (at least one of distributed and centralized) compiles these rules into more optimal packet classification rules. In some embodiments, this conversion is similar to the conversion of physical control plane data to physical transfer plane data. When a packet is received by the middlebox, it applies the best compiled rules to perform its actions in the packet efficiently and fast.

To add a middlebox configuration rule, the middlebox module receives at least one of the slice processing information and the connection information and sends and receives packets to and from the management switching element. This information corresponds to the information sent to the management switching element. As illustrated, in some embodiments, the physical controller 815 generates at least one of the slice processing information and the connection information for the middle box (ie, this information is at the input or logical controller level of the network control system. Not generated).

For distributed middleboxes, the physical controller receives information about the software ports of the management switching element in some embodiments. Here, the middlebox connects from the management switching element itself to its management switching element and passes this information to the middlebox. In other embodiments, however, the use of this port is directly agreed between the middlebox module and the management switching element in the host machine, so that the middlebox receives connection information from the physical controller. Does not need to be received. In some such embodiments, the management switching element nevertheless sends this information to the physical controller, which in turn sends and receives packets to and from the middlebox in the universal physical control plane. Customize the flow entry.

For a centralized middlebox, some embodiments provide tunneling connection data to the middlebox. This middlebox, in some embodiments, needs to know the type of tunnel encapsulation that various host machines will use to send packets to the middlebox. In some embodiments, the middlebox has a list of acceptable tunneling protocols (eg, STT, GRE, etc.) and the selected protocol is coordinated between the management switching element (s) and the middlebox. Will be done. The tunneling protocol can be entered by the user as part of the Middlebox configuration, or in different embodiments it can be determined automatically by the network control system. In addition to connecting to the host machine, a tunnel will be set up between the centralized middlebox and the pool node, as described with reference to Figure 4, which will take the processed packets out of it. Send to the pool node.

The slice processing information generated by the physical controller, in some embodiments, consists of an identifier for the middlebox instance used for the logical network. In some embodiments, as described above, the middlebox, which operates on a host machine or operates as a centralized electronic device, is virtualized for use by multiple logical networks. When the middlebox receives a packet from a management switching element, in some embodiments, the packet identifies a middlebox instance (ie, a particular set of configured rules) to use when processing the packet. Contains a prepended tag that identifies one of the (eg, similar to a VLAN tag).

As shown in FIG. 9, the middlebox converts this slice processing information into internal slice binding information. In some embodiments, the middlebox uses its own internal identifier (which is different from the tag prepending on the packet) to state the state inside the middlebox (eg, active TCP connections, various IPs). Identify (such as statistics about the address). Upon receiving an instruction to create a new middlebox instance and an external identifier (used on the packet) for the new instance, some embodiments will automatically create a new middlebox instance and its Assign a middlebox instance to an internal identifier. In addition, the middlebox stores bindings for its middlebox instance that maps the outer slice identifier to the inner slice identifier.

The figure above shows various physical and logical network controllers. FIG. 10 shows an example of the architecture of the network controller 1000 (eg, a logical controller or a physical controller). The network controller of some embodiments uses a table mapping engine to map data from a set of table inputs to data in a set of table outputs. The set of inputs for this table in this controller is the logical control plane (LCP) data that maps to logical transfer plane (LFP) data, the LFP data that maps to universal physical control plane (UPCP) data, and the customized physical control plane. Contains at least one of the UPCP data that is mapped to (CPCP) data. The set of inputs in this table can also include middlebox configuration data sent to at least one of another controller and a distributed middlebox instance. The network controller 1000 includes an input table 1015, a rule engine 1010, an output table 1020, an importer 1030, an exporter 1035, a translator 1035 and a permanent data storage device (PTD) 1040, as shown.

In some embodiments, the input table 1015 includes a table with various types of data, depending on the role of controller 1000 in the network control system. For example, if the controller 1000 acts as a logical controller for the user's logical transfer element, the input table 1015 contains LCP data and LFP data for the logical transfer element. If controller 1000 acts as a physical controller, input table 1015 contains LFP data. Input table 1015 contains middlebox configuration data received from the user or another controller. Middlebox configuration data is associated with logical data routing parameters that identify the logical switching elements that are integrated into the middlebox.

In addition to the input table 1015, the control application 1000 contains other miscellaneous tables (not shown) used by the rule engine 1010 to collect inputs for its table mapping operations. These miscellaneous tables contain a constant table, which stores a given value for the constants required by the rules engine 1010 to perform its own table mapping operation (eg, value 0, dispatch for re-execution). Port number etc.). The constant table also contains a function table, which stores the functions used by the rule engine 1010 to calculate the values for populating the output table 1025.

The rule engine 1010 performs a table mapping operation that identifies one way to convert input data to output data. Whenever one of the input tables changes (referred to as an input table event), the rules engine performs a set of table mapping actions, which causes one or more data in one or more output tables. You will get a change in the tuple.

In some embodiments, the rule engine 1010 includes an event processor (not shown), some query plans (not shown), and a table processor (not shown). Each query plan is a set of rules that identifies a set of join operations to be performed when an input table event occurs. The event processor of Rule Engine 1010 detects the occurrence of each of these events. In some embodiments, the event processor registers a callback in the input table for notification of changes to the records in input table 1015 and notifies from the input table if one of its records has changed. Detect input table events by receiving.

In response to the detected input table event, the event processor (1) selects the appropriate query plan for the detected table event and (2) instructs the table processor to execute the query plan. To execute a query plan, the table processor, in some embodiments, performs a join operation identified by the query plan to perform one or more inputs and a set of one or more data values from a miscellaneous table. Generate one or more records that indicate . The table processor of some embodiments (1) performs a selection operation to select a subset of data values from the records (s) generated by the join operation, and (2) a subset of the selected data values. To one or more output tables 1020.

Some embodiments allow an application developer to create a rules engine for a controller using a variant of the data log database language, which allows the controller to control a set of logical data paths. Identify how to map to your physical switching infrastructure. This variant of the datalog database language is read herein as nLog. A similar data log, nLog, provides several declarative rules and operators that allow developers to identify different actions that should be taken in response to the occurrence of different events. In some embodiments, nLog provides a subset of the limited operators provided by the data log to speed up the operation of nLog. For example, in some embodiments, nLog allows only the AND operator to be used in any declarative rule.

Declarative rules and operators identified through nLog are compiled by the nLog compiler into a larger set of rules. In some embodiments, the compiler translates each rule, which is meant to handle events, into a set of join actions for several databases. Thus, a large set of rules forms a table mapping rule engine called the nLog engine.

Some embodiments specify a first join action performed by the rule engine for input events that are based on logical data routing parameters. This specification means that if the rules engine has started a set of join actions related to a logical data path set (ie, a logical network) that is not managed by the controller, the rule engine join actions immediately stop and end. Compensate.

Similarly, the input table 1015 and the output table 1020 include tables with different types of data depending on the role of controller 1000. Controller 1000 acts as a logical controller, and output table 1015 contains LFP and UPCP data for the logical switching elements. If controller 1000 acts as a physical controller, output table 1020 contains CPCP data. Like the input table, the output table 1015 may contain middlebox configuration data. The output table 1015 may also include a slice identifier when the controller 1000 functions as a physical controller.

In some embodiments, the output table 1020 can be grouped into several different categories. For example, in some embodiments, the output table 1020 can be at least one of a rule engine (RE) input table and an RE output table. The output table is an RE input table when changes in the output table cause the rules engine to detect input events that require the execution of the query plan. The output table is also the RE input table, which generates an event that causes the rules engine to execute another query plan. The output table is the RE input table when changes in the output table cause the exporter 1025 to export the changes to another controller or MSE. The output table is also an RE input table, an RE output table, or an RE input table and an RE output table.

Exporter 1025 detects changes to the RE output table in output table 1020. In some embodiments, the exporter registers a callback in the RE output table for notification of changes to the records in the RE output table. In some embodiments, exporter 1025 detects an output table event when it receives a notification from the RE output table indicating that one of its records has changed.

Depending on the output table event detected, exporter 1025 gets each modified data tuple in the modified RE output table and uses this modified data tuple on one or more other controllers or one. Propagate to the above MSE. When sending an output table record to another controller, the exporter, in some embodiments, uses a communication signal channel (eg, an RPC channel) to send the data contained in the record. When sending RE output table records to the MSE, the exporter uses two channels in some embodiments. One channel is established using a switch control protocol (eg, OpenFlow) for writing flow entries to the MSE control plane. The other channel is established using a database communication protocol (eg JSON) for transmitting configuration data (eg port configuration, tunnel information).

In some embodiments, the controller 1000 does not hold data for a logical data path set (ie, a logical network managed by another logical controller) that the controller is not responsible for managing in the output table 1020. However, such data is converted by the translator 1035 into a format that can be stored in the PTD 1040 and stored in the PTD. The PTD1040 propagates this data to the PTDs of one or more other controllers, so that these other controllers responsible for managing the logical data path set can process this data. it can.

In some embodiments, the controller also brings the data stored in the output table 1020 to the PTD with the elasticity of the data. Therefore, in these embodiments, the controller PTD has all of the configuration data for all logical data path sets managed by the network control system. That is, each PTD contains a global view of the logical network configuration for all users.

The importer 1030 is an interface for input data from several different sources and uses that input data to modify or create the input table 1010. The importer 1020 of some embodiments receives input data from another controller. The importer 1020 is also an interface to the PTD 1040, which transforms and uses the data received through the PTD from other controller instances as input data to modify or create the input table 1010. can do. The importer 1020 also detects changes associated with the RE input table in the output table 1030.

IV. Illustrative implementation of some middleboxes The above describes various principles for implementing and configuring a distributed middlebox and a centralized middlebox. The illustrated network shown in FIG. 2 is a simplified example with only a single middlebox. Figure 11, on the other hand, conceptually illustrates the more complex logical network topology 1100 with multiple middleboxes in between.

The logical network 1100 contains three logical L2 domains, which are the web server 1105-1115 connected to the first logical L2 switch 1140 and the application connected to the second logical L2 switch 1145. There are servers 1120 and 1125, and data servers 1130 and 1135 connected to the third logical switch 1150. Each of these logical switches 1140-1150 is connected to the logical router 1155 (via various middleboxes).

There is a load balancer between each logical switch and the logical router 1155, which schedules incoming traffic to a particular logical L2 domain. That is, the first load balancer 1160 balances traffic (eg, per transport connection) with these three web servers 1105-1115 by performing destination network address translation (D-NAT). The second load balancer 1165 balances the traffic between the two application servers 1120 and 1125, and the third load balancer 1170 runs D-NAT with the two data servers 1130. Balance traffic with 1135. In addition, the logical router 1155 and the second logical switch 1145 are firewall 1175.

The logical router 1155 also connects the three logical L2 domains to the external network 1195, from which client requests can come into that network. In addition, the three middleboxes mediate the L3 router 1155 to handle traffic between the management network and the external network. These middleboxes include a firewall 1180 to handle incoming traffic and a source NAT1185 to translate the IP address of outbound traffic into one or more virtual IP addresses. These middleboxes are efficiently placed between the management network and the external network. However, the logical topology is because the physical implementation involves sending packets to the middlebox and also receiving packets from the middlebox (to send to the external network 1195 or a suitable host machine). These middleboxes are shown as out-of-band middleboxes with routers in between. IDS119 also intervenes in the logical router 1155. In network 1100, the logical router forwards all duplicates of processed packets to the IDS 1190 for analysis.

Figure 12 conceptually illustrates one particular physical implementation 1200 of network 1100 in a hosted virtual environment. As shown, seven virtual machines 1105-1135 are distributed across five different host machines 1205-1225. Some host machines manage only one virtual machine, while the remaining host machines manage two VMs. The host machines 1205-1225 are connected to each other and to the pool node 1230, which is connected to the gateway 1235 (also known as the extender). Gateway 1235 connects the management network 1200 to an external network 1240 (eg, the Internet, a different management network, an external private network, etc.). This example shows that the gateway 1235 is connected only to the host machine 1205-1225 through the pool node 1230, but some embodiments provide a direct connection between the gateway and the host machines. Realize.

As shown, each host machine 1205-1225, as well as pool nodes 1230 and gateway 1235, contains management switching elements. All of the host machines 1205-1225 are configured to include flow entries for logical switch 1140-1150 as well as logical router 1155. Thus, the second host machine 1210, which includes the application server 1125 and the application server 1110, and the fifth host machine 1225, which includes only the data server 1130, implement the same management switching elements. The management switching element in gateway 1235 implements all three logical switches 1140-1150 of logical network 1100, not just logical router 1155. The pool node is also the management switching element that implements the L3 router 1155 and the three logical switches 1140-1150 in some embodiments.

In implementation 1200, some logical middleboxes are distributed, while the remaining logical boxes are centralized. For example, the intrusion detection service 1190 is implemented as a centralized IDS electronic device 1245. Each of the host machines 1205-1225 is directly connected to IDS electronics 1245 as a gateway 1235. As shown, these machines only send packets to IDS electronics and do not receive packets. This is because the IDS just receives the duplicate packets (incoming packets from gateway 1235 and outgoing packets from host machine 1205-1225) and performs an analysis to detect the threat, and analyze them. After that, the packet is not sent anywhere.

The S-NAT Middlebox 1185 is distributed across each of the host machines (eg, as a daemon running within the hypervisor). This allows all of the virtual machines to send packets to an external network that requires IP address translation to hide the actual IP address under the virtual IP address. Firewall 1175 is similarly distributed, but only on host machines 1210 and 1220. This is because they are the nodes that manage the application server virtual machines 1120 and 1125 under this firewall.

The three load balancers 1160-1170 are implemented across various host machines 1205-1225 and gateways 1235. As shown, the load balancer is implemented within the load balancer element in each of the host machines, so that the load balancer element is virtualized (ie sliced) and has several different loads. Realize a balancer. The firewall 1175 is placed in each host machine group where the application server is realized, while each specific logical load balancer is placed in each node that manages the machines that are not in charge of the specific load balancer. .. It determines, for example, which of the two data servers the load balancer 1170 receives any packet destined for the virtual IP address representing the data servers 1130 and 1135 and forwards the packet to which of the two data servers. This is because the destination IP address is modified so that it is reflected in the selected data server. This feature is not needed by the nodes that manage the data servers (as long as other virtual machines are also managed), as processing is done in the first hop (packet source) whenever possible. , Required in nodes managing other virtual machines capable of sending packets to data servers 1130 and 1135. Therefore, the load balancer element in gateway 1235 is sliced to implement all three load balancers. This is because incoming packets from the external network 1240 may be directed to any of the three logical L2 domains.

In addition, as shown, some embodiments implement any distributed middlebox for the logical network within pool node 1230 and gateway 1235. This can be difficult to determine where the middlebox is when it is needed (and the user may subsequently modify the routing policy, middlebox configuration or network architecture). As such, some embodiments do not presume that certain physical machines will not require a particular middlebox. According to this, some embodiments do not distribute different middlebox groups to different subsets of hosts. Instead, all middleboxes for the logical network are implemented on each of the hosts present in the logical network.

The firewall 1180 for handling incoming packets between the external network 1180 and the management network is centralized within a virtual machine inside the gateway (rather than a module running inside the hypervisor). When the gateway receives an incoming packet, the gateway routes the packet to the firewall VM in some embodiments. In this example, the firewall VM is located inside the gateway, while some embodiments are as virtual machines in a host machine (eg, a host machine that is different from the host machine that manages the user VMs). Alternatively, a firewall is realized by using a firewall electronic device.

To configure the network 1100, in some embodiments, the user inputs the network topology into a logical controller (eg, via an input conversion controller), as shown with reference to FIGS. 8 and 9. To do. In some embodiments, the user inputs a connection between a group of switches, a group of routers, a group of middleboxes, and a group of virtual machines. Based on the location of various network components, the input transformation controller or logical controller generates logical control plane data and transforms it into a set of flow entries in the logical transfer plane. However, for middleboxes such as Firewall 1180, S-NAT1185 and IDS1190, the user must enter policy routing rules that indicate when packets are sent for these components. For example, the routing policy for firewall 1180 will send all packets with source IPs outside the logical network, while the routing policy for S-NAT1185 will send packets with source IPs inside the logical network. All will be sent. After converting the flow entries to the universal physical control plane, the logical controller identifies which physical controller should receive which flow entry and delivers these flow entries. A group of physical controllers flows specific port information and other customization information (as long as the host machine group contains a group of chassis controllers that perform the conversion from the universal physical control plane to the customized physical control plane). And deliver them to the management switching elements.

In addition, the user inputs middlebox configurations for various load balancers, firewalls, etc. For example, this information will include scheduling algorithms to use for different load balancers, virtual IP to real IP mappings for S-NAT, packet processing rules for firewalls, and so on. The user enters this information through APIs for various middleboxes, and some embodiments convert this information into records with the same format as flow entries (eg, nLog). This logical controller identifies which middlebox configuration needs to be sent to the host machine or centralized middlebox electronics (for example, records for firewall 1175 go to host machines 1210 and 1220). The S-NAT records go to all five host machines 1205-1225), while only needing. The logical controller group delivers the record group to the appropriate physical controller group, where it adds slice processing information (and, in some cases, tunneling information) and delivers that information to the middle box group described above. ..

The operation of a network is explained by referring to a group of packets arriving from an external network, a group of packets originating from an external network, and a group of packets transmitted from one logical L2 domain to another logical L2 domain. When a packet is sent from a host machine (eg, a web server), it first reaches the MSE running on the host. The packet is first submitted to logical L2 processing by the logical switch, which logical switch sends the packet to the logical router (because it is sent out so it does not need to be sent to the local load balancer). Send to). The logical router (also handled by the MSE on the host machine) sends a copy of the packet to IDS electronics 1245 and sends the packet to S-NAT processing on the host. The S-NAT process corrects the source IP address and returns a new packet to the MSE. In some embodiments, when the packet is part of an active TCP session, S-NAT sends a set of flow entries to MSE that makes it possible to implement the modification to MSE without the intervention of S-NAT processing. You may be sending. The logical router implemented by MSE identifies the logical exit port as a port facing the external network, which maps the physical port for sending packets to pool node 1230. The pool node forwards the packet to gateway 1235, which sends the packet to external network 1240.

When a packet is received from an external network 1240 at gateway 1240, switching element processing within that gateway first sends the packet to a firewall virtual machine. If the firewall does not drop the packet, then the packet is sent back to the switching element process, which identifies the correct slice of the load balancer, tags the packet with slice information, and loads the packet. Send to balancer (load balancing) processing. The load balancer selects the actual destination IP and sends a new packet with the modified IP address to reflect on the selected destination machine. At this point, the MSE in the gateway sends the packet to the correct host. If the destination VM is one of the application servers 1120 or 1125, the logical router in the gateway's MSE first sends a packet to the firewall 1175 for processing and then receives a reply from the firewall element. Then send the packet to the correct host. The MSE then delivers the packet to the destination machine.

Packets traveling from one logical domain to another do not have to travel through gateway 1235. The packet is first received at the MSE, where it performs L2 switching and then L3 routing. The logical router identifies the destination domain, tags the packet with the correct load balancer slicing information, and then sends the tagged packet to the load balancer. The load balancer modifies the destination IP address and sends the packet back to the MSE, which forwards the packet to the correct host machine for delivery to the VM.

V. Electronic system Many of the features and applications described above are implemented as software processes identified as a set of instructions recorded on a computer-readable storage medium (also known as a computer-readable medium). If these instructions are executed by one or more computers or processing units (s) (eg, one or more processors, processor cores, or other processing units), they are processing units (s). Let the group) execute the operation indicated by the instruction group. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, random access memory (RAM) chips, hard drives, erasable programmable read-only memory (EPROM), and electrically erasable programmable. Read-only memory (EEPROM) and the like are included. Computer-readable media does not include carrier waves and electronic signals that pass through wireless or wired connections.

As used herein, the term "software" is meant to include firmware resident in read-only memory, or an application stored in magnetic storage, which is stored in memory for processing by a processor. be able to. Also, in some embodiments, the plurality of software inventions can be realized as part of a large program, while maintaining another software invention. In some embodiments, the plurality of software inventions can also be realized as separate programs. And any combination of separate programs that together realize the software inventions described herein is within the scope of the present invention. In some embodiments, a software program, when installed to operate on one or more electronic systems, defines one or more specific machine implementations that perform and function in the operation of the software program.

FIG. 13 is a diagram conceptually showing an electronic system 1300 in which some embodiments of the present invention are realized. The electronic system 1300 can be a computer (eg, desktop computer, personal computer, tablet computer, etc.), server, dedicated switch, telephone, PDA, or any other type of electronic or computer device. Such electronic systems include interfaces for various types of computer-readable media and various other types of computer-readable media. The electronic system 1300 includes a bus 1305, a processing unit (group) 1310, a system memory 1325, a read-only memory 1330, a permanent storage device 1335, an input device 1340, and an output device 1345.

Bus 1305 collectively represents any system bus, peripheral bus and chipset bus, which communicate and connect several internal devices of the electronic system 1300. For example, the bus 1305 communicates and connects the processing unit (group) 1310 with the read-only memory 1330, the system memory 1325, and the permanent storage device 1335.

From these various memory units, the processing unit (group) 1310 acquires an instruction for execution and data for the process in order to execute the process of the present invention. This processing unit (s) can be a single processor or a multi-core processor in various embodiments.

Read-only memory (ROM) 1330 stores static data and instructions required by the processing unit (group) 1310 and other modules of the electronic system. The permanent storage device 1335, on the other hand, is a read and write memory device. This device is a non-volatile memory unit that stores instructions and data even when the electronic system 1330 is off. Some embodiments of the present invention use a large capacity storage device (eg, a magnetic disk or optical disk and a corresponding disk drive) as the permanent storage device 1335.

Other embodiments use removable storage devices (eg, floppy (registered trademark) disks, flash memory devices, etc., and corresponding drives) as permanent storage devices. Like the permanent storage device 1335, the system memory 1325 is a read and write memory device. However, unlike the storage device 1335, the system memory 1325 is a non-volatile read and write memory, such as a random access memory. System memory 1325 stores some instructions and data that the processor needs at runtime. In some embodiments, the process of the invention is stored in at least one of system memory 1325, permanent storage 1335, and read-only memory 1330. From these various memory units, the processing unit (group) 1310 executes the processes of some embodiments by acquiring instructions and executing data for the process.

Bus 1305 is also connected to input device 1340 and output device 1345. The input device 1340 allows the user to communicate information and select commands for the electronic system. The input device 1340 includes an alphabetic keyboard and pointing device (also referred to as a "cursor control device"), a camera (eg, a webcam), a microphone or a similar device for receiving voice commands. Output device 1345 displays an image produced by the electronic system, or outputs data otherwise. Output device 1345 includes a printer and display device, such as a cathode ray tube (CRT) or liquid crystal display (LCD), as well as a speaker or similar audio output device. Some embodiments include devices such as touch screens that act as both input and output devices.

Then, as shown in FIG. 13, the bus 1305 connects the electronic system 1300 to the network 1365 via a network adapter (not shown). In this way, the computer can be part of a computer's network (eg, a local area network (LAN), a wide area network (WAN), or an intranet, or a network of networks, such as a network of networks, eg. It can be part of the Internet. Any or all components of the electrical system 1300 can be used with the present invention.

Some embodiments store computer program instructions in electronic components such as microprocessors, machine-readable or computer-readable media (optionally referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Includes storage and memory. Some examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROMs), recordable compact discs (CD-R), rewritable compact discs (CD-RW), and read-only compact discs. Digital versatile discs (eg DVD-ROM, dual layer DVD-ROM), various recordable / rewritable DVDs (eg DVD-RAM, DVD-RW, DVD + RW, etc.), flash memory (eg SD card) , MiniSD card, microSD card, etc.), magnetic hard drive and solid state hard drive, read-only and recordable Blu-ray® discs, high density optical discs, any other optical or magnetic media, and floppy disks. .. Computer-readable media can be executed by at least one processing unit and can store a computer program containing a set of instructions for performing various operations. Examples of computer programs or computer code include, for example, machine code generated by a compiler and files containing high-level code executed by a microprocessor using a computer, electronic component or interpreter.

While the discussion above is primarily directed to microprocessors or multi-core processors running software, some embodiments have one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programs. Performed by a Mable Gate Array (FPGA). In some embodiments, such an integrated circuit executes instructions stored in the circuit itself. In addition, some embodiments run software stored in programmable logic devices (PLDs), ROM devices, and RAM devices.

As used herein and in any claims, the terms "computer", "server", "processor", and "memory" all mean electronic devices or other technical devices. These terms exclude people or groups of people. For the purposes of the specification, the term display or display means display on an electronic device. As used in the specification of the present application and any claim, the terms "computer-readable medium", "computer-readable medium" and "machine-readable medium" are tangible, storing information in a form readable by a computer. Fully restricted to physical objects. These terms exclude any radio signal, wired download signal, and any other transient signal.

Although the present invention has been described with reference to some specific details, those skilled in the art will recognize that the present invention can be realized in other particular forms without departing from the scope of the invention. Will. That is, one of ordinary skill in the art will appreciate that the present invention is not limited to the details illustrated above, but is defined by the appended claims.