Route configuration for logical router
20 claims: 3 independent, 17 dependent
- 1論理ネットワークにおける論理ルータを実施する方法であって、前記方法は、 前記論理ルータのための静的経路の構成を受信することであって、前記論理ルータは、(i)他の論理フォワーディング要素とのインタフェースを有する分散化ルーティング構成要素と、(ii)外部物理ネットワークとのインタフェースを有する1つ以上の集中化ルーティング構成要素とを含み、前記分散化ルーティング構成要素と前記集中化ルーティング構成要素のそれぞれとは、別々のルーティングテーブルを有する、受信することと、 ど のル ーティング構成要素が、前記静的経路の構成を実施するための対応するルーティングテーブルへの経路の追加を要するかを識別することと、 前記識別されたルーティング構成要素の前記対応する別々のルーティングテーブルに前記経路を追加することと、を含む、方法。
- 2請求項1に記載の方法であって、前記静的経路の前記構成は、プレフィックスと次ホップアドレスとを含む、方法。
- 3請求項2に記載の方法であって、前記静的経路の前記構成は、更に出力インタフェースを含む、方法。
- 4請求項1に記載の方法であって、前記静的経路は、前記論理ネットワークにおけるアドレスである次ホップアドレスを含み、前記論理ルータの前記分散化ルーティング構成要素と前記集中化ルーティング構成要素のそれぞれとは、対応するルーティングテーブルへの経路の追加を要するものとして識別される、方法。
- 5請求項4に記載の方法であって、前記分散化ルーティング構成要素と前記識別された集中化ルーティング構成要素のそれぞれのため の別 々の 前記 ルーティングテーブルに経路を追加することは、 前記分散化ルーティング構成要素 のための前記ルーティングテーブル に、前記静的経路を構成済として追加することと、 前記構成済の静的経路と同一のプレフィックスのための経路を、前記集中化ルーティング構成要素のそれぞれ のための前記ルーティングテーブル に追加することであって、 前記集中化ルーティング構成要素のそれぞれのための前記ルーティングテーブルに追加される 前記経路は、前記集中化ルーティング構成要素が接続する前記分散化ルーティング構成要素のインタフェースの次ホップアドレスを含む、追加することとを含む、方法。
- 6請求項1に記載の方法であって、前記静的経路は、前記外部物理ネットワークのアドレスである次ホップアドレスを含み、且つ、出力インタフェースは前記静的経路のために特定されておらず、ど のル ーティング構成要素が前記対応するルーティングテーブルへの経路の追加を要するかを識別することは、前記次ホップアドレスへの接続性を有する集中化 ルーティング 構成要素を識別することを含む、方法。
- 7請求項6に記載の方法であって、前記識別されたルーティング構成要素に対し、経路を前記対応するルーティングテーブルに追加することは、前記次ホップアドレスへの接続性を有する前記識別された集中化ルーティング構成要素のそれぞれ のための前記ルーティングテーブル に、前記静的経路を構成済として追加することを含む、方法。
- 8請求項7に記載の方法であって、 ど のル ーティング構成要素が前記対応するルーティングテーブルへの経路の追加を要するかを識別することは、更に、前記構成済の静的経路がデフォルト経路であるかを判定することを含み、 前記識別されたルーティング構成要素の前記対応するルーティングテーブルに経路を追加することは、前記経路がデフォルト経路でない場合にのみ、前記分散化ルーティング構成要素の前記ルーティングテーブルに経路を追加することを含み、 前記分散化ルーティング構成要素のための前記ルーティングテーブルに 追加される 各 経路に対する前記次ホップアドレスは、前記構成済みの静的経路が追加された前記集中化ルーティング構成要素のインタフェースのアドレスである、方法。
- 9請求項1に記載の方法であって、前記方法は、前記論理ルータを管理する集中化ネットワークコントローラによって実行される、方法。
- 10請求項9に記載の方法は更に、前記追加された経路を有する前記ルーティングテーブルを、前記分散化ルーティング構成要素を実施す る複 数の物理マシンを動作させるローカルネットワークコントローラに分配することを含む、方法。
- 11請求項1に記載の方法であって、特定のルーティングテーブルに経路を追加することは、前記特定のルーティングテーブルのエントリを追加又は修正することを含む、方法。
- 12少なくとも1つの処理ユニットで実行されると論理ネットワークにおける論理ルータを実施するコンピュータプログラムであって、 前記論理ルータのための静的経路の構成を受信することであって、前記論理ルータは、(i)他の論理フォワーディング要素とのインタフェースを有する分散化ルーティング構成要素と、(ii)外部物理ネットワークとのインタフェースを有する1つ以上の集中化ルーティング構成要素とを含み、前記分散化ルーティング構成要素と前記集中化ルーティング構成要素のそれぞれとは、別々のルーティングテーブルを有する、受信することと、 ど のル ーティング構成要素が、前記静的経路の構成を実施するための対応するルーティングテーブルへの経路の追加を要するかを識別することと、 前記識別されたルーティング構成要素の前記対応する別々のルーティングテーブルに前記経路を追加することと、の命令の組を含む、コンピュータプログラム。
- 13請求項12に記載のコンピュータプログラムであって、前記静的経路の前記構成は、次ホップアドレスと出力インタフェースの少なくとも1つと、プレフィックスとを含む、コンピュータプログラム。
- 14請求項12に記載のコンピュータプログラムであって、前記静的経路は、前記論理ネットワークにおけるアドレスである次ホップアドレスを含み、前記論理ルータの前記分散化ルーティング構成要素と前記集中化ルーティング構成要素のそれぞれとは、対応するルーティングテーブルへの経路の追加を要するものとして識別される、コンピュータプログラム。
- 15請求項14に記載のコンピュータプログラムであって、前記分散化ルーティング構成要素と前記識別された集中化ルーティング構成要素のそれぞれのため の別 々の 前記 ルーティングテーブルに経路を追加することの前記命令の組は、 前記分散化ルーティング構成要素 のための前記ルーティングテーブル に、前記静的経路を構成済として追加することと、 前記構成済の静的経路と同一のプレフィックスのための経路を、前記集中化ルーティング構成要素のそれぞれ のための前記ルーティングテーブル に追加することであって、 前記集中化ルーティング構成要素のそれぞれのための前記ルーティングテーブルに追加される 前記経路は、前記集中化ルーティング構成要素が接続する前記分散化ルーティング構成要素のインタフェースの次ホップアドレスを含む、追加することの命令の組を含む、コンピュータプログラム。
- 16請求項12に記載のコンピュータプログラムであって、 前記静的経路は、前記外部物理ネットワークにおけるアドレスである次ホップアドレスを含み、且つ、出力インタフェースは前記静的経路のために特定されておらず、 ど のル ーティング構成要素が前記対応するルーティングテーブルへの経路の追加を要するかを識別することの前記命令の組は、前記次ホップアドレスへの接続性を有する集中化 ルーティング 構成要素を識別することの命令の組を含み、 前記識別されたルーティング構成要素に対し、経路を前記対応するルーティングテーブルに追加することの前記命令の組は、前記次ホップアドレスへの接続性を有する前記識別された集中化ルーティング構成要素のそれぞれ のための前記ルーティングテーブル に構成済として前記静的経路を追加することの命令の組を含む、コンピュータプログラム。
- 17請求項16に記載のコンピュータプログラムであって、 ど のル ーティング構成要素が前記対応するルーティングテーブルへの経路の追加を要するかを識別することの前記命令の組は、更に、前記構成済の静的経路がデフォルト経路であるかを判定することの命令の組を含み、 前記識別されたルーティング構成要素の前記対応するルーティングテーブルに経路を追加することの前記命令の組は、前記経路がデフォルト経路でない場合にのみ、前記分散化ルーティング構成要素の前記ルーティングテーブルに経路を追加することの命令の組を含み、 前記分散化ルーティング構成要素のための前記ルーティングテーブルに 追加される 各 経路に対する前記次ホップアドレスは、前記構成済みの静的経路が追加された前記集中化ルーティング構成要素のインタフェースのアドレスである、コンピュータプログラム。
- 18請求項12に記載のコンピュータプログラムであって、前記コンピュータプログラムは集中化ネットワークコントローラコンピュータによって実行され、前記コンピュータプログラムは、更に、前記追加された経路を有する前記ルーティングテーブルを、前記分散化ルーティング構成要素を実施す る複 数の物理マシンを動作させるローカルネットワークコントローラに分配することの命令の組を含む、コンピュータプログラム。
- 19請求項12に記載のコンピュータプログラムであって、特定のルーティングテーブルに経路を追加することの前記命令の組は、前記特定のルーティングテーブルのエントリを追加又は修正することの命令の組を含む、コンピュータプログラム。
- 20電子デバイスであって、 処理ユニットの組と、 前記処理ユニットの少なくとも1つで実行されると、請求項1から11のいずれか1項に記載の方法を実施するプログラムを格納する機械可読媒体と、を含む、電子デバイス。
Independent claims20
176 paragraphs, as filed
A typical physical network includes multiple physical routers for performing L3 forwarding (ie, routing). If the first machine attempts to send a packet to a second machine located on a different IP subnet, the packet is sent to a router, which uses the IP address to which the packet is sent to physical which machine. Decide if the packet should be sent through the interface. Larger networks include multiple routers so that if one of the routers goes down, packets can be routed along different paths between the first and second machines.
In a logical network, it may be necessary for user-defined data computing nodes (eg, virtual machines) in different subnets to communicate with each other as well. In this case, the tenant may define a network for virtualization, including both logical switches and logical routers. There is a need for a way to implement a logical router that appropriately provides such a virtualized logical network in a data center.
Some embodiments provide a way to implement a logical router in a network (eg, in a data center). In some embodiments, this method is performed by a management plane that centrally manages the network (eg, implemented in a network controller). In some embodiments, the method receives the definition of a logical router (eg, through an application programming interface (API)) and defines some routing components for the logical router. Each of these routing components is assigned a set of routes and a set of logical interfaces separately. The method may also receive route specifications for a logical router and connections from a logical router to another logical router. Some embodiments automatically identify which routing component's routing table to push the received route to and how to propagate the route, based on connections to other logical routers. To do.
In some embodiments, some routing components defined for a logical router include one decentralized routing component and several centralized routing components. In addition, the management plane of some embodiments defines a logical switch (referred to as a transit logical switch) for handling communication between the internal components of a logical router. Decentralized routing components and transit logic switches are implemented in a decentralized manner by multiple machines in the data center, while centralized routing components are each implemented on a single machine. Some embodiments implement decentralized components in the data paths of managed forwarding components on various machines, while centralized routing components implement multiple VMs (or other) on that single machine. It is carried out at the data calculation node). Other embodiments also implement centralized components in the data path of the assigned machine.
In some embodiments, the centralized component may be configured to be in active-active mode or active-standby mode. In active-active mode, all centralized components are fully operational at the same time, and traffic is (over various centralized components) as long as the connectivity from the centralized component to the external network is the same across the components. You can enter or leave the logical network through a centralized component that uses the equal-cost multi-path (ECMP) forwarding principle. In this mode, each of the separate centralized components has its own network layer (eg IP) address and data link layer (eg MAC) address for communicating with the external network. In addition, each of the separate centralized components has its own network layer address to connect with the transit logical switch to send packets to and receive packets from the decentralized routing component. And a data link layer address.
In some embodiments, the logical router is part of a two-tier logical network configuration. The two-tier configuration of some embodiments is referred to as a single logical router (provider logical router (PLR)) for connecting a logical network to a network outside the data center, eg, in a data center. (Managed by the owner) and multiple logical routers (referred to as tenant logical routers (TLRs)) that connect to a single logical router and do not communicate individually with the external network, eg different data centers. (Managed by the tenant) and includes. Some embodiments perform PLR in active-active mode whenever possible and only if stateful services (eg NAT firewalls, load balancers, etc.) are configured for logical routers. Use active-standby mode.
For PLR, some embodiments allow route exchange with external networks. Each of the PLR's centralized components advertises a logical network prefix and performs dynamic routing protocol processing to receive routes to external networks. These routes are transmitted to other centralized and decentralized routing components through the network control system of the network controller, which is located both in the center of the data center and on the machine running the logical network. In some embodiments, the routing information of the centralized component is relative to a route learned directly from the external network and a route learned from a different peer centralized component that has learned the route from the external network. Use different management standards in base, RIB). Therefore, the centralized component selects a route learned directly rather than a route containing redirection through the peer centralized component of the logical router.
When the logical router is a TLR, some embodiments do not use centralized components when a stateful service for the logical router is configured, or two centralized configurations in active-standby mode. Use elements. Each of the two centralized components has the same network layer address, and only the active component replies to the ARP request. To connect with the PLR, some embodiments also assign each of the two components the same network layer address (but different from the address used to connect with its own decentralized component). In addition, the management plane defines a transit logic switch between the decentralized component of the PLR and the centralized component of the TLR.
The management plane of some embodiments additionally constitutes a routing table for each logical router configuration (eg, for each of the decentralized and centralized components). Routes in these routing tables may include routes based on connected logical switches, static routes entered by the user, and dynamic routes based on the connections of other logical routers. For example, any logical switch directly connected to a logical router by an administrator will eventually have a variety of connected routes in the decentralized component's routing table and the next hop that is the decentralized component's interface. It is a route for various centralized components.
When the user adds a static route, some embodiments provide various rules for adding the static route to various logical router components. For example, some embodiments have a downward-facing static (having the next hop in the logical network). routes) directly to the decentralized component (which has an interface with other logical forwarding elements such as the identified next hop), while providing an interface for the decentralized component that is accessible by the centralized component. Add the route to point again to each of the centralized components. If the configured static path is upward-facing (with the next hop, identifying a particular output interface that is on the external network and / or correlates with one of the centralized components) Some embodiments (i) add a route to one or more centralized components depending on the identified interface and / or the address of the next hop, and (ii) add the interface of each centralized component to: Add a route to the decentralized component that you identify as a hop. However, some embodiments do not add a route to the decentralized component if the configured static route is the default route. This is because the management plane has already automatically generated an equivalent cost default route for the decentralized component routing table that points to each of the centralized components when the logical router is created.
In addition, when the TLR connects to the PLR, some embodiments identify any logical switch subnet that is connected to the TLR and should be advertised to the external network (via a dynamic routing protocol). In this case, rather than running a dynamic routing protocol between two logical routers, the management plane (knowing both logical router configurations) has a separate routing table for each of its separate routing components. Automatically perform routing exchanges while dealing with the fact that they actually have. Therefore, the management plane has (i) the route having the next hop as the interface of the TLR connecting to the PLR as the decentralized component, and (ii) the next hop as the interface of the decentralized component accessible from the centralized component. Centralize routes with routes Add routes for these logical networks to each of the routing components, both to.
The preceding summary is intended to be provided as a brief introduction to some embodiments of the present invention. It does not mean an introduction or summary of all the subjects of the invention disclosed in this document. The drawings referenced in the following detailed description and detailed description will further illustrate other embodiments and the embodiments described in the abstract. Therefore, in order to understand all the embodiments described in this document, a summary, detailed description and a thorough review of the drawings are required. Moreover, the claimed subject matter is limited by the abstract, detailed description and detailed examples in the drawings, as the claimed subject matter can be embodied in other concrete forms without departing from the idea of the subject matter. Rather, it should be defined by the attached claims.
The novel features of the present invention will be revealed by the appended claims. However, for purposes of illustration, some embodiments of the invention will be apparent in the figures below.
<figref num="1">FIG. 1 shows a configuration diagram of a logical router representing a logical network as directed by a user.</figref>
<figref num="2">FIG. 2 shows the scope of the management plane of the logical network of FIG. 1 when the logical routers are implemented in a decentralized manner.</figref>
<figref num="3">FIG. 3 shows a physically decentralized implementation of the logical router of FIG.</figref>
<figref num="4">Figure 4 conceptually shows a logical network that uses two layers of logical routers.</figref>
<figref num="5">Figure 5 shows the extent of the management plane of the logical topology of Figure 4 when the TLRs in the logical network are fully decentralized.</figref>
<figref num="6">FIG. 6 shows the extent of the management plane of the logical topology of FIG. 4 when the TLR in the logical network has centralized components.</figref>
<figref num="7">Figure 7 conceptually shows a more detailed configuration of a logical network topology, including network addresses and interfaces assigned by the administrator.</figref>
<figref num="8">Figure 8 shows the configuration of the logical topology of Figure 7 for each management plane.</figref>
<figref num="9">FIG. 9 conceptually illustrates the processing of some embodiments that make up the PLR based on user specifications.</figref>
<figref num="10">FIG. 10 conceptually illustrates the processing of some embodiments that make up the TLR based on user specifications.</figref>
<figref num="11">FIG. 11 conceptually illustrates the logical network configuration used to show the RIB generation rules of some embodiments.</figref>
<figref num="12">FIG. 12 conceptually illustrates the logical network generated by the management plane based on the configuration shown in FIG.</figref>
<figref num="13">FIG. 13 conceptually illustrates the processing of some embodiments that process south-facing static routes that are input as configuration information for the PLR.</figref>
<figref num="14">Figure 14 shows the set of static paths configured for the PLR and the configuration for the set of TLRs, and the output for the DR of the PLR and the three SRs generated by the management plane based on the configuration data. Is conceptually shown.</figref>
<figref num="15">FIG. 15 conceptually illustrates the processing of some embodiments that process a north-facing route that is input as configuration information for the PLR.</figref>
<figref num="16">FIG. 16 conceptually illustrates the processing of some embodiments that generate a RIB for the PLR's routing construct when the TLR is attached to the PLR.</figref>
<figref num="17">FIG. 17 conceptually illustrates the processing of some embodiments that generate a route for the RIB of the PLR's routing configuration when a new interface is attached to the PLR.</figref>
<figref num="18">FIG. 18 conceptually illustrates an electronic system in which some embodiments of the present invention are implemented.</figref>
Some embodiments provide a way to implement a logical router in a network (eg, in a data center). In some embodiments, this method is performed by a management plane that centrally manages the network (eg, implemented in a network controller). In some embodiments, the method receives a logical router definition (eg, through an application programming interface (API)) and defines multiple routing components for the logical router. Each of these routing components is assigned a set of routes and a set of logical interfaces separately. The method may also receive route specifications for a logical router and connections from a logical router to another logical router. Some embodiments automatically identify which routing component's routing table to push the received route to and how to propagate the route, based on connections to other logical routers. To do.
In some embodiments, some routing components defined for a logical router are one decentralized routing component (hereinafter referred to as a decentralized router or DR) and a service router (hereinafter referred to as a service router). Includes several centralized routing components (also referred to as SR). In addition, the management plane of some embodiments defines a logical switch (referred to as a transit logical switch) for handling communication between the internal components of a logical router. DR and pass-through logic switches are performed in a decentralized manner by multiple machines in the data center, while SRs are each performed on a single machine. Some embodiments implement DR and pass-through logical switches in the data paths of managed forwarding components (MFEs) on various machines, while SRs perform multiple VMs (or other) on that single machine. It is carried out at the data calculation node). Other embodiments also implement centralized components in the data path of the assigned machine.
In certain embodiments, the SR may be configured to be in active-active mode or active-standby mode. In active-active mode, all SRs are fully operational at the same time, and traffic is equivalent (balancing traffic across different SRs) as long as the connectivity from the centralized component to the external network is the same across the components. You can enter or leave the logical network through SR using the equal-cost multi-path (ECMP) forwarding principle. In this mode, each of the separate SRs has its own network layer (eg IP) address and data link layer (eg MAC) address for communicating with the external network. In addition, each of the separate SRs has its own network layer address and data link to connect with the transit logical switch to send packets to and receive packets from the decentralized routing component. It has a layer address.
In some embodiments, the logical router is part of a two-tier logical network configuration. Some embodiments of the two-tier configuration are referred to as a single logical router (provider logical router (PLR)) for connecting a logical network to a network outside the data center, eg, in a data center. (Managed by the owner) and multiple logical routers (referred to as tenant logical routers (TLRs)) that connect to a single logical router and do not communicate individually with the external network, eg different data centers. (Managed by the tenant) and includes. Some embodiments implement SR of the PLR in active-active mode whenever possible, where stateful services (eg NAT firewalls, load balancers, etc.) are configured for logical routers. Use active-standby mode only for. The following description primarily describes active-active modes for PLR.
For PLR, some embodiments allow route exchange with external networks. Each SR in the PLR advertises a logical network prefix and performs dynamic routing protocol processing to receive routes to the external network. These routes are transmitted to other SRs and DRs through the network control system of the network controller located both in the center of the data center and on the machine running the logical network. Some embodiments differ in the routing information base (RIB) of the centralized component for routes learned directly from the external network and routes learned from SRs of different peers that have learned routes from the external network. Use management standards. Therefore, the SR selects the route learned directly rather than the route including the redirection through the peer SR of the logical router.
When the logical router is a TLR, some embodiments use no SR or two SRs in active-standby mode when stateful services to the logical router are configured. Each of the two SRs has the same network layer address, and only the active component replies to the ARP request. To connect with the PLR, some embodiments also assign each of the two SRs the same network layer address (but different from the address used to connect with their own DR). In addition, the management plane defines a transit logic switch between the DR of the PLR and the SR of the TLR.
The management plane of some embodiments additionally constitutes a routing table for each logical router configuration (eg, for DR and each SR). Routes in these routing tables may include routes based on connected logical switches, static routes entered by the user, and dynamic routes based on the connections of other logical routers. For example, any logical switch directly connected to a logical router by an administrator is eventually in the connected path in the DR's routing table and in the path for various SRs with the next hop that is the DR. ..
If the user adds a static route, some embodiments provide various rules for adding the static route to various logical router components. For example, some embodiments add a downward static path (having the next hop in the logical network) directly to the DR (having an interface with other logical forwarding elements such as the identified next hop). On the other hand, add a route to each SR that points again to the DR interface accessible by the SR. If the configured static route is upward (with the next hop, which identifies a particular output interface that is on the external network and / or correlates with one of the SRs), some embodiments (have a next hop). i) Add a route to one or more SRs depending on the identified interface and / or the address of the next hop, and (ii) add a route to the DR that identifies the interface of each centralized component as the next hop. However, some embodiments do not add a route to the DR if the configured static route is the default route. This is because the management plane has already automatically generated an equivalent cost default route for each of the SR's DR routing tables when the logical router is created.
In addition, when the TLR connects to the PLR, some embodiments identify any logical switch subnet that is connected to the TLR and should be advertised to the external network (via a dynamic routing protocol). In this case, rather than running a dynamic routing protocol between two logical routers, the management plane (knowing both logical router configurations) has a separate routing table for each of its separate routing components. Automatically perform routing exchanges while processing the fact that they actually have. Therefore, the management plane shall (i) route the route having the next hop as the interface of the TLR connecting to the PLR to the DR, and (ii) route the route having the next hop as the interface of the DR accessible from the SR to the SR. Add routes for these logical networks to both.
The above describes the concept of a two-tier logical router and certain aspects of the configuration and implementation of a logical router in some embodiments. Hereinafter, Section I focuses on the overall high-level design of the logical router of some embodiments, while Section II describes the configuration of various logical router components. Section III then describes the routing table configuration for these logical router components. Finally, Section IV describes an electronic system in which some embodiments of the present invention are implemented.
I. Logical router and physical implementation
The following description describes the design of a logical router for some embodiments, along with the implementation of a logical router by a network controller in some embodiments. In some embodiments, the logical router exists in three different forms. The first of these forms is an API view, or configuration view, defined by a user (eg, a data center provider or tenant). The second view is a control or management plane view of how the network controller internally defines the logical router. Finally, the third view is the physical reification or implementation of the logical router as to how it is actually implemented in the data center. That is, a logical router is an abstraction that represents a set of functions (eg, routing, NAT, etc.) that a user configures for a logical router. The logical router is then implemented by these machines in the data center based on instructions that the network controller set distributes to various machines using instructions generated by the network controllers according to the configuration provided by the user.
In the control plane view, the logical routers of some embodiments include one or both of a single DR and one or more SRs. In some embodiments, the DR extends to a managed forwarding component (MFE) that connects directly to a VM or other data compute node that logically connects directly or indirectly to a logical router. The DR of some embodiments also extends to the gateway to which the logical routers are coupled. In some embodiments, the DR is responsible for the decentralized routing of the first hop between the logical switch and at least one of the other logical routers logically connected to the logical router. The SR of some embodiments is responsible for delivering services that are not implemented in a decentralized form (eg, some stateful services).
In some embodiments, the physical embodiment of the decentralized logic router always has a DR (ie for first hop routing). A logical router is one in which (i) the logical router is a PLR and therefore connects to an external physical network, or (ii) a service configured so that it does not have a decentralized implementation (eg, in some embodiments). , NAT, load balancing, DHCP), if the logical router has SR, in any case. Some embodiments use SR in implementations to centralize connections to external networks, even if there are no stateful services configured on the PLR.
A. Single-layer logical router
Figure 1-3 shows three different views of the DLR implementation. FIG. 1 shows a configuration view of a logical router, in particular representing a logical network 100 as directed by a user. As shown, the logical router 115 is part of a logical network 100 that includes the logical router 115 and two logical switches 105 and 110. Each of the two logical switches 105 and 110 has a VM that connects to a logical port. Although shown as a VM in the three figures, it should be understood that in some embodiments other types of data compute nodes (eg namespaces, etc.) connect to the logical switch. The logical router 115 also includes two ports that connect to the external physical network 120.
FIG. 2 shows the management plane view 200 of the logical network 100. The logical switches 105 and 110 are identical in this view to the configuration view, but the network controller generates two service routers 205 and 210 for the logical router 115, and a decentralized router 215 and a transit logical switch 220. ing. The DR215 includes a southbound interface to each of the logical switches 105 and 110 and a single northbound interface to (and from now on) the transit logical switch 220. Each of the SR205 and 210 includes a single southbound interface to the transit logic switch 220 (used to communicate with the DR215 and, in certain circumstances, to communicate with each other). Each of the SR 205 and 210 also corresponds to the uplink port of the logical router (connecting to the external network), so each of the SRs has a single such interface.
The detailed configurations of the northbound and southbound interfaces of the various router configurations 205-215 and their connections to the transit logic switches 220 are described in more detail below. In some embodiments, the management plane generates a separate routing information base (RIB) for each of router configurations 205-215. In addition to having separate objects created in the management / control plane, each router configuration 205 is treated as a separate router with different routes.
Finally, Figure 3 shows the physical implementation of Logical Router 100. As shown, each of the VMs connected to one of the logical switches 105 and 110 in the logical network 100 resides on the host machine 305. In some embodiments, the MFE310 running on these host machines is a virtual switch (eg Open vSwitch (OVS), ESX) running within a hypervisor or other virtualization software on the host machine. These MFEs perform first-hop switching and routing for implementing the logical switches 105 and 110 and the logical router 115 for packets transmitted by the VMs of the logical network 100. MFE310 (or a subset of these) also implements logical switches (and decentralized logical routers) for other logical networks if the other logical network has VMs that also reside on host machine 305. May be good.
The two service routers 205 and 210 run on different gateway machines 315 and 320, respectively. Gateway machines 315 and 320 are host machines that are similar to host machines 305 in some embodiments, but are host SRs rather than user VMs. In some embodiments, the gateway machines 315 and 320 each include an SR and an MFE so that the MFE can handle routing and logical switching for the DR215. For example, packets sent from the external network 120 can be routed by an implementation of the SR routing table on one gateway, and then switched and routed (according to the DR routing table) by the MFE on the same gateway.
SR can be implemented in a namespace, in a virtual machine, or as a VRF in different embodiments. In some embodiments, the SR may operate in active-active or active-standby mode, depending on whether any stateful service (eg, firewall) is configured on the logical router. When a stateful service is configured, some embodiments require only a single active SR. In some embodiments, the active and standby service routers are provided in the same configuration, but the MFE sends packets through the tunnel to the active SR (or to the MFE on the gateway machine with the active SR). It is configured as follows. The MFE sends a packet to the standby gateway only if the tunnel goes down.
B. Multi-Tier Topology
The previous example shows only a single layer of logical routers. For logical networks with multi-layered logical routers, some embodiments have both DR and SR at each level, or only upper level DR and SR (PLR layer) and lower level (TLR layer) DR. , Can be included. FIG. 4 conceptually illustrates the multi-layer logical network 400 of some embodiments, along with FIGS. 5 and 6 showing views of two different management planes of the logical network.
Figure 4 conceptually illustrates a logical network 400 that uses two layers of logical routers. As shown, the logical network 400 includes a provider logical router 405 and some tenant logical routers 410-420 at the layer 3 level. The first tenant logical router 410 has two installed logical switches 425 and 430, along with one or more data compute nodes connected to each of the logical switches. Typically, the other TLR415-420 has an attached logical switch (connected to the data compute node), but for simplicity, only the logical switch attached to the first TLR410 is shown.
In some embodiments, any number of TLRs may be attached to PLRs such as PLR405. Some data centers may have only a single PLR to which all TLRs performed within the data center are installed, while other data centers may have multiple PLRs. For example, a large data center may want to use different PLR policies for different tenants, or attach all TLRs to a single PLR (eg because the routing table for the PLRs is very large). You can have many different tenants. The part of the PLR's routing table contains routes to all of its logical switch domains, so by attaching multiple TLRs to the PLR, several routes to each TLR that are simply based on the subnet attached to the TLR. Generate. As shown, the PLR 405 provides a connection to the external physical network 435, and in some embodiments, the PLR provides such a connection so that the data center provider can manage this connection. Make it possible. Each of the separate TLRs 410-420 is configured independently through part of the logical network 400 (although a single tenant can have multiple TLRs if selected so).
5 and 6 show the possible management planeviews of the logical network 400, depending on whether the TLR405 contains centralized components. In these examples, TLR405 routing is always decentralized using DR. However, if the TLR405 configuration includes the provision of stateful services, the TLR's management planeview (and thus the physical implementation) includes active and standby SRs for these stateful services.
Therefore, FIG. 5 shows the management planeview 500 for the logical topology 400 when the TLR405 is fully decentralized. For simplicity, only the details of the first TLR410 are shown, but other TLRs each have their own DR in addition to the SR in some cases. As shown in FIG. 2, the PLR405 includes a DR505 and three SR510-520s that are connected to each other by a transit logic switch 525. In addition to the pass-through logic switch 525 in the PLR405 implementation, the management plane also defines separate pass-through logic switches 530-540 between each of the TLRs and the DR505 of the PLR. In the case where the TLR410 is fully decentralized (Figure 5), the transit logic switch 530 connects to the DR545 that implements the TLR410 configuration. Therefore, as described in more detail in US Provisional Application No. 62 / 110,061 filed on January 30, 2015, packets sent by a data compute node attached to logical switch 425 to a destination in the external network. Is processed through the pipeline of logical switch 425, DR545 of TLR410, passing logical switch 530, DR505 of PLR405, passing logical switch 525, and one of SR510-520. In some embodiments, in all of some embodiments, the presence and definition of transitive logical switches 525 and 530-540 are users who configure the network through the API (eg, with the exception of troubleshooting purposes). It is hidden by the administrator).
FIG. 6 shows the management planeview 600 for the logical topology 400 when the TLR405 has centralized components (eg, because a stateful service that cannot be decentralized is defined for the TLR). In some embodiments, stateful services such as firewall, NAT, load balancing, etc. are provided only in a centralized form. However, other embodiments allow some or all of the decentralized services. As in the previous figure, only the details of the first TLR410 are shown for simplicity, the other TLRs are similarly defined components (DR, passing LS, and two SRs), or figures. It may have only DR as in example 5. PLR405 is implemented in the same manner as in the previous figure, using DR505 and three SR510s connected to each other by transit logic switch 525. In addition, as in the previous example, the management plane places transit logic switches 530-540 between the PLRs and TLRs, respectively.
A partially centralized implementation of the TLR410 includes a DR605 fitted with logical switches 425 and 430 and two SR610s and 615s. Like the PLR implementation, each of the DR and the two SRs has an interface to the transit logic switch 620. In some embodiments, the transit logic switch serves the same purpose as the switch 525. For TLRs, some embodiments implement an active-standby SR with one SR designated as active and the other as standby. Therefore, as long as the active SR is operational, packets sent by the data compute node attached to one of the logical switches 425 and 430 will be sent to the active SR rather than the standby SR. In some embodiments, the transit logic switch 530 contains only a single port that connects to the TLR410, which port is the first SR610 unless the SR fails and the connection moves to the second SR615. Connect to. Thus, this connection is shown as a dashed line in the figure.
The figure above shows the management plane view of the logical routers of some embodiments. In some embodiments, the administrator or other user provides a logical topology (in addition to other configuration information) through APIs. This data is provided to the management plane, which defines the implementation of the logical network (eg by defining DR, SR, transit logical switches, etc.). In addition, in some embodiments, the user associates each logical router (eg, each PLR or TLR) with a set of physical machines (eg, a set of predefined machines in a data center) for deployment. For a purely decentralized router, as in TLR405 implemented in Figure 5, DR is performed across the management forwarding components that reside on the host along with the data compute nodes connected to the logical network, so the physical machine The pair is not important. However, if the logical router implementation involves SRs, each of these SRs will be deployed on a particular physical machine. In some embodiments, a group of physical machines is a set of machines designated for the purpose of hosting an SR (as opposed to a user VM or a data compute node attached to a logical switch). In another embodiment, the SR is deployed on the machine along with the user data compute node.
In certain embodiments, the logical router user definition includes a certain number of uplinks. As described below, the uplink is the northbound interface of the logical router in the logical topology. For TLRs, the uplink connects to the PLR (generally, all uplinks connect to the same PLR). For PLR, its uplink connects to an external router. Some embodiments require that all of the PLR's uplinks have connectivity to the same external router, while other embodiments allow the uplink to connect to different sets of external routers. To. Once the user selects a group of machines for the logical router, if SR is required for the logical router, the management plane will link the logical router to physical machines in the selected group of machines. Assign each. The management plane then generates an SR on each machine to which the uplink is assigned. In some embodiments, it is possible to assign multiple uplinks to the same machine, in which case the SR on the machine will have multiple northbound interfaces.
As mentioned above, in some embodiments, the SR can be implemented as a virtual machine or other container, or as a VRF context (eg, in the case of a DPDK-based SR implementation). In some embodiments, the choice for SR implementation may be based on the services selected for the logical router and which SR type provides these services best.
In addition, the management plane of some embodiments produces a transit logic switch. For each of the passing logical switches, the management plane assigns a unique VNI to the logical switch, creates a port on each SR and DR that connects to the passing logical switch, and any SR and DR that connects to the logical switch. Assign an IP address to. Some embodiments require that the subnet assigned to each transit logic switch be unique within a logical L3 network topology with a large number of TLRs (eg, network topology 400), and each of the TLRs has its own transit logic. May have a switch. That is, in FIG. 6, the transit logic switch 525 in the PLR implementation, the transit logic switch 530-540 between the PLR and the TLR, and the transit logic switch 620 (and the transit logic switch in the implementation of any other TLR). Each requires its own subnet. In addition, in some embodiments, the SR may need to initialize a connection to a VM in the logical space, such as an HA proxy. To ensure that return traffic works, some embodiments avoid using link-local IP addresses.
Some embodiments impose various restrictions on the connection of logical routers in a multi-layer configuration. For example, some embodiments allow any number of layers of logical routers (eg, multiple layers of TLRs plus PLR layers that connect to external networks), others only have a two-tier topology. Allows (one layer of TLR that connects to the PLR). In addition, some embodiments allow each TLR to connect to a single PLR, and each user-generated logical switch (ie, not a transit logical switch) is in one PLR or one TLR. Allows you to connect. Some embodiments also add the constraint that each of the logical router's southbound ports must be on a different subnet. Therefore, two logical switches will not have the same subnet when connecting to the same logical router. Finally, some embodiments require that different uplinks of PLR must be on different gateway machines. It should be understood that some embodiments do not include any of these requirements, or include various different combinations of requirements.
II. SR and DR configuration
When a user configures a logical router (by configuring its interfaces and delivery routes), this configuration is used to configure SR and DR for the logical router by the management plane. For example, the logical router 115 in Figure 1 has four interfaces (two to the logical switch and two to the uplink). However, the implementation of its decentralized management plane in FIG. 2 includes a DR with each having three interfaces and an SR with two interfaces (seven interfaces in total). The IP and MAC addresses and other configuration details assigned to the four interfaces as part of the logical router configuration are used to generate configurations for the various components of the logical router.
In addition, as part of the configuration, some embodiments generate a routing information base (RIB) for each of the logical router components. That is, at least one of the management and control planes of some embodiments will generate a separate RIB for the DR and each SR, even if the administrator defines only a single logical router. To do. For the SR of the PLR, in some embodiments, the management plane first generates the RIB, but the physical implementation of the SR also performs dynamic routing protocol processing (eg BGP, OSPF, etc.). Supplement RIB locally.
Some embodiments include multiple types of routes within the RIB of a logical router, i.e., within the RIB of its component routers. In some embodiments, all routes have a management distance value (ie, two routes for the same prefix) used to determine the priority, with higher values indicating the type of route with lower priority. If present, routes with lower distance values are used). If multiple routes for the same prefix are in the RIB with the same distance value, traffic to these prefixes will be distributed across the different routes (eg, using the ECMP principle to balance traffic equally). connected (0): Prefix configured on logical router port static (1): Management plane internal (10) configured by administrator / user: Default Route-When a TLR is connected to a PLR, a default route to the PLR is added to the TLR's RIB, allowing the user to redistribute the subnet and NATing the subnet when a logical switch is connected to the TLR. If not, the default route to the subnet's TLR is added to the PLR's RIB EBGP (20): The following four types are routes learned via dynamic routing protocols OSPF internal (30) OSPF external (110) IBGP (200).
It should be understood that in some embodiments, not all logical routers include both BGP and OSPF routes, and some logical routers cannot. For example, a logical router that does not include a connection to an external network cannot use any routing protocol, and some logical routers have one type of routing protocol (route) rather than both BGP and OSPF. -sharing protocol) can only be executed.
Further, in some embodiments, the SR of the PLR (using a dynamic routing protocol) drives the RIB (including static, connected, and internal paths of the management plane) received from the centralized controller. Integrate into routes learned from physical routers (via target routing protocols). The SR internally calculates its FIB based on the integration of these dynamic routes to facilitate route convergence, rather than sending the learned routes back to the centralized controller for recalculation. For DR, the centralized controller of some embodiments pushes down the entire RIB to the local control plane that computes the FIB.
A. DR configuration
In some embodiments, the DR is always located on the southbound side of the logical router implementation (ie, in contact with the data compute nodes of the logical network rather than in contact with the external physical network). Unless the logical router has a centralized component, the logical router uplink is not configured for the DR and the DR's northbound interface instead connects to a transit logical switch that is part of the logical router. ..
Figure 7 conceptually shows a more detailed configuration of the logical network topology 700, including network addresses and interfaces assigned by the administrator. As shown, the logical switches 705 and 710 are assigned to their own subnets, 1.1.1.0/24 and 1.1.2.0/24, respectively, and all of the data compute nodes attached to the logical switch 705 are in the corresponding subnets. Has an IP address in. The logical router 715 has an interface L1 to the first logical switch 705, which has an IP address of 1.1.1.253, which is the default gateway to the data compute node on subnet 1.1.1.0/24. The logical router 715 also has an interface L2 to the second logical switch 710, which has an IP address of 1.1.2.253, which is the default gateway to the data compute node on subnet 1.1.2.0/24.
The northbound side of the logical router 715 has two uplinks, U1 and U2. The first uplink U1 has an IP address of 192.168.1.252 and connects to a first physical router 720 with an IP address of 192.168.1.252. The second uplink U2 has an IP address of 192.168.2.253 and connects to a second physical router 725 with an IP address of 192.168.2.252. The physical routers 720 and 725 connect the logical network to the external network, although they are not actually part of the logical network. In the illustrated example, each of the uplinks connects to a single different physical router, but in some cases, each of the uplinks connects to the same set of multiple physical routers. That is, both U1 and U2 are connected to both physical routers 720 and 725. Although the illustrated example is not such an example, some embodiments require each external router whose uplink connections provide the same connectivity. Instead, the first logical router 720 connects to subnet 10.0.0.0/8, while the second router 725 connects to both subnets 10.0.0.0/8 and 11.0.0.0.0/8.
For a logical router with decentralized components, some embodiments configure DR as follows. The southbound interface is configured in the same way as the southbound interface of a logical router. These interfaces connect to logical switches or low-level logical routers in the logical topology (for example, the PLR's southbound interface connects to the TLR). The DR of some embodiments is assigned a single northbound interface, which is assigned an IP address and a MAC address. Assuming that the logical router has one or more SRs, the DR's northbound interface connects with a transit logical switch.
The DR's RIB is assigned a route to be connected based on subnets configured on various southbound and northbound interfaces. There is a subnet configured for (i) a transit logical switch configured between the DR and SR components of a logical router, and (ii) any logical switch on its southbound interface. The logical switch on the southbound interface may be a user-defined logical domain to which the data compute node connects, or a transit logical switch located between the DR of the PLR and any TLR connected to the PLR.
In addition, any static routes exiting the logical router's uplink are included in the DR's RIB, but these routes are modified so that the IP address of the next hop is set to that of the uplink's SR. For example, the static route "abc0 / 24 via 192.168.1.252" (192.168.1.252 is the address of the external physical network router) is "abc0 / 24 via". Fixed to "SR Southbound Interface IP". Conversely, static routes out of the logical router's southbound are included in the RIB of the unmodified DR. In some embodiments, for each SR of the logical router, a default route inside the type management plane is added to the RIB of the DR. In other embodiments, instead, a dynamic route learned by a particular SR is added to the RIB with the IP address of the next hop modified as the Southbound IP of the particular SR. This is an alternative to the default route, because the management plane internal type has a higher priority than the dynamic route otherwise learned by SR. However, in some embodiments, for TLRs, the SR does not implement a dynamic routing protocol, so a default route with a next-hop IP address pointing to the interface of the active SR is used instead.
Figure 8 shows configuration 800 of the logical topology 700 with the management plane. As shown, the logical switches 705 and 710 are configured as specified by the user settings. As in the previous example, the logical router 715 includes a DR805, two SR810s and 815s, and a transit logical switch 820. The DR is assigned two southbound interfaces on the logical router 705, which connect to the logical switches 705 and 710. The transit logical switch is assigned the subnet 192.168.100.0/24, which must meet the requirement that it be unique among the logical switches that logically connect (directly or indirectly) to the logical router 705. is there. Each of the three management plane router configurations 805-815 also includes an interface connected to a transit logical switch and has an IP address within the subnet of the transit logical switch. Southbound interfaces U1 and U2 are assigned to the two SR810 and 815, the configuration of which is described below.
By using the rules of some embodiments described above for RIB generation, the DR805 RIB includes the following routes: 1.1.1.0/24 output to L1 1.1.2.0/24 output to L2 192.168.100.0 / 24 output to DRP1 192.168.1.0/24 via IP1 192.168.2.0/24 via IP2 10.0.0.0/8 via IP1 10.0.0.0/8 via IP2 11.0.0.0/8 via IP2 0.0.0.0/0 via IP1 0.0.0.0 / 0 via IP2
The above routes include three connected routes to the logical switch domains (1.1.1.0/24, 1.1.2.0/24 and 192.168.100.0/24) connected to the DR. In addition, the subnet (192.168.1.0/24) where the first uplink is located is reached via the southbound interface of the first SR810 (IP1), while the subnet where the second uplink is located. (192.168.2.0/24) is reached via the Southbound interface of the second SR815 (IP2). In addition, three static routes have been added by the user to the logical router 715, which the management plane automatically modifies for the DR805. Specifically, the route includes network 10.0.0 / 8 through the southbound interface of both SRs and network 11.0.0 / 8 through the southbound interface of SR2. Finally, a default route pointing to these identical southbound interfaces is included. The IP addresses IP1, IP2 and IP3 generated by the management plane for the ports of the logical router construct an interface with transit logical switches, all on the subnet 192.168.100.0/24.
B.SR configuration
Like the logical router DR, the management plane also configures each SR of the logical router with separate RIBs and interfaces. As mentioned above, in some embodiments, both the PLR and TLR SRs provide services (ie, functions beyond mere routing such as NAT, firewall, load balancing, etc.) and SRs for PLRs. Also provides a connection between the logical network and the external physical network. In certain embodiments, SR practices are designed to achieve several objectives. First, implementation ensures that the service scales out. That is, the service assigned to the logical router can be realized by any of several SRs of the logical router. Second, some embodiments configure the SR so that the service policy can rely on routing decisions (eg, interface-based NAT). Third, the SR of a logical router requires the involvement of a centralized control plane or management plane (although in some embodiments the SR can operate under reduced capacity or in suboptimal aspects). It has a function to handle its own malfunction (for example, malfunction of the physical machine on which SR operates, malfunction of the tunnel to the physical machine, etc.). Finally, the SR ideally eliminates unnecessary redirects within itself. That is, if the SR has a function of forwarding packets internally, the packets must be forwarded to the external physical network, and if necessary, only packets to different SRs are forwarded. Of course, forwarding between SRs must avoid packet loops.
As shown in FIG. 8, each SR has one southbound interface that connects to the transit logic switch 820 between the SR and DR. Moreover, in some embodiments, each SR has the same number of northbound interfaces, such as a logical router. That is, only one uplink can be assigned to the physical machine running the SR, but all of the logical router interfaces are defined on the SR. However, while some of these interfaces are local interfaces, some of these interfaces are referred to as dummy interfaces.
In some embodiments, the local northbound interface is an interface that allows packets to exit directly from the SR (directly to the physical network). An interface configured based on the uplink (or one of multiple uplinks) assigned to the SR is a local interface. On the other hand, an interface configured based on one of the other uplinks of a logical router assigned to a different SR is referred to as a dummy interface. Providing an SR with a configuration for a dummy interface allows the MFE on the first hop to send a packet for one of the uplinks to any SR, at which time the packet is destined for its local interface. The SR can process the packet even if it does not. In some embodiments, after processing a packet for a dummy interface in one of the SRs, the packet can be forwarded to the appropriate SR where the interface is local so that the other SR can send the packet to the external physical network. To. The use of dummy interfaces allows the centralized controller (or set of controllers) that manages the network to push service policies that rely on routing decisions to all SRs so that services can be delivered by any SR. Become.
In some embodiments, the SR exchanges routing information with the physical network (eg, using a route advertisement protocol such as BGP or OSPF), as described in Section IV below. One purpose of routing is to make routing decisions always on the SR's local interface or on a dummy interface that corresponds to the uplink of a logical router on a different SR, regardless of which SR routes the packet to the physical network. It is to be directed to either. Therefore, the policy associated with the logical router uplink can be applied by the SR even if the uplink is not assigned to the SR, which allows for stateful service scale-out. In some embodiments, only when the SR cannot send packets directly to the physical network router because the route received from the peer SR has a greater distance value than the route learned directly from the physical next-hop router. , Ensure that the SR sends packets to its peer SR.
For a logical router with one or more centralized components, some embodiments configure SR as follows. For northbound interfaces, the SR has as many such interfaces as logical routers, and each of these interfaces inherits the IP and MAC address of the corresponding logical router interface. A subset of these interfaces are marked as local interfaces (for uplinks assigned to SRs), while the remaining interfaces are marked as dummy interfaces. In some embodiments, the service policy defined for the logical router is pushed evenly to all SRs. This is because service policies are similarly configured from a network and interface perspective. The dynamic routing configuration for the port / uplink of a particular logical router is forwarded to the local interface of the SR to which that particular uplink is assigned.
As mentioned, each SR is assigned a single southbound interface (also a local interface) that connects to the transit logical switch, where each SR's southbound interface is connected to the same transit logical switch. .. The IP address of each of these southbound interfaces is in the same subnet as the northbound interface assigned to the DR (that of the transit logical switch). Some embodiments have different IP address assignments between SRs, depending on whether the SR is in active-active or active-standby mode. For active-active mode (ie, when all SRs are treated equally for routing purposes), different IP and MAC addresses are assigned to the southbound interfaces of all SRs. On the other hand, for active-standby mode, the same IP is used for both the southbound interfaces of the two SRs, while each interface is assigned a different MAC address.
As shown in the above subsection for DR, the user can configure a static route to the logical router. The static route (or connected route) of the logical router exiting the uplink is copied to the SR RIB. If the uplink out of the route is assigned to the SR, the distance criteria for such a route will not be modified. However, if the SR uplink is a dummy interface, some embodiments prefer the route out of that local interface when the SR can reach the network through the dummy interface without redirecting packets to different SRs. Add a value to this criterion to do so. In addition, the SR (of the top-level logical router) learns the dynamic path and puts it in the RIB (although some embodiments do this internally without the use of a centralized controller). In some embodiments, the dynamic path learned from the peer SR is installed without distance reference adjustment. This is because by default the criteria for routes learned from IBGP (SR to SR peers) or OSPF are greater than the criteria for routes learned from EBGP.
For each of the southbound interfaces of the logical router, some embodiments add a route for the network corresponding to the RIB of each SR. This route points to the northbound DR interface as the IP address of the next hop. In addition, any other route configured for the logical router out of the southbound interface is copied to the SR along with the same northbound DR interface as the IP address of the next hop.
Returning to the example in Figure 8, the management plane defines two service routers 810 and 815 so that the logical router 715 has two uplinks. The first service router 810 has a local interface to U1 and a dummy interface to U2 referred to as U2'. Similarly, the second service router 815 has a local interface to U2 and a dummy interface U1'to the first uplink U1. The functionality of these dummy interfaces is described above as they are used to redirect packets to other SRs. Each of these SRs is assigned a Southbound interface with a different IP address and MAC address (as the SRs are in an active-active configuration). IP address IP1 (for the first SR810) and IP2 (for the second SR815) are on subnet 192.1.100.0/24, like IP3 (the northbound interface of the DR805).
Assuming that the rules of some embodiments are used and that a routing protocol (eg BGP) is enabled for SR, the RIB of the first SR810 includes the following routes: 10.0.0.0/8 output to U1 via 192.168.1.252, metric 20 (via EBGP) 10.0.0.0/8 output to U2'via 192.168.2.252, metric 200 (via IBGP) 11.0.0.0/8 output to U2' via 192.168 .2.252, metric 200 (via IBGP) 192.168.1.0/24 output to U1, metric 0 (connected) 192.168.100.0/24 output to SRP1, metric 0 (connected) 1.1.1.0/24 via IP3, metric 10 (connected) In the management plane) 1.1.2.0/24 via IP3, metric 10 (in the management plane)
Similarly, the RIB of the second SR815 includes the following routes: 10.0.0.0/8 output to U2 via 192.168.2.252, metric 20 (via EBGP) 10.0.0.0/8 output to U1'via 192.168.1.252, metric 200 (via IBGP) 11.0.0.0/8 output to U2 via 192.168. 2.252, metric 20 (via EBGP) 192.168.2.0/24 output to U2, metric 0 (connected) 192.168.100.0/24 output to SRP2, metric 0 (connected) 1.1.1.0/24 via IP3, metric 10 (managed) In the plane) 1.1.2.0/24 via IP3, metric 10 (in the management plane)
C. Management plane processing
FIG. 9 conceptually illustrates the process 900 of some embodiments that make up the PLR based on user specifications. In some embodiments, process 900 is performed by a management plane (eg, a set of modules in a centralized controller that manages a network of data centers). The management plane performs configuration processing and then uses the controller's centralized control plane (or different network controllers) to send data to different local control planes on different host machines running the configured logical routers. Disperse.
As shown, processing 900 begins by receiving the PLR specification (at 905). The PLR specification is based on the administrator input (the administrator hired by the data center owner) that defines the PLR. In some embodiments, this specification defines any service that the PLR should provide, whether the PLR is configured in active-active or active-standby mode (although some embodiments are stateful services). Automatically use active-active mode unless configured), how many uplinks are configured in the PLR, uplink IP and MAC addresses, uplink L2 and L3 connectivity, and any Southbound interface on the PLR. Subnet (one interface if the PLR has a two-tier topology, any number of interfaces if the user's logical switches connect directly in the one-tier topology), any static path to the PLR's RIB, and Includes other data. It should be understood that different embodiments may include different combinations of data or other data listed in the PLR configuration data.
Process 900 then uses this configuration data (at 910) to define the DR. This assumes that the management plane will not generate DR if the PLR is not fully centralized. For the DR southbound interface, the management plane uses the PLR southbound interface configuration. That is, the DR IP address and MAC address are those specified for the logical router.
In addition (in 915) the process assigns each of the uplinks identified for the PLR to the gateway machine. As mentioned above, some embodiments allow (or require) the user to identify a particular set of physical gateway machines with respect to the SR location of the logical router. In some embodiments, a set of gateway machines exists within a particular rack of servers or a group of racks of servers, or otherwise associated with a tunnel connecting all machines within the set. The management plane then assigns each uplink to one of the selected set of gateway machines. Some embodiments allow multiple uplinks to be assigned to the same gateway machine (unless the logical router has only two uplinks configured in active-standby mode), while other embodiments. Allows only a single uplink to be assigned to the gateway for PLR regardless of whether it is in active-active or active-standby mode.
After assigning the uplink to the gateway machine, process 900 (at 920) defines an SR on each of the two gateway machines. For each SR, the process uses the configuration for the uplink assigned to the gateway machine as the configuration for the SR's northbound interface. This configuration information has the IP and MAC addresses of the uplinks, in addition to any uplink-specific policies. In situations where at least one of the different policies and L 3 connectivity is allowed and used between different uplinks, some embodiments are also dummy on the SR to redirect packets as needed. It should be understood that it constitutes an interface.
The process also defines a transit logic switch that connects with the defined SR and DR (in 925). In some embodiments, the management plane assigns a unique VNI (logical switch identifier) to the transit logical switch. In addition, some embodiments require that the subnet assigned to the transit logical switch be unique within the logical network topology. That is, a pass-through logic switch is any user-defined logic switch that has a direct interface with the PLR, a pass-through logic switch between any TLR connected to the PLR, and all pass-through logic switches within these TLRs. And you must use a different subnet than any user-defined logical switch connected to these TLRs.
Process 900 then assigns the DR (at 930) a northbound interface. In some embodiments, the northbound interface is assigned both a MAC address and an IP address (used for packets transmitted internally between the components of the PLR). In some embodiments, the IP address is within the subnet assigned to the transit logical switch as defined in 925. The configuration of the transit logical switch includes the association between this MAC address and one of its logical ports.
The subsequent process (in 935) determines if the PLR is configured in active-active mode (or active-standby mode). In some embodiments, as mentioned above, this determination is made by the administrator as part of the configuration settings for the PLR. In another embodiment, the management plane automatically defines the SR for the PLR in an active-active configuration when stateful services are not configured. And if stateful service is configured, SR is defined in active-standby mode.
When the PLR is configured in active-standby mode, the process (at 940) assigns each southbound interface of two SRs (or two or more SRs if there are multiple standbys). For active-standby, all these southbound interfaces have the same IP address, which is within the subnet of the transit logical switch defined in operation 925. The two interfaces accept the same IP address, but assign different MAC addresses so that they are different destinations for Northbound packets routed by the DR.
The subsequent process (in 945) assigns one of the SRs to the active and one of the SRs to the standby. Some embodiments allow this decision to be made randomly, while others attempt to balance the active and standby allocations of SRs across gateway machines. More details can be found in US Patent Gazette 2015/0063364. The SR assigned as active responds to ARP requests for the Southbound interface and advertises the prefix from the Northbound interface to the external physical network. The standby SR, on the other hand, does not respond to ARP requests (to avoid receiving northbound traffic) and does not advertise the prefix (but receives routes from the external network in the event of an active SR malfunction). Maintain a BGP session for).
Finally, processing 900 (at 950) produces a separate RIB for the DR and each SR. Separate RIBs are generated based on the configuration data in the format shown in the previous subsection and section V. After that, the process ends. In some embodiments, the management plane also calculates the FIB intensively, while in other embodiments, the local control plane (running on host and gateway machines) is the actual packet from the logical router component. Perform RIB traversal to generate FIB for use in forwarding. In either case, the RIB is updated on the SR based on a dynamic path learned from the external network, and this data is transmitted to the DR via the central controller. The calculation of the FIB by the network controller in some embodiments is described in more detail in US Patent Application No. 14 / 214,545, filed March 14, 2014.
On the other hand, if the PLR is configured in active-active (EMCP) mode, the process (in 955) assigns a southbound interface to each of the SRs. When active-active, these southbound interfaces are assigned different MAC addresses, as well as different IP addresses within the subnet of the transit logical switch defined in operation 925. The different IP addresses allow each of the SRs to process northbound packets based on the IP address selected for a given packet by the DR pipeline in the host machine.
The process then (in 960) assigns a rank to the SR. As detailed below, SRs use ranks to determine which SR will take over the responsibilities of a stopped SR in the event of a malfunction. In some embodiments, the next highest ranked SR takes over the SR that was stopped by taking over its southbound interface, otherwise carrying northbound traffic sent to the IP address of the stopped SR. Pull in.
Finally, the process (in 965) produces a DR and a separate RIB for each SR. Separate RIBs are generated based on the configuration data in the format shown in the previous subsection and section IV. After that, the process ends. In some embodiments, the management plane also calculates the FIB intensively, while in other embodiments, the local control plane (running on host and gateway machines) is the actual packet from the logical router component. Perform RIB traversal to generate FIB for use in forwarding. In either case, the RIB is updated on the SR based on a dynamic path learned from the external network, and this data is transmitted to the DR via the central controller.
The description in Figure 9 above shows the operation of the management plane to generate various components for the PLR (Upper Logical Router). FIG. 10 conceptually illustrates process 1000 of some embodiments that make up a TLR based on user specifications. In some embodiments, process 1000 is performed by a management plane (eg, a set of modules in a centralized controller that manages a network of data centers). The management plane performs configuration processing and then uses the controller's centralized control plane (or different network controllers) to send data to different local control planes on different host machines running the configured logical routers. Disperse.
As shown, processing begins by receiving the TLR specification (at 1005). The TLR specification is based on administrator input (the administrator hired by the data center tenant) that defines the TLR. In some embodiments, this specification defines any service that the TLR should provide, which PLR the TLR should connect to through its uplink, any logical switch that connects to the TLR, the IP of the TLR interface, and Includes MAC address, any static route to the TLR's RIB, and other data. It should be understood that different embodiments may include different combinations of data or other data listed in the TLR configuration data.
Process 1000 then determines (at 1010) whether the TLR has a centralized component. In some embodiments, if the TLR does not provide stateful services, then no SR is defined for the TLR, which is only implemented in a decentralized form. On the other hand, some embodiments require SR in active-standby mode when stateful services are provided, as shown in this figure.
If the TLR does not provide stateful services or otherwise requires a centralized component, the process (in 1015) defines the DR with the logical router specifications for both southbound and northbound interfaces. The DR can have a number of southbound interfaces, depending on how many logical switches are defined to connect to the TLR. On the other hand, some embodiments limit the TLR to a single northbound interface that sends and receives packets from the PLR. The process then produces a RIB for the DR (at 1020). The RIB for DR includes all of the routes for the logical router generated as described above.
On the other hand, if the TLR provides stateful services or requires centralized components for other reasons, the process (at 1025) defines the DR with the received configuration data. For the DR southbound interface, the management plane uses the TLR southbound interface configuration. That is, the IP address and MAC address for each southbound interface are specific for the port of the logical router to which the various logical switches connect.
In addition (at 1030) the process assigns each of the uplinks identified for the TLR to the two gateway machines. Some embodiments allow the TLR to operate in active-active mode with multiple uplinks, but process 1000 is a single uplink in active-standby mode (links transfer TLRs to other logical routers). For embodiments that limit the TLR to (also referred to as a router link) as a connection. As mentioned above, some embodiments allow (or require) a user to identify a particular set of physical gateway machines with respect to the SR location of the logical router. In some embodiments, a set of gateway machines exists within a particular rack of servers or a group of racks of servers, or otherwise associated with a tunnel connecting all machines within the set. The management plane then assigns uplinks to two of the selected set of gateway machines.
After assigning the uplink to the gateway machine, process 1000 (at 1035) defines an SR on each of the two gateway machines. For each SR, the management plane uses the configuration for a single uplink as the configuration for the SR's northbound interface. Since it has only one northbound interface, the process applies the same configuration to both SRs. That is, not only is the same IP address used for both northbound interfaces, but the services on the interfaces are configured in a similar way. However, different MAC addresses are used for the northbound interface so that the active SR and the standby SR are different.
The process also defines a transit logic switch that connects with the defined SR and DR (in 1040). In some embodiments, the management plane assigns a unique VNI (logical switch identifier) to the transit logical switch. In addition, some embodiments require that the subnet assigned to the transit logical switch be unique within the logical network topology. That is, the management plane connects to the same PLR as any user-defined logical switch that has an interface with the TLR and any transit logical switch between the TLR (or other TLR) and the PLR. All passing logical switches in other TLRs, passing logical switches in PLRs, and user-defined logical switches connecting to other TLRs must be assigned different subnets to passing logical switches.
The process then assigns a northbound interface to the DR (at 1045). In some embodiments, this interface is assigned both a MAC address (used for packets transmitted internally between TLR components) and an IP address. In some embodiments, the IP address is within the same subnet assigned to the transit logical switch at 940. The process also assigns (at 1050) the southbound interface for each of the two SRs. Since this is an active-standby configuration, these southbound interfaces have the same IP address, which is within the subnet of the transit logical switch defined in operation 940. The two interfaces accept the same IP address, but are assigned different MAC addresses so that they are different destinations for Northbound packets routed by the DR.
Then processing 1000 (at 1055) assigns one of the SRs to the active and one of the SRs to the standby. Some embodiments allow this decision to be made randomly, while others attempt to balance the active and standby allocations of SRs across gateway machines. The SR assigned as active responds to ARP requests for the southbound interface (from the DR of this TLR) and the northbound interface (from the DR of the PLR). The standby SR, on the other hand, does not respond to ARP requests (to avoid receiving northbound or southbound traffic).
Finally, the process (at 1060) produces a separate RIB for the DR and each SR. Separate RIBs are generated based on the configuration data in the format shown in the previous subsection and Section IV below. In some embodiments, the management plane also calculates the FIB intensively, while in other embodiments, the local control plane (running on host and gateway machines) is the actual packet from the logical router component. Perform RIB traversal to generate FIB for use in forwarding. In either case, the RIB is updated on the SR based on a dynamic path learned from the external network, and this data is transmitted to the DR via the central controller.
Regardless of whether the TLR was generated with or without an SR, process 1000 (at 1065) defines another transit logic between the TLR and the PLR to which the TLR connects. This transit logic switch has a unique VNI and a subnet to which the TLR's uplink IP address belongs. In addition, the interfaces on the DR of the PLR are created in the same subnet to connect to the transit logic switch. After that, the process ends.
Although processes 900 and 1000 indicate a particular order in which these various actions are performed, it should be understood that these processes are merely conceptual. In a variety of different embodiments, the management plane may perform actual operations in a variety of different orders or in parallel. For example, the management plane can first define a transit logical switch before defining any SR or DR, and then completely define all of the logical router components before assigning them to separate physical machines.
III. Routing table configuration
As mentioned above, in some embodiments, the management plane receives the logical router configuration and (i) for one or more logical router configurations for each logical router and (ii) for each logical router configuration. Generate a routing information base (RIB). In a variety of different embodiments, the calculation of the forwarding information base (FIB) from the RIB may be performed by the management plane or by a centralized or local control plane. For example, in some embodiments, the RIB is distributed to each machine that implements SR or DR, and the local network controller running on that machine performs route traversal to compute the FIB. This section describes in more detail the rules for generating RIBs for various logical router configurations in some embodiments. The route may come from the logical switch's connection to the logical router, the connection of each other's logical routers, or the input of a static route (received as part of the logical router configuration).
FIG. 11 shows a more complex logical network configuration 1100 used to show the RIB generation rules of some embodiments. Specifically, the logical network configuration 1100 is a PLR1105 with an attached logical switch 1120, including two TLR1110s and a PLR1105 also attached with 1115. The first TLR1110 has two attached logical switches 1125 and 1130, and the second TLR1115 has two attached logical switches 1135 and 1140. PLR1105 is configured with three uplinks U1-U3, U1 has an IP address of 1.1.3.2 and is connected to an external physical router 1.1.3.1. U2 connects to two external physical routers with IP addresses of 1.1.4.2 and on the same subnet with IP addresses of 1.1.4.1 and 1.1.4.11. U3 is on the same subnet with the same L3 connectivity as U2 and has an IP address of 1.1.4.21. The PLR also has three south-facing interfaces, one for logical switch 1120 (with an IP address of 1.1.0.1 and within subnet 1.1.0.0/24 of logical switch 1120) and D1 for TLR1110 and 1115. And one for each of D2. In some embodiments, the user who attaches the TLR to the PLR (ie, the owner of the TLR) does not configure this interface. Instead, it will be shown in the management plane view of Figure 12 that follows.
In some embodiments, the data center administrator configures the PLR (ie, the uplink and its connectivity) and the first logical switch 1120. The first tenant constitutes the first TLR1110 and two logical switches 1125 and 1130, while the second tenant independently constitutes the second TLR1115 and two logical switches 1135 and 1140. In other embodiments, a single administrator (eg, in the case of an enterprise network) constitutes the entire logical network 1100. When different users configure different logical routers, some embodiments limit the subnets available for the various logical switches 1120-1140 and make the IP address public (eg, uplink U1-U3). Make sure that different subnets are used for each of the logical switches when advertising to an external network via. Some embodiments allow the logical switch to reuse a subnet unless the IP address is advertised externally (including to other TLRs that connect to the same PLR).
FIG. 12 conceptually illustrates the logical network 1100 generated by the management plane based on the configuration shown in FIG. 11 (as user input via API). In this case, neither TLR1110 nor 1115 includes stateful services, so SR is not required for TLRs. As explained in the previous section, TLR1110 and 1115 include DR1205 and 1210, respectively. The south-facing port of the TLR to which the logical switch connects is the same as the IP address given to the TLR in the logical network configuration 1100 (eg 1.1.5.1, 1.1.6.1, etc.) (in the subnet of these installed logical switches). ) An IP address is assigned. In addition, for each connection between the TRL and PLR, the management plane assigns transit logical switches and assigns the IP addresses of the north-facing interfaces to DR1205 and 1210 on these logical switches. In some embodiments, each of these transit logical switches requires only two addresses, so the management plane always allocates / 31 subnets from the pool of subnets so as not to waste IP address space. In this case, the first pass logical switch 1215 has a subnet of 192.168.10.0/31, while the second pass logical switch 1220 has an adjacent subnet of 192.168.10.2/31.
The management plane also defines the DR1225 and three SR1230-1240s (for each uplink) for the PLR1105. The DR1225 has three south-facing interfaces of the PLR1105 (as described in the previous section) in addition to the four interfaces P1 defined by the management plane. This interface P1 connects to a transit logic switch 1245 with a subnet of 169.0.0.0/28, which also connects to the south-facing interfaces of the three SR1230-1240s. These four interfaces P1-P4 have IP addresses of 169.0.0.1, 169.0.0.2, 169.0.0.3 and 169.0.0.4, respectively.
Figures 13, 15, 16 and 17 show how many logical router configuration data executed by the management plane (eg in a centralized network controller) are transformed into RIBs for SR of DR and PLR in some embodiments. The various processes of the embodiment are conceptually shown. Specifically, FIG. 13 and FIG. 15 illustrate the treatment of different types of static routes for PLRs, while FIG. 16 shows the operation of connecting TLRs to PLRs and generating routes for PLR constructs. The treatment for the target routing process is explained, and FIG. 17 describes the generation of new logical interfaces on the PLR (both uplinks to external networks and downlinks to TLRs or logical switches). In some embodiments, these operations are performed by a management plane (eg, a set of modules in a centralized controller that manages a network of data centers). The management plane performs the configuration process and then uses the centralized control plane of the controller (or different network controller) to distribute the data to different local control planes on different host machines that implement the logical router configuration. Let me. These processes are partially described by referring to the logical network 1100 shown in FIGS. 11 and 12 and the route configuration inputs and RIB outputs shown in FIG.
FIG. 13 conceptually illustrates process 1300 in some embodiments that process a static route towards the south that is input as configuration information for PLR (eg for PLR1105). As shown, processing 1300 begins (at 1305) by receiving a static route with the next hop in the logical network. In some embodiments, the static route is entered with a prefix to be routed, the next hop (or gateway) of the route, and, in some cases, an interface that outputs the route. When in use, any packet with a destination IP address that matches the prefix (and does not match a route with a longer prefix) is output to a given next hop address via a given interface. If the next hop is in one of the subnets of the logical switch attached to the PLR, or one of the subnets of the logical switch attached to the TLR attached to the PLR, then the route is within the logical network. It is a route to the south with the next hop of.
FIG. 14 conceptually illustrates the set of static routes 1405 configured for PLR1205 and the configuration for TLRs described below with reference to FIG. This figure also shows the output 1400 produced by the management plane for the DR and the three SRs based on this configuration data. This output includes the RIB1410 for the DR1225 and the RIB1415-1425 for the three SR1230-1240s. As shown, set of static routes 1405 configured for PLR identifies all packets sent to 1.1.0.2 with a destination address within subnet 2.2.2.0/24, South Includes the direction route (2.2.2.0/24 via 1.1.0.2). This address belongs to the VM attached to logical switch 1120.
When such a route is received, process 1300 adds the route (at 1310) to the RIB of the DR of the PLR without modification. That is, the next hop of the route added to the RIB of the DR is the same as the next hop specified in the configuration data. In the example shown in Figure 14, the configured route 2.2.2.0/24 via 1.1.0.2 (the fourth route listed in the RIB) is added to RIB1410 for DR without modification.
Process 1300 also adds a route as the next hop (without removing any output interface) to the RIB of each SR with the DR's north-facing interface. That is, the route for the identified prefix is that of the DR interface that connects to the same transit logical switch as the SR's south-facing interface, along with the next hop (packets sent from the external physical network to a specific IP address). Is added to each of the SRs). For logical network 1100, this is interface P1 with an IP address of 169.0.0.1. As such, each RIB1415-1425 is route 2.2.2.0/24 via It has 169.0.0.1. After adding a route to each RIB of the logical router configuration for the PLR, processing 1300 ends. In some embodiments, the management plane adds a south-facing static route to the SR's RIB without modifying it, rather than modifying it, as long as the SR has the ability to estimate recursive routes. In such an embodiment, the management plane of this example recognized that once the route was traversed, route 1.1.0.0/24 via 169.0.0.1 would result in the correct result in the SR FIB. Above, add route 2.2.2.0/24 via 1.1.0.2.
While process 1300 is used to handle southbound static routes, FIG. 15 shows some embodiments that process northbound routes that are entered as configuration information for PLR (eg for PLR1105). The processing 1500 of is conceptually shown. In some embodiments, processes 1300 and 1500 have different actions that are performed depending on the decision made by the process as to whether the route is south-oriented or north-oriented, in fact a single operation. It is part of software processing.
As shown, processing 1500 begins by receiving a static route (at 1505) with the next hop for the external physical network. At this time, if the next hop is that of a physical router that provides connectivity to the logical network to the external physical network, the route is a north-direction route having the next hop for the external physical network. The set of static routes 1405 configured for PLR consists of three different routes for the prefix 0.0.0.0/0 (ie the default route), routes for 172.16.0.0/16, and 10.0.0.0/ Includes a number of northbound routes, including routes for 8. Each of these routes identifies the next hop belonging to an external router (ie, one of 1.1.3.1, 1.1.4.1 and 1.1.4.11).
The process then determines (in 1510) whether the output interface has been identified for the route. As mentioned above, in some embodiments, the configured static route contains the address of the next hop, and in some cases, an egress packet with a destination address within the subnet through the interface of the logical router. Is routed. As long as the route appears in PLR configuration data 1405, the three default routes (for 0.0.0.0/0) identify the output interface. In particular, these routes represent next-hop physical routers for each of the three north-facing interfaces U1, U2, and U3.
If the route identifies an output interface, the process identifies (in 1515) the SR associated with the identified interface. In some embodiments, different SRs are assigned to each of the PLR uplinks, as described in the previous section. For example, in the logical network 1100 example, the PLR1105 uplinks U1, U2, and U3 are associated with the three SR1230, 1235, and 1240, respectively.
Subsequent processing 1500 (at 1520) adds a route to the RIB of the identified SR. The route is not added to any other SR because the northbound packets processed by the route should not be sent to other SRs. Therefore, in the example of Figure 14, the route 0.0.0.0/0 via 1.1.3.1 output to U1 is added only to the RIB1415 for the first SR1230 owned by the output interface U1. Similarly, route 0.0.0.0/0 via 1.1.4.1 output to U2 is added only to RIB1420 for the second SR1235, and route 0.0.0.0/0 via 1.1.4.11 output to U3 is the third SR1240. Only added to RIB1420 for.
On the other hand, if the route does not identify an output interface, the process identifies (at 1525) all SRs that have connectivity to the identified next hop. If all SRs have equal L3 connectivity (eg, all uplinks connect to the same set of physical routers), this is all of the SRs. However, in some cases, the administrator may configure PLR uplinks with different L3 connectivity, in which case some SRs will not have connectivity to the next hop for a particular route. Maybe.
The process then adds a route (at 1530) to the RIB for each of the identified SRs. PLR configuration 1405 contains two static routes that do not have a specified output interface. The first such route is for the prefix 172.16.0.0/16 with the next hop address 1.1.3.1. In this case, the management plane only adds a route to RIB1415 for this first SR, as only SR1230 has connectivity to the next hop. This route is not added to RIB 1420 and 1425. The other route is for the prefix 10.0.0.0/8 with the next hop 1.1.4.1. Routes are added to RIB1420 and 1425, respectively, of SR1235 and 1240, and not to RIB1415, because the two uplinks (U2 and U3) have connectivity to this next hop.
In addition to adding routes to one or more SRs, process 1500 also adds routes to the DR in some cases. Specifically, the process determines (in 1535) whether the route is the default route. In some embodiments, the default route is for the prefix 0.0.0.0/0 (ie, the entire IPv4 address space). In the longest prefix matching system, such a route is used when the packet does not match any other route in the routing table. When a logical network end machine (eg, a VM) acts as a server for remote clients accessing the machine over the Internet, then the default route is often used for reply communication.
If the route is the default route, process 1500 does not add any route to the DR and terminates the process. Thus, in the example, the three default routes are not added to the DR. However, in some embodiments, the management plane always adds a default route to the DR as part of the configuration that produces multiple logical router configurations for the PLR. It sends a packet (using the ECMP principle) unless it matches some SR. Therefore, RIB1410 for DR1225 includes default routes 0.0.0.0/0 via 169.0.0.2, 169.0.0.3, 169.0.0.4 (addresses of south-facing interfaces P2-P4 of the three SRs). This route sends a packet to one of the three SRs.
For non-default static routes, the process adds to the DR (in 1540) a route with the SR's south-facing interface as the next hop for each SR to which the route has been added. Process 1500 is then terminated. For example, only route 172.16.0.0/16 via 1.1.3.1 was added to RIB1415 for the first SR1230. This route is then added to the RIB1410 for the DR1225 as 172.16.0.0/16 via 169.0.0.2 (address of P2, south-facing interface of the first SR1230). Meanwhile, routes 10.0.0.0/8 via 1.1.4.1 have been added to RIB 1420 and 1425. At this time, the management plane adds a route with two next hops to RIB1410 for DR as 10.0.0.0/8 via 169.0.0.3, 169.0.0.4. For example, a packet with a destination IP address of 10.0.1.1 is sent to either SR1235 or 1240, depending on how the ECMP principle is applied by the DR implementation.
As can be seen in Figure 14, the RIB for various logical router configurations for logical routers will include other routes in addition to static routes. These routes include connected routes to which various logical switches are connected to the router configuration. For example, the DR1225 is connected to four logical switches, namely three transit logical switches 1215, 1220 and 1245, and a user-generated logical switch 1120. For each of these, a route with an output interface (rather than the next hop) is added to the DR. Therefore, the RIB1410 is route 1.1.0.0 output to D3 (for user-generated logical switches), route 192.168.10.0/31 output to D1 and 192.168.10.2/31 output to D2 (for transit logical switches between PLR and TLR). , And path 169.0.0.0/28 output to P1 (for transit logic switch inside PLR). In some embodiments, the SR is a connected path for a transit logic switch inside the PLR (eg 169.0.0.0/28 for the same RIB1415). Has output to P2). However, in some embodiments within this example, the SR does not have to have a connected path for external L2 connectivity. However, in other embodiments, if L2 connectivity is configured by the administrator, the SR will have such a connected path (eg, 1.1.3.0/24 output to U1 for RIB1415 of the first SR1230, and Includes 1.1.4.0/24 output to U2 / U3) for the other two SR1235 and 1240 RIB1420 and 1425. In addition, for each uplink interface IP (1.1.3.2, 1.1.4.2 and 1.1.4.21), some embodiments are the Southbound interface of the SR corresponding to the next hop, these particular "/ 32". Routes for IP addresses (ie 1.1.3.2 / 32 via 169.0.0.2, 1.1.4.2 / 32 via 169.0.0.3, and 1.1.4.21 / 32 via 169.0.0.4) is automatically inserted into the DR. Some embodiments are also routes for subnets (ie 1.1.3.0/24 via 169.0) where the uplink connects to the DR's RIB, again the next hop is the corresponding SR southbound interface, as illustrated. Insert .0.2 and 1.1.4.0/24 via 169.0.0.3, 169.0.0.4).
In addition, the PLR's logical router configuration may include routes dynamically generated by the management plane based on the connectivity of other logical routers (eg, TLRs). FIG. 16 conceptually illustrates the process 1600 of some embodiments that generate a RIB for the PLR's routing construct when the TLR is attached to the PLR. Since logical router configuration information is known by the management plane (eg, a set of network controllers), some embodiments should avoid performing BGP or similar dynamic routing protocols that exchange routes within the logical network. (As mentioned above, BGP / OSPF may be used to advertise routes to external physical networks).
As shown, processing 1600 begins by receiving a TLR northbound interface connection (at 1605) to the PLR's southbound interface. In some embodiments, the TLR owner (data center tenant) enters this as configuration data via the management plane API. The tenant does not have to need the interface information for the PLR to identify such a connection. Instead, the tenant requests to connect to the TLR and the management plane automatically generates a transit logical switch between them (eg from a pool of available subnets) along with the TLR northbound and PLR southbound interfaces. (Ensure that these interfaces are IP addresses within the subnet attached to the transit logical switch). In the logical network 1100 of Figure 11, the user has just installed TLR1110 and 1115 on PLR1105 (possibly as separate actions).
The process then identifies (at 1610) any path from the attached TLR. The TLR has a connected path for each of its logical switches. In addition, the TLR may also include, in certain embodiments, a static route configured by the user. However, in other embodiments, the TLR user does not have to configure a static route, or the static route does not have to be propagated to the PLR via dynamic routing (eg, different tenants are the same). Because you don't have to configure different routes to the address). In addition, in some embodiments, the tenant should have these logical switch subnets advertised (ie, these subnets are outside the TLR and (other TLR South machines or machines outside the logical network). You may specify (whether or not to communicate) with any of the above. In the example in Figure 14, the first TLR1110 advertises to both its subnets 1.1.5.0/24 and 1.1.6.0/24, while the second TLR1115 is 1.1. Advertise only on 7.0 / 24 (do not advertise on 1.1.8.0/24).
Subsequent processing 1600 transmits these routes to DR and SR. As shown, the process adds (in 1615) a route identified to the DR (of the PLR) with the TLR's northbound interface as the next hop. In this example, TLRs 1110 and 1115 are fully dispersed, so these northbound interfaces belong to TLRs DR1205 and 1210. Therefore, RIB1410 of DR1225 of PLR is the route for 1.1.5.0 and 1.1.6.0 with the next hop address of 192.168.10.1 (northbound interface of DR1215) and of 192.168.10.3 (northbound interface of DR1210). Includes a route for 1.1.7.0 with the next hop address. If the TLR includes stateful services, then the SR's northbound interface will have the same IP address (in active standby mode), which will be the next hop IP address used for the route in the DR of the PLR. ..
The process also adds the identified route to each SR (in the PLR) (at 1620) using the DR's northbound interface (in the PLR) as the next hop IP address. After that, the process ends. In the example, each of RIB1415-1425 contains routes for 1.1.5.0, 1.1.6.0 and 1.1.7.0, all having a next hop IP address of 169.0.0.1 (ie, the address of P1). Therefore, for each of these subnets, packets received from an external network with a destination address within the subnet are transmitted from the receiving SR to the DR and then from the DR to the appropriate TLR.
FIG. 17 conceptually illustrates processing 1700 of some embodiments that generate a route for the RIB of the PLR's routing configuration when a new interface is attached to the PLR, as described above. Specifically, this process refers to the connected route (as opposed to the dynamic route added by process 1600 for TLR connections) and the internal route of other PLRs. This new interface can be a new uplink or downlink (ie a connection to a logical switch or TLR).
As shown, processing begins by receiving a new interface for PLR (in 1705). In some embodiments, the owner of the PLR inputs this as configuration data via the management plane API. In addition, some embodiments will automatically generate a downlink if a new TLR requests to connect to the PLR. The processing is then whether the interface is oriented towards the external network (ie, the interface is directed to the external network) or (towards a logical switch or transit logical switch for TLR connectivity). Determine (in 1710) whether it is a downlink.
If the interface is downlink, the process has the next hop as the DR's north-facing interface (in addition to the route connected for the interface itself to the DR, of course), the route for the subnet to which the interface connects. Is added (at 1715) to each SR of the PLR. Therefore, in the example in Figure 14, each of the SR's RIB1415-1425 contains a route for 1.1.0.0/24 (subnet of logical switch LS1) with the next hop of 169.0.0.1 (DR1225 north-facing interface). ..
In addition, downlink generation may allow the management plane to make static routes previously disabled. In some embodiments, static routes pointing to nonexistent interfaces are stored for logical routers, but are disabled (thus not added as routes to the RIB of various logical router configurations). If the interface is subsequently generated, processing 1700 makes (i) the route to the DR's RIB and (ii) the next hop as the DR's north-facing interface for each of such currently invalidated routes. Add the corresponding route you have to the RIB of each SR (in 1720).
On the other hand, if the new interface is an uplink (facing an external network), process 1700 will generate a new SR (in 1725) for the uplink if necessary, as described above. The process also adds (in 1730) a route to the DR for the subnet to which the new interface connects, which has the next hop as the SR's south-facing interface. The process also adds (in 1735) a route to the DR for the IP address of the new uplink interface, which also has the next hop as the SR's southbound interface. For example, in Figure 14, the DR has routes for interfaces 1.1.3.2 via 169.0.0.2, 1.1.4.2 via 169.0.0.3, and 1.1.4.21 via 169.0.0.4, respectively. In addition, in some embodiments, the DR will have a similar route for the subnet to which each of these interfaces connects.
Finally, for each route with the next hop on the new interface (eg, a static route pointing to an external router), the process adds a route for the new interface (in 1740) to the SR and DR. Update the above route. For example, a default route on a DR can also be generated to point to the south-facing interface of a new SR (which has the same priority as the default route pointing to another SR) and can be accessed through the new interface. Any static route for an external network pointing to an external physical router interface will have a corresponding route added to the SR and SR (eg according to process 1500).
In some situations, the configuration data may change and as a result the management plane may modify the RIB of various logical router configurations. For example, if the IP address of the TLR uplink changes (eg because the pool of available subnets for transit logical switches has been modified), then the management plane is the DR with the next hop in this subnet. The route must be modified. In some embodiments, these are arbitrary routes based on the subnet advertised from the TLR (eg, in the above example, route 1.1.5.0 via 192.168.10.1) and for a newly modified interface. It is a connected route (for example, in the above example, route 192.168.10.0/31 output to D1). Similarly, if the internal transit logical switch for a logical router changes, then the connected routes on the various logical router configurations for that network are changed, and (DR interface for north). And if the South facing SR interface is modified as a result of this change) the internal path between the DR and SR will be modified.
In addition, the administrator may change the IP address of the PLR port. If this modified port is a downlink (connecting to a logical switch, for example, logical switch 1120 in the example), the connected route for that logical switch will be automatically modified. However, some embodiments do not allow configuration modifications if there is any static route in the network with the next hop. Other embodiments invalidate the static route instead, allowing configuration changes.
If the modified PLR port is an uplink, then auto-generated in the RIB (unless the connected route is used for external L2 connectivity and this connectivity does not change). None of the routes made need to be modified. However, BGP peering with the external network may need to be modified because the next hop IP address of the route advertised to the external router must be changed. In addition, static routes configured on the PLR can be sent to a new set of SRs. For example, if the first SR1230 is modified to have address 1.1.4.4 and is connected to the second and third external routers (addresses 1.1.4.1 and 1.1.4.11), the static route 10.0.0.0/8 While via 1.1.4.1 has also been added to RIB1415 for the first SR1230, it can be problematic for static route 172.16.0.0/16 via 1.1.3.1 (this modified configuration is not allowed and also this Static routes are not invalidated).
During operation, the administrator may add or remove uplinks (and corresponding SRs). When the uplink was added, the management plane was connected as described above (eg, adding a default equivalent cost route to the DR and a logical switch connected to the PLR, or the connection selected for route advertisement. Add a route to the DR and the new SR as described in (Adding any route based on the logical switch from the TLR). Moreover, in some embodiments, the static route configured for the PLR needs to be added to the RIB of the new SR (in the same way described above), depending on the L3 connectivity of the uplink. There may be. When the uplink (and SR) is removed, in some embodiments the management plane is first interface-based. Make sure that the routes) (eg, the routes that identify the uplink as the output interface) are not configured for the SR and the PLR affected by the uplink removal. In addition, the equivalent cost default route for sending packets to the SR is removed from the DR, the port on the internal transit logical switch is removed, and as a result any static route that is the route of the DR pointing to the SR is also removed from the DR. Will be deleted.
IV. Electronic system
Many of the features and applications described above can be performed as software processing identified as a set of instructions recorded on a computer-readable recording medium (also referred to as a computer-readable medium). If the instructions of these programs are executed by one or more processing units (eg, one or more processors, processor cores, or other processing units), the instructions of these programs are indicated in the instructions. Cause the action to be executed by (multiple) processing units. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, RAM chips, hard drives, EPROMs, and the like. Computer-readable media does not include carrier waves and electronic signals that pass through wireless or wired connections.
As used herein, the term "software" includes firmware in read-only memory, or an application stored in magnetic recording, which can be read into memory for processing by a processor. Also, in some embodiments, the plurality of software inventions may be implemented as sub-parts of a larger program while maintaining the individual software inventions. In some embodiments, the software inventions may also be implemented as separate programs. Finally, any combination of individual programs that carry out the software inventions described herein together is within the scope of the invention. In some embodiments, when a software program is installed to operate one or more electronic systems, it defines one or more specific mechanical implementations and performs the operation of the software program.
FIG. 18 conceptually illustrates an electronic system 1800 in which some embodiments of the present invention are implemented. Electronic system 1800 can be used to run any of the controls, virtualization, or operating system applications described above. The electronic system 1800 can be a computer (eg, a desktop computer, a personal computer, a tablet computer, a server computer, a mainframe, a blade computer, etc.), a telephone, a PDA, or any other type of electronic device. Such electronic systems include interfaces to media readable by various types of computers, and media readable by various other types of computers. Electronic system 1800 includes bus 1805, processing unit 1810, system memory 1825, read-only memory 1830, persistent storage device 1835, input device 1840, and output device 1845.
Bus 1805 collectively represents all system buses, peripheral buses, and chipset buses that communicatively connect many internal devices of electronic system 1800. For example, bus 1805 communicatively connects the processing unit 1810 to read-only memory 1830, system memory 1825, and persistent storage device 1835.
The processing unit 1810 searches for instructions to be executed and data to be processed from these various memory units in order to execute the processing of the present invention. The processing unit can be a single processor or a multi-core processor in different embodiments.
Read-only memory (ROM) 1830 records static data and instructions required by the processing unit 1810 and other modules of the electronic system. Persistent storage device 1835, on the other hand, is a read / write memory device. This device is a non-volatile memory unit that records instructions and data even when the electronic system 1800 is off. Some embodiments of the invention use mass storage devices (such as magnetic or optical disks and corresponding disk drives) as permanent storage devices 1835.
Other embodiments use removable storage devices (such as floppy disks, flash drives, etc.) as persistent storage devices. Like the persistent storage device 1835, the system memory 1825 is a read / write memory device. However, unlike the storage device 1835, the system memory is a volatile read / write memory such as a random access memory. System memory may store some or all instructions and data that the processor needs at runtime. In some embodiments, the processing of the present invention is recorded in system memory 1825, persistent storage device 1835, and / or read-only memory 1830. The processing unit 1810 retrieves instructions to be executed and data to be processed from these various memory units in order to execute the processing of some embodiments.
Bus 1805 also connects to input device 1840 and output device 1845. Input devices allow users to communicate information and select commands to electronic systems. The input device 1840 includes an alphabetical keyboard and a pointing device (also referred to as a "cursor control device"). Output device 1845 displays an image produced by an electronic system. Output devices include display devices such as printers, vacuum tubes (CRTs) or liquid crystal displays (LCDs). Some embodiments include devices such as touch screens that act as both input and output devices.
Finally, as shown in FIG. 18, bus 1805 also connects electronic system 1800 to network 1865 via a network adapter (not shown). In this way, a computer can be part of a computer's network, such as (local area network (LAN), wide area network (WAN), intranet, network of networks such as the Internet, and so on. Any or all components of the electronic system 1800 may be used in connection with the present invention.
In some embodiments, the microprocessor, computer program instructions are machine-readable or computer-readable medium (or computer-readable storage media, machine-readable medium, or machine-readable). Includes electronic components such as storage and memory stored in (referred to as storage media). Some examples of computer-readable media include RAM, ROM, read-only compact discs (CD-ROMs), recordable compact discs (CD-R), rewritable compact discs (CD-RW), and read-outs. Dedicated digital versatile discs (eg DVD-ROM, dual layer DVD-ROM), various recordable / rewritable DVDs (DVD-RAM, DVD-RW, DVD + RW, etc.), flash memory (eg SD card, Mini SD cards, micro SD cards, etc.), magnetic and / or solid state hard drives, read-only and recordable Blu-Ray® discs, ultra-high density optical discs, any other optical or magnetic media, and floppy disks. including. A computer-readable medium can be executed by at least one processing unit and records a computer program containing a set of instructions for performing various operations. Examples of computer programs or computer code include machine code as generated by a compiler and files containing higher level code executed by a microprocessor using a computer, electronic components, or interpreter.
Although the above description has primarily referred to microprocessors or multi-core processors running software, some embodiments may be one, such as an application specific integrated circuit (ASIC) or field programmable gate array (FPGA). It is executed by the above integrated circuit. In certain embodiments, such integrated circuits execute instructions stored in the circuit itself.
As used herein, the terms "computer," "server," "processor," and "memory" all refer to electronic or other technical devices. These terms do not include people and groups of people. For the purposes of this specification, the terms display and display mean display on an electronic device. As used herein, the terms "computer-readable medium," "computer-readable medium," and "machine-readable medium" as a whole are in computer-readable form. Limited to tangible and physical objects that record information. These terms do not include any wireless signal, wired downloaded signal, and any other ad hoc signal.
This specification refers to computing and network environments, including virtual machines (VMs), from beginning to end. However, a virtual machine is just an example of a data compute node (DCN) or data compute end node, and is also referred to as an addressable node. A DCN may include a non-virtualized physical host, a virtual machine, a container running on the host operating system without the need for a hypervisor or another operating system, and a hypervisor kernel network interface module.
In some embodiments, the VM operates with its own guest operating system on the host, using the resources of the host virtualized by virtualization software (eg, hypervisor, virtual machine monitor, etc.). The tenant (ie, the owner of the VM) can choose which application to run on the guest operating system. On the other hand, some containers are components that run on the host operating system without the need for a hypervisor or another guest operating system. In some embodiments, the host operating system uses namespaces to separate containers from each other, thus providing operating system-level segmentation of different groups of applications running within different containers. This segmentation is similar to the VM segmentation provided in a hypervisor-virtualized environment that virtualizes system hardware, and is therefore viewed as a form of virtualization that personalizes different groups of applications running in different containers. be able to. Such containers are lighter than VMs.
In some embodiments, the hypervisor kernel network interface module is a non-VM DCN that includes a network stack with a hypervisor kernel network interface and receive / send threads. An example of a hypervisor kernel network interface module is the vmknic module, which is part of VMware's ESXi® hypervisor.
Although this specification refers to VMs, given examples may be any type of DCN, including physical hosts, VMs, non-VM containers, and hypervisor kernel network interface modules. In fact, network examples may include different types of DCN combinations in some embodiments.
Although the present invention has been described with reference to a number of specific details, those skilled in the art will recognize that the present invention can be practiced in other particular forms that do not depart from the idea of the invention. In addition, a number of figures (including Figures 9, 10, 13, 15, 16 and 17) conceptually show the process. Certain actions of these processes may not be performed in the exact order shown and described. Certain actions may not be performed in one sequence of actions, and different specific actions may be performed in different embodiments. In addition, the process can be performed using some sub-process or as part of a large macro process. Accordingly, one of ordinary skill in the art will appreciate that the invention is not limited to the details described above, but rather is defined by the appended claims.
18 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17 Sheet 18
Every citation, both ways
| Document | Relation | Office |
|---|---|---|
| US20150063360A1 | Cites | United States of America |
| WO2015015787A1 | Cites | World Intellectual Property Organization (WIPO) |
| KR1020110099579A | Cites | Republic of Korea |
27 members in 7 offices
Members27
| Document | Office | Kind | |
|---|---|---|---|
| CA2994810A1 | Canada | A1 | |
| CA3139646A1 | Canada | A1 | |
| US2017048129A1 | United States of America | A1 | |
| US2017048130A1 | United States of America | A1 | |
| WO2017027073A1 | World Intellectual Property Organization (WIPO) | A1 | |
| AU2016307199A1 | Australia | A1 | |
| CN107925617A | China | A | |
| EP3335389A1 | European Patent Office (EPO) | A1 | |
| JP2018523422A | Japan | A | |
| US10129142B2 | United States of America | B2 | |
| US10230629B2 | United States of America | B2 | |
| US2019199625A1 | United States of America | A1 | |
| AU2016307199B2 | Australia | B2 | |
| JP6720292B2This record | Japan | B2 | |
| AU2020204224A1 | Australia | A1 | |
| JP2020167715A | Japan | A | |
| US10805212B2 | United States of America | B2 | |
| CN107925617B | China | B | |
| US2021029028A1 | United States of America | A1 | |
| CN112437026A | China | A | |
| EP3335389B1 | European Patent Office (EPO) | B1 | |
| CA2994810C | Canada | C | |
| AU2020204224B2 | Australia | B2 | |
| JP7033287B2 | Japan | B2 | |
| US11533256B2 | United States of America | B2 | |
| CN112437026B | China | B | |
| CA3139646C | Canada | C |
17 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Written notification of registration of transferJAPANESE INTERMEDIATE CODE: R350R350 | R350 | |
| Request for change of ownership or part of ownershipJAPANESE INTERMEDIATE CODE: R313111S111 | S111 | |
| Written notification of registration of transferJAPANESE INTERMEDIATE CODE: R350R350 | R350 | |
| Written request for registration of change of domicileJAPANESE INTERMEDIATE CODE: R313531S531 | S531 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD | |
| Request for written amendment filedJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Notification of reasons for refusalJAPANESE INTERMEDIATE CODE: A131A131 | A131 | |
| Report on retrievalJAPANESE INTERMEDIATE CODE: A971007A977 | A977 | |
| Written request for application examinationJAPANESE INTERMEDIATE CODE: A621A621 | A621 | |
| Request for written amendment filedJAPANESE INTERMEDIATE CODE: A523A521 | A521 |
Numbers
- Publication
- 6720292
- Application
- 2018506898
Titles2
- Japanese
- 論理ルータのためのルート設定
- English
- Route configuration for logical routers
Classification
- CPC, 8
- H04L49/354
- H04L45/54
- H04L45/586
- H04L45/64
- H04L49/70
- H04L45/44
- H04L45/74
- H04L45/42
- IPC, 8
- H04L45 02
- H04L45 58
- H04L45 42
- H04L45 586
- H04L45 74
- H04L12 775
- H04L12 70
- H04L12 717
