Protecting networks from cyber attacks and overloading

BACKGROUND [02] The TCP/IP network protocols (e.g., the Transmission Control Protocol (TCP) and the Internet Protocol (IP)) were designed to build large, resilient, reliable, and robust networks. Such protocols, however, were not originally designed with security in mind. Subsequent developments have extended such protocols to provide for secure communication between peers (e.g., Internet Protocol Security (IPsec)), but the networks themselves remain vulnerable to attack (e.g., Distributed Denial of Service (DDoS) attacks).

[03] The largest TCP/IP network, the Internet, has become critical communications infrastructure for many of the world’s countries, such as the United States of America (US). The US government, US military, and critical US commercial interests (e.g., utilities, banks, etc.) have become operationally dependent on the Internet as the communications medium supporting distributed applications such as the telephone system, utilities grids, and e-commerce. For the US and many other countries, it is a matter of national security that the Internet, as well as some of the distributed applications that the Internet supports, hereafter called Internet applications, be available for use by certain organizations during episodes of extreme loading. Extreme loading, or overloading, of the Internet occurs when the volume of network traffic exceeds the effective transmission capacity of the network. Overloading of Internet applications occurs when application servers attached to the Internet (eg., distributed application servers) cannot handle the volume of service requests that are delivered to the servers by the Internet. Either of these overload cases may occur during cyber attacks launched by malicious adversaries or during periods of heavy usage by legitimate users.

CA 2902206 2018-03-13

CA 02902206 2015-00-20

WO 2014/143673 PCT/US2014/027723

104] Often for reasons of national security, some organizations need to have the Internet and certain Internet applications available to them during overload events. This type of availability requirement has been imposed on pre-Intemet telephony systems by some governments. For example, the US Government Emergency Telecommunications Service (GETS) ensures that certain organizations and personnel have emergency access and priority processing for telephone calls on the Public Switched Telephone Network (PSTN). Because of significant differences in protocols, architecture, organization, and operations between the PSTN and the Internet and Internet applications, the technologies, methods, and systems that support GETS cannot be readily ported to the Internet environment.

[05] Accordingly, there is a critical need for technologies, methods, and systems that can meet availability requirements for the Internet and Internet applications during overload episodes.

SUMMARY [06] The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosure. It is neither intended to identify key or critical elements of the disclosure nor to delineate the scope of the disclosure. The following summary merely presents some concepts in a simplified form as a prelude to the detailed description below.

[071 The core Internet is composed of many Autonomous System (AS) networks. An AS is defined in Internet Engineering Task Force (IETF) Request for Comments (RFC) 1930 as a connected group of one or more IP prefixes run by one or more network operators which has a single and clearly defined routing policy. An AS may be owned and operated by a commercial business (e.g., an Internet Service Provider (ISP)). An ISP may provide Internet connectivity to its subscribers, which are often enterprises that operate their own networks (e.g., private networks) to which associated endpoints (e.g., enterprise-affiliated desktop computers, servers, mobile devices, etc.) may be attached. These endpoints may host Internet application instances (e.g., web servers, web clients, voice telephony, instant messaging, social networking, etc.). These endpoints may be identified with Internet addresses that follow the Internet Protocol (IP), i.e., IP addresses. The application instances hosted by a given endpoint may be identified with ports _7_

CA 0290220« 2015-00-20

WO 2014/143673 PCT/US2014/027723 associated with the given endpoint. For example, a web server instance may listen for requests sent to port 80 of the endpoint hosting the web server instance.

[08] An ISP may need to provide its subscribers with connectivity or reachability to other endpoints that may not be attached to the ISP’s subscribers’ networks; instead, the other endpoints may be attached to networks of subscribers to different ISPs. To provide connectivity or reachability, an ISP may connect its AS networks to the AS networks of other ISPs. These points-of-connection are commonly called peering points, and ISPs that are directly connected to each other’s AS networks are commonly called peers. The ISPs may be sufficiently interconnected via peering points such that the Internet allows any endpoint with an Internet IP address to send packets (e.g., via routing) to any other endpoint with an Internet IP address.

[09J The Internet’s open connectivity may be exploited by cyber adversaries to launch attacks (e.g., Denial-of-Service (DoS) attacks) against targets. In a DoS attack, network resources (e.g., routers, links, endpoints, servers, etc.) may be flooded with so many illegitimate service requests that legitimate requests are starved (e.g., the legitimate requests may be effectively denied service). A DoS attack may be carried out by a botnet, a large collection of compromised hosts which are controlled and directed by a central command and control agent to send packets to a target victim. One type of DoS attack, commonly called a “bandwidth” attack, may flood the network routers and links that are immediately upstream of the target with so much malicious traffic that the network cannot service (e.g., forward) many legitimate packets that are being routed to the target. Another type of DoS attack, commonly called an “application-level” DoS attack, may flood an application server (e.g., a web server) with so many illegitimate service requests (e.g., HTTP GET requests for web page downloads) that the application server is unable to service many legitimate requests, effectively denying service to legitimate users.

[10] It is generally believed that a determined adversary, such as a government that is hostile to another country’s government, could launch massive attacks (e.g., DoS attacks) against another country’s Internet infrastructure that are sufficiently large and intense to effectively disable the target country’s Internet and Internet applications. There is much empirical evidence to support this belief. Some of this evidence is gleaned from episodes of heavy usage by legitimate users, such as the Web flood by legitimate users — J —

CA 0290220« 2015-08-20

WO 2014/143673 PCT/US2014/027723 that occurred immediately after the September 11, 2001 terroriste attacks on the US. More evidence is gleaned from the attacks launched against US banks and financial institutions beginning in the Fall of 2012, and from attacks launched by the loosely associated hacktivist group known as “Anonymous.” In both the malicious attack scenario and the legitimate flood scenario (and potentially other overload scenarios), for reasons of national security, the Internet and some Internet applications may need to be available to certain organizations and personnel.

[11] Aspects of this disclosure may relate to ensuring availability of the Internet and some Internet applications to certain organizations and personnel, or users, when the Internet is experiencing overload conditions. Aspects of this disclosure may also relate to restoration of availability of the Internet and some Internet applications to progressively larger sets of users when the Internet is experiencing overload conditions. Said progression may terminate when normal availability is restored to all legitimate users.

[12] Tn some embodiments, packet filtering devices may be located in the Internet at AS network boundary pointe, such as peering pointe and subscriber access points (e.g., Internet access points). The packet filtering devices may apply sets of filtering rules or policies, to packets traversing network links of the peering or subscriber pointe. If a packet matches a filter rule, the packet may be allowed to continue towards its destination or prevented or blocked from continuing towards its destination (e.g., the packet may be dropped), depending on the packet handling action specified by the matching rule. Some packet filtering devices may implement a packet handling action that rate-limits packets that match the associated rule (e.g., the action may both block and allow packets depending on whether or not a rate threshold has been exceeded).

[13] Packet filtering devices may include network firewalls and router access control lists. A packet filtering device may be referred to herein as a Packet Security Gateway (PSG).

[14] Packet security gateways may be associated with one or more policy management servers. Each packet security gateway may receive a policy from a policy management server. A policy management server may instruct the packet security gateway to enforce the policy (e.g., to apply rules specified in the policy to packet traffic passing through the packet security gateway). The packet security gateways may receive multiple policies from policy management servers. These policies may be stored locally by the packet

-4CA 0290220« 2015-08-20

WO 2014/143673 PCT/US2014/027723 security gateways and may not need to be transmitted from policy servers to packet security gateways (e.g., during overload conditions). Additionally or alternatively, the policy servers and packet security gateways may be interconnected by an “out-of-band” management network, which may be physically separate from the Internet infrastructure, and may thus be unaffected by Internet overload conditions.

[15] When an overload condition is detected, some policy management servers may direct some packet security gateways to enforce a first set of policies. Policies in this first set may contain rules that block all packets except for packets associated with protocols and applications that are necessary for the Internet and critical Internet applications to operate. These protocols and applications may include, for example, Border Gateway Protocol (BGP), the Domain Name System (DNS), and the Network Time Protocol (NTP). When this first set of policies is being enforced, the packet traffic that caused the overload condition may be blocked from ingressing the Internet at Internet access points, or may be blocked at peering points. Additionally or alternatively, the packet traffic that caused the overload condition may be rate-limited when ingressing the Internet at Internet access points, or may be rate-limited at peering points. While this first set of policies is being enforced, ISPs and other network operators may take actions to eliminate or mitigate the sources of packet traffic that caused the overload condition.

[16] In some embodiments, the policy management servers may direct the packet security gateways to enforce a second set of policies. Policies in this second set may contain rules from the first set of policies, and may also contain one or more additional rules which may allow packets between some Internet applications being used by some critical users or systems. For example, in a national emergency situation, first responders associated with local, state, and federal government organizations may be allowed to use the Internet for telephone calls, text messages, e-mail, web-based services, etc. While this second set of policies is being enforced, ISPs and other network operators may continue to take actions to eliminate or mitigate the sources of packet traffic that caused the overload condition.

[17] In some embodiments, the policy management servers may direct the packet security gateways to enforce a third set of policies. Policies in this third set may contain rules from the first set of policies and rules from the second set of policies, and may also contain one or more additional rules which may allow packets between one or more

-5CA 0290220« 2015-00-20

WO 2014/143673 PCT/US2014/027723 additional critical organizations, personnel, and applications. While this third set of policies is being enforced, ISPs and other network operators may continue to take actions to eliminate or mitigate the sources of packet traffic that caused the overload condition.

[18] In some embodiments, a cycle of enforcing sets of policies with progressively broader scopes of users and applications may be repeated until normal operation is restored (e.g., until legitimate users have the Internet and Internet applications available to them as they did before the overload conditions occurred).

[19] Other details and features will be described in the sections that follow.

BRIEF DESCRIPTION OF THE DRAWINGS [20] The present disclosure is pointed out with particularity in the appended claims. Features of the disclosure will become more apparent upon a review of this disclosure in its entirety, including the drawing figures provided herewith.

[21] Some features herein are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings, in which like reference numerals refer to similar elements.

[22] FIG. 1 illustrates an exemplary network environment in which one or more aspects of the disclosure may be implemented.

[23] FIG. 2 illustrates an exemplary network environment with packet security gateways located at AS network boundaries such as peering points and subscriber Internet access points.

[24] FIG. 3 illustrates an exemplary packet filtering policy which may be enforced by a packet security gateway located at a peering point.

[25] FIG. 4 illustrates an exemplary packet filtering policy which may be enforced by a packet security gateway located at an Internet access point.

[26J FIG. 5 illustrates an exemplary packet filtering policy which may be enforced by a packet security gateway, and which may allow certain users or certain Internet applications to communicate.

-6CA 0290220« 2015-00-20

WO 2014/143673 PCT/US2014/027723 [27] FIG. 6 illustrates an exemplary network environment with packet security gateways located at AS network boundaries, such as peering points and subscriber Internet access points, of an individual ISP that provides protections to its subscribers.

[28] FIG. 7 illustrates an exemplary method for protecting a network from overload conditions while allowing certain users and Internet applications to communicate across the network.

[29] FIG. 8 illustrates an exemplary packet security gateway.

DETAILED DESCRIPTION [30] In the following description of various illustrative embodiments, reference is made to the accompanying drawings, which form a part hereof, and in which is shown, by way of illustration, various embodiments in which aspects of the disclosure may be practiced. It is to be understood that other embodiments may be utilized, and structural and functional modifications may be made, without departing from the scope of the present disclosure.

[311 Various connections between elements are discussed in the following description. These connections are general and, unless specified otherwise, may be direct or indirect, wired or wireless. In this respect, the specification is not intended to be limiting.

[32] FIG. 1 illustrates an exemplary network environment in which one or more aspects of the disclosure may be implemented. Referring to FIG. 1, network environment 100 may be a TCP/IP network environment (e.g., the Internet).

[33] Network environment 100 may include autonomous system (AS) networks 101, 102, 103,104,105,106,107, and 108. AS networks 101-108 may be owned or operated by various ISPs. AS networks 101-108 may function as transit networks (e.g., they may not have Internet-addressable endpoints attached to them and may therefore not terminate any packet microflows generated by Internet applications). For example, packets that ingresses one or more of AS networks 101-108 may also egresses the AS network. Interconnections between any two AS networks 101-108 may be peering points (e.g., a link between AS network 101 and AS network 107 may be a peering point).

-7CA 0290220« 2015-00-20

WO 2014/143673

PCT/US2014/027723

134] Networks 110, 111, 112, 113, 114, and 115 may be owned or operated by various enterprises. One or more of networks 110-115 may or may not be an autonomous system network. One or more of networks 110-115 may not be a transit network and may be a private (non-public) network, and may therefore not be providing Internet service (e.g., an organization owning or operating one or more of networks 110-115 may not be an ISP). One or more endpoints (not shown in FIG. 1), such as desktop computers, servers, telephones, etc., may be affiliated with these enterprises and may be attached to one or more of networks 110-115. Such endpoints may host instances of various Internet applications, such as web servers and web clients (e.g., web browsers), text messaging servers and clients, IP telephony systems, etc. An owner or operator of one or more of networks 110-115 may want to allow endpoints attached to their network to be able to communicate with other endpoints attached to another of networks 1 ΙΟΙ 15. For example, an owner or operator of network 110 may want to allow an endpoint attached to network 110 to communicate with an endpoint attached to network 115, which may be owned or operated by a different organization than the organization that owns or operates network 110. To achieve such inter-network communications between networks 110-115, the owners or operators of networks 110-115 may subscribe to one or more ISPs for Internet service. An ISP may connect one or more of its networks to a subscriber’s network. For example, an ISP that owns or operates AS network 103 may connect network 103 with network 112, which may be owned or operated by an organization that has subscribed to the ISP. Connections between subscriber networks and ISP networks, such as the connection between network 112 and network 103, may be Internet access points.

[35] ISPs may install routers that support the Border Gateway Control (BGP) protocol, called BGP routers, at the boundaries of their AS networks. A BGP router may know which IP addresses can be reached from its interfaces. Using the BGP protocol, a BGP router may advertise its reachability information to one or more BGP routers located at the border of different AS networks. For example, a BGP router may advertise to other BGP routers that are located at the boundaries of peer AS networks. A given BGP router may not communicate with every other BGP router in the Internet. A BGP router may utilize reachability information received from other BGP routers to compute a local routing table. A router’s routing table may contain entries that associate an IP address with one of the router’s network interfaces. When a router receives a packet, it may look up the

-8CA 0290220« 2015-00-20

WO 2014/143673 PCT/US2014/027723 packet’s destination IP address in the routing table, and then forward the packet out the network interface specified in the routing table entry. The network interface may itself be connected to the network interface (e.g., an inbound network interface) of another router, which may repeat the lookup-and forward process. Eventually, the packet may reach its destination endpoint.

[36] Utilization of the BGP protocol may be critical for enabling a network’s packet routing service. In one or more implementations of a BGP router, the BGP protocol may also be used to determine if peer BGP routers are functioning, for example, via the use of KEEPALIVE messages. If a BGP router does not receive a KEEPALIVE response from a peer BGP router (e.g., after a configured timeout period), then the BGP router may determine that the peer BGP router is no longer functioning, and may stop forwarding packets to the peer BGP router. Accordingly, for a network such as the Internet to provide its packet routing service, BGP protocol communications between peer BGP routers may need to be maintained.

1371 Internet applications may represent machine-readable IP addresses of endpoints (e.g., 173.194.75.103) using human-readable domain names (e.g., www.google.com). When an Internet application instance sends packets over the Internet to an endpoint, the packets may be required to contain the IP address of the endpoint in the destination IP address field of the packets’ IP headers. An Internet application may know the domain name of a destination endpoint but may not know its IP address. An Internet application instance may issue a request to a Domain Name System (DNS) to resolve the domain name into an IP address, and the DNS may respond to the request with an IP address that corresponds to the domain name. The DNS may be a collection of servers distributed across the Internet that resolve domain names into IP addresses. The DNS and endpoints using the DNS may use the DNS protocol to inter-communicate. Although the Internet may not require the DNS to provide its packet routing service, and although in theory Internet applications may not need the DNS to intercommunicate, in practice the DNS may be critical to the function and operation of many Internet applications. Thus, for Internet applications to function, DNS protocol communications between the DNS and Internet applications may need to be maintained.

[38] The Network Time Protocol (NTP) is a protocol for clock synchronization between computer systems attached to a TCP/IP network (e.g., the Internet). NTP may be

-9CA 0290220« 2015-00-20

WO 2014/143673 PCT/US2014/027723 architecturally similar to DNS in that there may be a hierarchical collection of clocks and associated time servers distributed across the Internet that computer systems may access. Internet applications may depend on synchronized time in order to function correctly; thus NTP protocol communications between time servers and Internet applications may need to be maintained.

[39] There may be other systems and protocols associated with a network that may need to be functional or effectively communicating in order for the network or one or more critical network applications to function correctly.

[40] Overload conditions may occur in a network (e.g., the Internet) when any of several scenarios occur. One scenario may be when many legitimate users, who may be distributed widely across the network, request services (e.g., web page downloads) from the same resource (e.g., a web application server) or from a set of resources that arc attached to the same subnet. For example, many legitimate users executing Internet application clients (e.g., web browsers) hosted by endpoints attached to networks 11 ΟΙ 14 may request service from an Internet application seiver (e.g., a web application server) attached to network 115, during the same small time window. As the packets containing the requests traverse the Internet and converge on network 115 or the destination Internet application server, the volume of aggregate packet traffic may exceed the capacity of one or more network elements (e.g., routers, switches, network links, gateways, etc.) that are located close to, or immediately upstream from, the Internet application server. Finite packet queues contained in the various network elements may overflow, causing packets to be dropped. Accordingly one or more requests contained in the dropped packets may not be serviced by the Internet application server (e.g., the requesting users and applications may be denied service because of the overload condition).

[41] It may also be the case that even if the incoming requests do not cause an overload condition, the volume of packets containing responses to the requests may cause an overload condition, for example, in the network elements located immediately downstream from the Internet application server. For example, this scenario may occur when the Internet application is asymmetric (e.g., when the average size, measured in bytes, of responses exceeds the average size of requests). Even though all of the requests may have been properly serviced by the Internet application server, some of the packets

-10CA 0290220« 2015-00-20

WO 2014/143673 PCT/US2014/027723 containing responses may have been dropped; thus, from the perspective of the service requestors, service may be denied because they may never receive responses to their requests.

[42] In another scenario, the volume of requests may not cause an overload condition to occur in the network elements immediately upstream from the Internet application server; however, the Internet application server may not have the processing capacity to service all of the requests. For example, if the instantaneous rate of incoming requests exceeds the service rate of an Internet application server, the requests may be queued. If the state-of-excess is sustained for a sufficient duration of time, then the request queue may overflow, causing some requests to be dropped, thereby denying service to the users who issued the dropped requests.

[43] Overload conditions may also be caused by one or more malicious agents. An overload condition that is caused by malicious agents may be a DoS attack. In a DoS attack, a logical network, or botnet, of malicious agents, or bots, may generate attack packet traffic when a so-called command-and-contiol agent directs the bots to launch an attack. Botnets may be created when an adversaiy is able to infect many endpoints distributed across the Internet with malware that implements the hot. Botnets may be composed of hundreds, thousands, or even millions of bots that have been identified on the Internet.

[44] The network architecture of a DoS attack may be similar to the network architecture of an overload condition caused by legitimate users. For example, a botnet’s bots may be hosted by one or more endpoints attached to networks 110-114. Upon direction from the botnet’s command-and-control agent, the bots may send many service requests to an Internet application server attached to network 115. These malicious service requests or their associated responses may exceed the capacity of the network elements immediately upstream or downstream from the Internet application server, or the malicious service requests may exceed the capacity of the Internet application server. Accordingly, some legitimate users may be denied service.

[45] Regardless of the cause of an overload condition, some users may require the Internet or one or more Internet applications be available for their use during the overload condition (e.g., that the services provided by the Internet or Internet application(s) not be denied to them). One approach to meeting this requirement may be to prevent packets from non

-11CA 0290220« 2015-00-20

WO 2014/143673 PCT/US2014/027723 required users, agents, endpoints, and Internet applications from traversing the Internet and reaching their respective destinations, while simultaneously allowing packets from required users, agents, endpoints, and Internet applications to traverse the Internet and reach their respective destinations. In one embodiment such an approach may utilize one or more packet security gateways to discriminate between packets that should be allowed and packets that should be blocked.

[46] FIG. 2 illustrates an exemplary network environment with packet security gateways located at AS network boundaries such as peering points and subscriber Internet access points. Referring to FIG. 2, packet security gateways (PSGs) 200-220 may have been deployed in network environment 100 for the purpose of filtering required and nonrequired packets in such a way that during overload conditions, services may not be denied to certain users, agents, endpoints, or Internet applications. The packet security gateways me be located at the boundary points of AS networks 101-108 and subscriber networks 110-115 (e.g., at peering points and Internet access points). During an overload condition, one or more of packet security gateways 200-220 may enforce one or more policies (e.g., collections of packet filtering rules), which may determine which packet traffic is blocked and which packet traffic is allowed. The policies enforced by the packet security gateways may be changed over time in order to change the determination of which packet traffic is blocked and which packet traffic is allowed. For example, near the beginning of an overload condition, the scope of packet traffic being blocked or allowed, may be broad or narrow, respectively, in order to ensure that much of the traffic causing the overload condition is blocked, or to ensure that required communications are allowed and fully supported by the Internet or one or more associated Internet applications. Over time, as the sources of traffic causing overload conditions arc identified and mitigated, or possibly decontaminated from malware applications such as bots, the policies may be changed to narrow' the scope of packet traffic being blocked, or to broaden the scope of packet traffic being allowed.

[47] When an overload condition is detected, a first set of policies may be enforced by packet security gateways 200-220 to mitigate the overload condition and ensure that some users, endpoints, or Internet applications are able to inter-communicate via network environment 100. Regardless of which users’, endpoints’, or Internet applications’ Internet communications are supported by this first set of policies, there may be critical

-12CA 0290220« 2015-00-20

WO 2014/143673 PCT/US2014/027723 communications between network elements and systems that may need to be supported in order for the Internet or Internet applications to function properly. These critical communications may be allowed in the first set of policies and in all subsequent sets of policies. For example, these communications may include one or more of: BGP communications between peer BGP routers located at boundary points of ISP-operated AS networks and some subscriber networks; DNS protocol communications between Internet applications and DNS servers distributed across the Internet; and NTP communications between Internet elements, applications, or time servers distributed across the Internet. Additionally or alternatively, there may be other protocols that arc considered critical; accordingly, a first set of policies may also support communications for these other protocols.

[48] FIG. 3 illustrates an exemplary packet filtering policy which may be enforced by a packet security gateway located at a peering point. Referring to FIG. 3, policy 300 may contain one or more filtering role representations. For example, packet security gateways may filter on five (5) fields in an IP packet: source and destination IP address fields, source and destination port fields (e.g., those contained in the encapsulated transport protocol packet, if any), and protocol (for IP version 4, as shown) or next header (for IP version 6, not shown). The five fields may be referred to as a “5-tuple”. 5-tuple filtering rules may specify values for any number of the five fields (e.g., a filtering role may only filter packets on a single field such as source IP address, or a filtering rule may filter on any combination of two, three, or four fields, or all five fields). Each rule may be associated with a packet handling action, which may be, for example, BLOCK (e.g., drop the packet) or ALLOW (e.g., forward the packet towards its destination).

[49] The roles in policy 300 may allow certain BGP protocol communications, certain DNS protocol communications, and certain NTP protocol communications. Policy 300 may, for example, be enforced by a packet security gateway located at a peering point between two transit networks. For example, packet security gateway 220 may be located at a peering point between AS network 107 and AS network 108. A BGP router (not illustrated) may be located at each end of a network link connecting AS network 107 and AS network 108. An owner or operator of AS network 107 may assign IP version 4 address 123.56.89.0 to a network interface on the BGP router at the boundary of AS

-13CA 0290220« 2015-00-20

WO 2014/143673 PCT/US2014/027723 network 107, and an owner or operator of AS network 108 may assign IP version 4 address 87.65.21.0 to a network interface on the BGP router at the boundary of AS network 108. A network link may connect interface 123.56.89.0 to network interface 87.65.21.0. This network link may pass through packet security gateway 220, but as the network interfaces of packet security gateway 220 may not have IP addresses assigned to them, at the IP level, packet security gateway 220 may be transparent to the BGP routers.

[50] Rule 1 301 of policy 300 may allow BGP packets sent by a BGP client from the network interface 123.56.89.0 and from any source port (as denoted by the wildcard symbol) to network interface 87.65.21.0 and port 179, (e.g., a port associated with a BGP listener or BGP server). Rule 2 302 may allow BGP packets to be sent by a BGP client from the network interface 87.65.21.0 and from any source port to network interface 123.56.89.0 and port 179. Rule 3 303 and rule 4 304 may respectively allow packets containing responses to any requests or messages contained in packets allowed by rule 2 302 or rule 1 301 to be sent back to their requestors. BGP may use TCP as its transport protocol; accordingly, the protocol field value in rules 1-4 301-304 may be set to TCP.

[51] Rule 5 305 and rule 6 306 may allow DNS protocol packets to pass through packet security gateway 220. Rules 5 305 and 6 306 may not include restrictions on the source IP addresses and destination IP addresses. For example, because DNS clients and DNS servers may be located in subscriber networks connected to the edge of network environment 100 (e.g., networks 110-115) packet filtering rules applied by a packet security gateway located at a peering point between two transit networks (e.g., packet security gateway 220 located between transit networks 107 and 108) may not have restrictions on the source and destination IP addresses of DNS protocol packets (e.g., because potentially any pair of DNS clients and servers could be communicating through the peering point). Rule 5 305 may allow packets that contain any DNS client’s request and that arc destined for any DNS server, which may be listening for requests on one or more ports (e.g., on port 53). Rule 6 306 may allow packets that contain DNS server responses to any requests contained in the packets allowed by rule 5 305. The DNS protocol may be transported using either TCP or the User Datagram Protocol (UDP); accordingly, the Protocol field in rule 5 305 and rule 6 306 may allow any value.

[52] Rule 7 307 and rule 8 308 may allow NTP protocol packets to pass through packet security gateway 220. Similar to DNS, NTP clients and NTP servers may be located in

-14CA 0290220« 2015-08-20

WO 2014/143673 PCT/US2014/027723 subscriber networks connected to the edge of network environment 100 (e.g., networks 110-115); thus, packet filtering rules applied by a packet security gateway located at a peering point between two transit networks (e.g., packet security gateway 220 located between transit networks 107 and 108) may not have restrictions on the source and destination IP addresses of NTP protocol packets because potentially any pair of NTP clients and servers could be communicating through the peering point. Rule 7 307 may allow packets that contain any NTP client’s request and that are destined for any NTP server, which may be listening for requests on one or more ports (e.g., 123). Rule 8 308 may allow packets that contain NTP server responses to any requests contained in the packets allowed by rule 7 307. NTP may use UDP as its transport protocol; accordingly, the Protocol field in rule 7 307 and rule 8 308 may be set to UDP.

[53] Rule 9 309 may block any packet that does not match any of rules 1-8 301-308. For example, packet security gateway 220 may apply rules to packets in the order in which they appear in the policy that contains them. Accordingly, rule 9 309 may blocks packets that that do not match any of rules 1-8 301-308 (e.g., one or more packets associated with the creation of an overload condition).

[54] Policy 300 may be enforced by one or more packet security gateways at any peering point or Internet access point in network environment 100. In some embodiments, more restrictive rules may be contained in policies enforced by packet security gateways located near the edge of network environment 100 (e.g., at Internet access points). For example, to mitigate or even eliminate overload conditions at locations near the edge. In one type of DoS attack, known as an open DNS resolver attack, a botnet may cause many DNS servers to send packets to a target resource (e.g., a subscriber network’s Internet access points or a company’s public e-commerce web server) located at or near the edge of the Internet. Rule 5 305 and rule 6 306 of policy 300 may not block such packets. At an Internet access point, however, the IP addresses of the DNS clients and servers that are expected to be communicating across the Internet access point may be known to the operators of either the subscriber network or the ISP network connected by the Internet access point. Packet filtering rules that filter DNS protocol packets and that specify specific IP addresses of DNS endpoints in their source and destination IP address fields, may be enforced by packet security gateways located at Internet access points and

-15CA 0290220« 2015-08-20

WO 2014/143673 PCT/US2014/027723 may block most or all of the packets generated by an open DNS resolver attack, thereby mitigating or eliminating any overload conditions caused by such an attack.

[55] FIG. 4 illustrates an exemplary packet filtering policy which may be enforced by a packet security gateway located at an Internet access point. Referencing FIG. 4, rules 10 401 and 11 402 may be contained in policy 400. Policy 400 may be enforced by packet security gateway 200, which may be located at an Internet access point between subscriber network 110 and AS network 102. Subscriber network 110 may have been allocated IP version 4 addresses with subnet prefix 32.10,87.0/24. DNS clients attached to network 110 may have all of their DNS requests routed to a DNS server with IP address 13.57.92.46, which may be external to network 110, and which may be considered to be trusted by the operators of network 110. Rule 10 401 may allow packets containing requests from DNS clients attached to network 110 and destined for port 53 on DNS server 13.57.92.46. Rule 11 402 may allow packets containing responses from DNS server 13.57.92.46 and destined for one or more DNS clients attached to network 110. Rule 12 403 may block any DNS server packets destined for network 110, as such packets may be part of an open DNS resolver attack, or may otherwise be packets from a DNS server that were not requested by a DNS client attached to network 110. In some embodiments, rule 12 403 may not be included in policy 400. For example, the last rule in the policy 400 may be a block rule like rule 9 309 in policy 300.

[56] An overload condition may be highly mitigated or even eliminated by having packet security gateways 200-220 in network environment 100 enforce a first set of policies which is composed of policies similar to policy 300 and policy 400. This first set of policies may, however, also prevent one or more legitimate users or their Internet applications from communicating across network environment 100. For example, overload conditions may occur when there is a large DoS attack or many DoS attacks. Overload conditions may also occur when there is a widespread emergency condition that causes many legitimate users to attempt to access the same resources (e.g., a telephony system or news web site). While this first set of policies is being enforced, network operators may take actions to mitigate or eliminate the sources of packets that caused the original overload conditions. For example, network operators may prevent endpoints suspected of hosting bots from accessing the Internet or network operators

-16CA 0290220« 2015-00-20

WO 2014/143673 PCT/US2014/027723 may severely rate-limit some types of traffic that are believed to be causing the overload conditions.

[57] It may be desirable or may be required by local laws or regulations that some users (e.g., first responders) be guaranteed services from the Internet or from certain Internet applications, despite the overload conditions. To provide such guarantees, a second set of policies may be enforced by one or more of packet security gateways 200-220 in network environment 100. These policies may contain all of the rules contained in the first set of policies and one or more additional rules that allow certain users (e.g., first responders) or certain Internet applications to communicate over network environment 100.

[58] For example, all users with endpoints attached to network 110 and all users with endpoints attached to network 112 may be allowed to communicate, using the HTTP protocol, with web application servers attached to network 113. Network 110 may have been allocated IP version 4 addresses with subnet prefix 10.10.87.0/24. Network 112 may have been allocated IP addresses with subnet prefix 12.12.87.0/24, and network 113 may have been allocated IP addresses with subnet prefix 13.13.87.0/24.

[59] FIG. 5 illustrates an exemplary packet filtering policy which may be enforced by a packet security gateway, and which may allow certain users or certain Internet applications to communicate. Referring to FIG. 5, policy 500 may include one or more of the rules from policy 300 or policy 400. Policy 500 may also contain rules 13-16 501504. Rule 13 501 may allow packets sourced from HTTP clients (e.g., web browsers) attached to network 110 and destined for one or more HTTP servers (e.g., one or more web application servers on port 80) attached to network 113. Rule 14 502 may allow packets sourced by the HTTP servers attached to network 113 and destined for endpoints attached to network 110. Such packets may, for example, contain responses to HTTP requests issued by HTTP clients attached to network 110. Rule 15 503 and rule 16 504 may be similar to rule 13 501 and rule 14 502 except they may allow packets containing HTTP client requests and HTTP server responses between networks 112 and 113.

160] An overload condition may be highly mitigated or even eliminated, and certain users or certain Internet applications may be allowed to communicate over network environment 100, by having packet security gateways 200-220 in network environment 100 enforce a

-17CA 0290220« 2015-08-20

WO 2014/143673 PCT/US2014/027723 second set of policies which is composed of policies similar to policy 500. While this second set of policies is being enforced, network operators may take actions to mitigate or eliminate the sources of packets that caused the original overload conditions.

[61] Later, a third set of policies may be enforced by packet security gateways 200-220 in network environment 100 which may contain all of the rules contained in the second set of policies (which may themselves have contained all of the rules contained in the first set of policies) and may also contain one or more additional rules that allow more users and/or more Internet applications to communicate over network environment 100. While the third set of policies is being enforced, network operators may take further actions to mitigate or eliminate sources of packets that caused the overload conditions. Later, a fourth set of policies may be enforced that incorporates the third set of policies and broadens the scope of user and/or Internet applications that may communicate over network environment 100. Such a cycle may be repeated until the normal operation of one or more of network environment 100, its users, or its Internet applications, is restored, or the sources of traffic which caused the original overload conditions are sufficiently mitigated or eliminated such that users and Internet applications are not denied service because of overload conditions.

[62] In some embodiments, packet security gateways may be required to be located at all peering points or Internet access points in network environment 100. In other embodiments, this practice may be relaxed while still providing protection from overload conditions and while still providing some users and Internet applications with communications services. For example, an individual ISP may be able to offer protection from overload conditions and still support selected conununications for its subscribers.

[63] FIG. 6 illustrates an exemplary network environment with packet security gateways located at AS network boundaries, such as peering points and subscriber Internet access points, of an individual ISP that provides protections to its subscribers. Referring to FIG. 6, an ISP (e.g., SecurelSP) may own or operate AS networks 102,103, and 106 in network environment 100. SecurelSP may have located packet security gateways (e.g., packet security gateways 200-207, 210,213, 214, and 215) at all the peering points and Internet access points of its networks. One or more other ISPs that own or operate AS

-18CA 0290220« 2015-00-20

WO 2014/143673 PCT/US2014/027723 networks 101,104,105,107, and 108 may not have installed packet security gateways at peering points and Internet access points of their networks.

[64] An overload condition may occur in network 113, which may be owned or operated by a subscriber to SecurelSP. By enforcing one or more policies similar to policy 300 at its peering points and by enforcing policies similar to policy 400 at its Internet access points, SecurelSP may eliminate or highly mitigate the overload condition in network 113. For example, regardless of the source of the packet traffic that caused the overload condition (e.g., any combination of endpoints attached to networks 110, 111, 112, 114, and 115), the traffic may be filtered by a policy included in the first set of policies because the traffic may be required to attempt to pass through one of the packet security gateways operated by SecurelSP while being routed towards network 113. While the first set of policies is being enforced, SecurelSP may take actions to mitigate or eliminate one or more sources of the traffic causing the overload condition. For example, SecurelSP may take actions to mitigate or eliminate one or more sources of traffic that are attached to its subscribers’ networks.

[65] Later, after enforcing the first set of policies, SecurelSP may want to allow all users with endpoints attached to its subscriber’s network 110 and all users with endpoints attached to its subscriber’s network 112 to communicate, using the HTTP protocol, with web application servers attached to its subscriber’s network 113. Network 110 may have been allocated IP version 4 addresses with subnet prefix 10.10.87.0/24. Network 112 may have been allocated IP addresses with subnet prefix 12.12.87.0/24. Network 113 may have been allocated IP addresses with subnet prefix 13.13.87.0/24. By enforcing a second set of policies similar to policy 500 at its peering points and its Internet access points, SecurelSP may eliminate or highly mitigate the overload condition in network 113 while allowing HTTP clients (e.g., web browsers) attached to its subscribers’ networks 110 and 112 to communicate with HTTP servers (e.g., web application servers) attached to its subscriber’s network 113.

[66] Depending on the routing polices being used in network environment 100, packet traffic generated by HTTP clients and HTTP servers attached to networks 110,112, and 113 may be required to traverse one or more of AS networks 101, 104, 105, 107, and 108, which may not have packet security gateways located at their peering points and Internet access points. Packet traffic generated by HTTP clients and HTTP servers attached to

-19CA 0290220« 2015-00-20

WO 2014/143673 PCT/US2014/027723 networks 110,112, and 113 may traverse AS networks which may also be transporting traffic that may be causing overload conditions at various subscriber networks 110-115. Given the architecture, operation, and behavior of network environment 100, it may be unlikely that any one or more of AS networks 101, 104, 105, 107, and 108 are themselves experiencing overload conditions that may disrupt communications between HTTP clients and HTTP servers attached to networks 110,112, and 113. Accordingly, SecureISP may be able to offer effective protections from overload conditions to its subscribers, even though other ISPs may not offer similar protections and may transport some or most of the traffic that may be causing overload conditions in SccurcISP’s subcribcrs’ networks.

[67] FIG. 7 illustrates an exemplary method for protecting a network from overload conditions while allowing certain users and Internet applications to communicate across the network. Referring to FIG. 7, at step 702, packets may be received. For example, packet security gateway 200 may receive packets from network 110. At step 704, responsive to a determination that an overload condition has occurred, a first group of packet filtering rules may be applied to at least some of the packets. For example, an overload condition may occur in network 113, and responsive to a determination that the overload condition in network 113 has occurred, packet security gateway 200 may apply one or more of rules1-9 301-309 of policy 300 to at least some of the packets received from network 110. At step 706, responsive to a determination that the overload condition has been mitigated, a second group of packet filtering rules may be applied to at least some of the packets. For example, responsive to a determination that the overload condition in network 113 has been mitigated, packet security gateway 200 may apply one of more of rules 13-16 501-504 to at least some of the packets received from network 110.

[68] FIG. 8 illustrates an exemplary packet security gateway. Referring to FIG. 8, as indicated above, packet security gateway 220 may be located between AS networks 107 and 108. For example, packet security gateway 220 may be located at network boundary 802. Packet security gateway 220 may include one or more processors 804, memory 806, network interfaces 808 and 810, packet filter 812, and management interface 814. Processor(s) 804, memory 806, network interfaces 808 and 810, packet filter 812, and management interface 814 may be interconnected via data bus 816. Network interface

-20CA 0290220« 2015-00-20

WO 2014/143673 PCT/US2014/027723

808 may connect packet security gateway 220 to AS network 107. Similarly, network interface 810 may connect packet security gateway 220 to AS network 108. Memory 806 may include one or more program modules that when executed by processors) 804, may configure packet security gateway 220 to perform one or more of various functions described herein.

[69] Packet security gateway 220 may be configured to receive a policy (e.g, one or more of policies 300, 400, or 500) from one or more security policy management servers (not illustrated). For example, packet security gateway 220 may receive policy 818 from a security policy management server via management interface 814 (e.g., via out-of-band signaling) or network interface 808 (e.g., via in-band signaling). Packet security gateway 220 may include one or more packet filters or packet discriminators, or logic for implementing one or more packet filters or packet discriminators. For example, packet security gateway 220 may include packet filter 812, which may be configured to examine information associated with packets received by packet security gateway 220 and forward such packets to one or more of operators 820, 822, or 824 based on the examined information. For example, packet filter 812 may examine information associated with packets received by packet security gateway 220 (e.g., packets received from AS network 107 via network interface 808) and forward the packets to one or more of operators 820,822, or 824 based on the examined information.

[70] Policy 818 may include one or more rules and the configuration of packet filter 812 may be based on one or more of the rules included in policy 818. For example, policy 818 may include one or more rules specifying that packets having specified information should be forwarded to operator 820, that packets having different specified information should be forwarded to operator 822, and that all other packets should be forwarded to operator 824. Operators 820, 822, and 824 may be configured to perform one or more functions on packets they receive from packet filter 812. For example, one or more of operators 820, 822, or 824 may be configured to forward packets received from packet filter 812 into AS network 108, forward packets received from packet filter 812 to an IPsec stack (not illustrated) having an IPsec security association corresponding to the packets, or drop packets received from packet filter 812. In some embodiments, one or more of operators 820, 822, or 824 may be configured to drop packets by sending the

-21CA 0290220« 2015-00-20

WO 2014/143673 PCT/US2014/027723 packets to a local “infinite sink” (e.g., the /dev/null device file in a UNIX/LINUX system).

[71] The functions and steps described herein may be embodied in computer-usable data or computer-executable instructions, such as in one or more program modules, executed by one or more computers or other devices to perform one or more functions described herein. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types when executed by one or more processors in a computer or other data processing device. The computer-executable instructions may be stored on a computer-readable medium such as a hard disk, optical disk, removable storage media, solid-state memory, RAM, etc. As will be appreciated, the functionality of the program modules may be combined or distributed as desired in various embodiments. In addition, the functionality may be embodied in whole or in part in firmware or hardware equivalents, such as integrated circuits, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGA), and the like. Particular data structures may be used to more effectively implement one or more aspects of the disclosure, and such data structures are contemplated to be within the scope of computer executable instructions and computer-usable data described herein.

[72] Although not required, one of ordinary skill in the art will appreciate that various aspects described herein may be embodied as a method, an apparatus, or as one or more computer-readable media storing computer-executable instructions. Accordingly, those aspects may take the form of an entirely hardware embodiment, an entirely software embodiment, an entirely firmware embodiment, or an embodiment combining software, hardware, and firmware aspects in any combination.

[73] As described herein, the various methods and acts may be operative across one or more computing servers and one or more networks. The functionality may be distributed in any manner, or may be located in a single computing device (e.g., a server, a client computer, etc.).

[74] Aspects of the disclosure have been described in terms of illustrative embodiments thereof. Numerous other embodiments, modifications, and variations within the scope and spirit of the appended claims will occur to persons of ordinary skill in the art from a

-22CA 0290220« 2015-00-20

WO 2014/143673

PCT/US2014/027723 review of this disclosure. For example, one of ordinary skill in the art will appreciate that the steps illustrated in the illustrative figures may be performed in other than the recited order, and that one or more steps illustrated may be optional.