Software defined networking for edge nodes

TECHNICAL FIELD

0001The exemplary and non-limiting embodiments of this invention relate generally to wireless communication systems, methods, devices and computer programs and, more specifically, relate to software defined networking systems and methods, cloud computing and virtualization techniques, data centers, communications network sharing, gateways used in wireless communications system, in particular those gateways having dual control plane and user plane functionality such as edge nodes.

BACKGROUND

0002This section is intended to provide a background or context to the invention that is recited in the claims. The description herein may include concepts that could be pursued, but are not necessarily ones that have been previously conceived, implemented or described. Therefore, unless otherwise indicated herein, what is described in this section is not prior art to the description and claims in this application and is not admitted to be prior art by inclusion in this section.

0003Currently two major trends are becoming dominant in communication service provider (CSP) networks: network virtualization (NV) and software defined networking (SDN). However, at present there is no adequate technique to apply these emerging technologies to edge nodes, in particular those nodes that have dual control plane and user plane functionality and that are configurable to interface a mobile core network to other systems and networks.

SUMMARY

0004The foregoing and other problems are overcome, and other advantages are realized, in accordance with the exemplary embodiments of this invention.

0005In a first aspect thereof the examples of the embodiments of this invention provide a method that comprises decomposing functionalities of a node of a wireless communications system into a plurality of partitions comprised of at least one application part for executing at least signaling plane functions and that interfaces logically to other signaling entities, at least one control part for executing at least transport functions and at least one network element part for executing at least data forwarding functions. The method further comprises virtualizing the at least one application part and configuring at least one network element to perform at least one data forwarding function under the direction of the at least one control part.

0006In another aspect thereof the examples of the embodiments of this invention provide an apparatus that comprises at least one processor and at least one memory that includes computer program code. The at least one memory and computer program code are configured, with the at least one processor, to cause the apparatus at least to decompose functionalities of a node of a wireless communications system into a plurality of partitions comprised of at least one application part for executing at least signaling plane functions and that interfaces logically to other signaling entities, at least one control part for executing at least transport functions and at least one network element part for executing at least data forwarding functions. The at least one memory and computer program code are further configured, with the at least one processor, to cause the apparatus at least to virtualize the at least one application part and configure at least one network element to perform at least one data forwarding function under the direction of the at least one control part.

0007In yet another aspect thereof the examples of the embodiments of this invention provide a network element that comprises a flow table; a first interface for coupling with at least one software defined networking (SDN) controller; and a second interface comprising a plurality of ports configurable by the at least one SDN controller while observing constraints imposed by said flow table. The network element is preconfigured or is configurable to operate on data that is received at a first port, to process the received data in a manner that modifies the received data, and to send processed data from a second port.

BRIEF DESCRIPTION OF THE DRAWINGS

0008The foregoing and other aspects of the exemplary embodiments of this invention are made more evident in the following Detailed Description, when read in conjunction with the attached Drawing Figures, wherein:

0009<figref idref="DRAWINGS">FIG. 1</figref> is a high level diagram of a conventional mobile network and is useful when explaining the concepts of network virtualization (NV).

0010<figref idref="DRAWINGS">FIG. 2</figref> is a high level block diagram that is useful when describing the concepts of software defined networking (SDN).

0011<figref idref="DRAWINGS">FIG. 3</figref> shows a more specific example of the general SDN architecture of <figref idref="DRAWINGS">FIG. 2</figref> and is useful in explaining the issues involved by presenting an example of how SDN can be applied to a typical network services scenario. The NE depicted in Figure can be considered to represent a class <b>0</b> NE.

0012<figref idref="DRAWINGS">FIG. 4</figref> shows an embodiment of a class <b>1</b> NE in accordance with embodiments of this invention that can be used for tunneling applications.

0013<figref idref="DRAWINGS">FIG. 5</figref> shows a non-limiting example of a class <b>2</b> NE which enables rules to be stored to implement traffic shaping/policy enforcement.

0014<figref idref="DRAWINGS">FIG. 6</figref> shows a non-limiting example of a class <b>3</b> NE which enables rules to be stored to cipher/decipher selected data streams with applicable encryption parameters.

0015<figref idref="DRAWINGS">FIG. 7</figref> shows a non-limiting example of a class <b>4</b> NE which enables rules to be stored comprised of specific executable code that configures the NE to perform a desired function or functions relating to data received and sent via input/output ports.

0016<figref idref="DRAWINGS">FIG. 8</figref> shows a non-limiting example of an embodiment where an SDN controller can use a plurality of NEs having differing capabilities, and can query the NEs to determine their respective capabilities.

0017<figref idref="DRAWINGS">FIG. 9</figref> depicts a non-limiting example of one possible implementation of the exemplary mobile network shown in <figref idref="DRAWINGS">FIG. 1</figref>, where the implementation takes advantage of the various NE embodiments shown in <figref idref="DRAWINGS">FIGS. 3-5</figref> and <b>8</b>.

0018<figref idref="DRAWINGS">FIG. 10</figref> illustrates a data processing system that is owned by or otherwise usable by a network operator and that is configurable to perform at least some of the methods/processes in accordance with the embodiments of this invention.

0019<figref idref="DRAWINGS">FIG. 11</figref> is a logic flow diagram that illustrates the operation of a method, and a result of execution of computer program instructions, in accordance with the exemplary embodiments of this invention.

DETAILED DESCRIPTION

0020In order to provide an appreciation of the various examples of the embodiments of this invention a discussion will first be provided of current network virtualization (NV) approaches, followed by a discussion of current software defined networking (SDN) approaches.

0021With respect to NV, some network operators are using information technology (IT) data center technologies in their services network, e.g., video on demand (VoD) and content delivery network (CDN). In data centers there is an ongoing trend to “cloudify”, i.e., to use virtualization technologies to decouple applications from specific hardware deployments. Typically applications such as video or WEB servers run on virtual machines (VMs) which can be deployed on any suitable hardware. Thus servers and applications may be invoked and subsequently shut down on demand, and the hardware (e.g. IT servers in racks, data storage) may be used almost arbitrarily. If applications are constructed in a suitable manner the same underlying hardware may serve many different types of applications. As opposed to building a proprietary cloud or clouds some service providers may make use of third party clouds. In this case the service providers can add the additional storage and computing capacities available in the third party cloud to an existing service provider cloud. Alternatively a service provider may rely solely on the use of the third party resources (on the third party cloud provider).

0022The use of cloud and virtualization technologies provides a number of benefits, e.g., only one (or few) hardware platforms are needed, the use can be on demand, assignments can be flexible and the total cost of ownership (TCO) can be dramatically reduced.

0023In that many operators, including mobile network operators (MNOs), are currently using or considering the use of NV technologies in their services networks, it is a logical next step to explore whether these principles (using one cloud for different applications) can also be applied to other mobile network domains such as the mobile core. The typical mobile core domain can include, for example, an IP multimedia subsystem (IMS), a home subscriber server (HSS) and a packet core, e.g., evolved packet core (EPC).

0024<figref idref="DRAWINGS">FIG. 1</figref> is a high level diagram of a conventional mobile network <b>1</b> that wirelessly serves user equipment (UE) <b>2</b> by using a number of different domains. These domains, in an exemplary long term evolution (LTE) embodiment, can include the radio access network (RAN) <b>3</b> containing base stations (evolved NodeBs or eNBs <b>3</b>A) that are linked to the mobile core network <b>4</b> (evolved packet core or EPC) via at least one signaling gateway (S-GW) <b>4</b>A. The S-GW <b>4</b>A includes control functions in addition to data forwarding functions and can be considered as an “edge node” of the mobile core network <b>4</b> (i.e., at the edge between the mobile core network <b>4</b> and the RAN <b>3</b>).

0025The domains of the mobile network <b>1</b> can also include a transport domain <b>5</b>, a PSTN/ISDN domain <b>6</b>, a mobile network services (MNS) domain <b>7</b>, a WWW domain <b>8</b> and a backend domain including, for example, an operations and maintenance (OAM) function and a billing and charging function. The PSTN/ISDN <b>6</b> and WWW <b>8</b> domains may also be considered as adjacent networks or as foreign or alien networks.

0026The MNS domain <b>7</b> has previously been subjected to some virtualization in some deployments. The embodiments of this invention concentrate more specifically on the nodes of the mobile core <b>4</b> and most specifically on the edge nodes of the mobile core <b>4</b>.

0027The various control functions of the mobile core <b>4</b> can include, as non-limiting examples, an IMS-based call session control function (CSCF) <b>11</b>A, a policy control function (PCF) <b>11</b>B, the HSS <b>110</b>, a home location register (HLR) <b>11</b>D, a mobile switching center-server (MSC-S) <b>11</b>E, and a mobility management entity (MME) <b>11</b>F. In addition, at the edges of the mobile core network <b>4</b> there are specific nodes (gateways) which have control functions but that also have data forwarding functions. Through these edge nodes not only signaling messages are passed (control plane or C-Plane) but also payload traffic (user plane or U-Plane) is handled. These edge nodes are shown as the S-GW <b>4</b>A at the edge of core network <b>4</b> to the RAN <b>3</b> that was mentioned above, as well as a packet data gateway (P-GW) <b>4</b>B to the external (alien) networks and also a media gateway (M-GW) <b>4</b>C to circuit switched (CS) telephony services networks.

0028Generally the core network functions and nodes can be classified into two categories: (a) pure control plane functions that exchange signaling information and (b) those functions and nodes that have both control functions and data forwarding functions. The first category of pure control plane functions and nodes is currently a focus of network virtualization since these can be “tailored” such that they can run as one or several applications on top of virtual machines in a cloud environment. However, the edge nodes (e.g., S-GW <b>4</b>A, P-GW <b>4</b>B and M-GW <b>4</b>C), due to their dual nature (control and data forwarding) do not lend themselves to simple virtualization following pure data center principles. This is true at least for the reason that in pure IT data center setups there are no nodes that exhibit the characteristics of the dual edge node gateway functionality. Thus, current data center technologies do not provide a suitable solution for gateway virtualization. However, without virtualization of these edge nodes the virtualization of the mobile core remains incomplete and deficient since the full potential of network virtualization cannot be applied to the mobile core network <b>4</b>.

0029SDN is another trend that is gaining momentum in CSP networks. Currently the typical nodes found in transport networks comprise specific functionalities. A typical router, for example, comprises in one package: data switching circuitry which moves data packets between different input/output (I/O) ports; control logic that handles the complex routing protocols such as RSVP; and routing tables and related functions. Another example is a carrier Ethernet switch that provides Layer 2 data forwarding and control. In addition various multi-layer switches are used in transport networks to provide MPLS (multi-protocol label switching) functionality which, on top of the above-mentioned router or switch functionality, provide MPLS/G-MPLS signaling capability. In general, and depending on the specific purpose of a transport node, it can be more or less complex to provide data forwarding and control functions in one monolithic node.

0030One basic idea of SDN is to decouple control functions from data forwarding functions. In other words, and by example, everything that makes a router a router and everything that makes a switch a switch is taken out of the node and placed in a controller. The node can be referred to as a “network element” or NE. What remains in the NE in these two cases would be a pure data forwarding functionality. Using this approach routers, switches, MPLS nodes and the like could all have a similar NE for data forwarding, but a specific control element (which is outside of the NE) which contains the router or switch or MPLS functionality (less the data forwarding/handling functionality).

0031<figref idref="DRAWINGS">FIG. 2</figref> is useful when describing the principles of SDN. In the lower portion of the Figure are shown NEs <b>10</b>. An exemplary NE <b>10</b> includes a SDN client <b>10</b>A, software <b>10</b>B, switching hardware (datapath) <b>10</b>C and a flow table <b>10</b>D. The datapaths <b>10</b>C provide I/O ports <b>10</b>E. The software <b>10</b>B allows for configuration of the NE <b>10</b> while the flow table <b>10</b>D contains port-based rules for data forwarding. The software <b>10</b>B can be stored in any suitable type of non-transitory computer readable medium such as semiconductor memory including Flash-type memory.

0032A non-limiting example of packet handling depending on, for example, header information is now provided. In this example a certain rule may be that incoming packets on port <b>0</b> are analyzed such that depending on what information is in the packet header the packet is forwarded to port <b>2</b> or to port <b>3</b>. These rules, which are stored in the flow table <b>10</b>D, can be passed to the NE <b>10</b> from a controller which resides external to the NE <b>10</b> (denoted as the SDN control <b>12</b>). For this purpose a protocol for exchange is specified and both the SDN controller <b>12</b> and the NE <b>10</b> are able to mutually understand the protocol (embodied in the functionality of the SDN client <b>10</b>A of the NE <b>10</b>).

0033One representative example of an SDN control protocol is OpenFlow (a Trademark of Stanford University) as specified in the Open Network Foundation (ONF). Another representative SDN control protocol is known as Forwarding and Control Element Separation (ForCES, see RFC 5810, March 2010, as well as, e.g., RFC 3654, RFC 3746 and RFC 5812.)

0034Shown by way of example and not as a limitation is a FlowVisor <b>14</b>: a special purpose OpenFlow controller that acts as a transparent proxy between OpenFlow switches and multiple OpenFlow controllers.

0035Using these techniques a system for sharing transport equipment can be constructed. The NEs <b>10</b> and SDN controllers <b>12</b> can be cascaded and access can be limited. For example, introducing the FlowVisors <b>14</b> can limit access to certain parts of the flow table <b>10</b>D (e.g., can limit access of the SDN controller <b>12</b> to only ports <b>0</b>, <b>1</b>, <b>2</b> and <b>3</b> of the NE <b>10</b>). The SDN controllers <b>12</b> themselves may act as proxies to other controllers and may provide a (“northbound”) interface (i/f) <b>16</b> to one or more applications <b>18</b>. The applications <b>18</b> may acquire network resources via the interface <b>16</b> in an abstracted manner, e.g., “connectivity between topological point A and topological point B with a given bandwidth”. The SDN controllers <b>12</b> may then instruct one or more NEs <b>10</b> out of a pool of NEs to fulfill the request of the application <b>18</b>. In practice there could be several options available to the SDN controller <b>12</b> to satisfy the request made by the application <b>18</b>. The abstract interface <b>16</b> effectively hides the specifics and complexities of the network hardware from the applications <b>18</b>.

0036To summarize the description thus far it was shown that one major obstacle to providing a fully virtualized mobile core <b>4</b> is the presence of the edge gateways or edge nodes <b>4</b>A-<b>4</b>C in <figref idref="DRAWINGS">FIG. 1</figref>, as existing data center technologies do not offer appropriate capabilities. The concepts involved in VN and SDN were also explained.

0037An aspect of the embodiments of this invention is an appreciation that SDN represents a technology that can aid in overcoming the deficiencies of NV if the edge gateways can be decomposed such that the control part can be virtualized and moved into a data center cloud environment, or more simply into the cloud, while the remaining forwarding part is implemented by means of SDN. However, existing approaches to apply SDN are not adequate to properly address this problem.

0038<figref idref="DRAWINGS">FIG. 3</figref> shows a more specific example of the general SDN architecture of <figref idref="DRAWINGS">FIG. 2</figref> and is useful in explaining the issues involved. <figref idref="DRAWINGS">FIG. 3</figref> presents an example of how SDN could be applied to a typical network services scenario. At the top level there is an application <b>18</b> which uses two types of communication services, real time voice <b>18</b>A and best effort data <b>18</b>B. The application <b>18</b> could implement, as a non-limiting example, a shopping portal where a user can browse in a virtual shop while in parallel can conduct an ongoing voice-over-internet protocol (VoIP) call with a hotline to ask questions concerning displayed merchandise. In a virtualized network the application <b>18</b> may request connectivity to an SDN controller <b>12</b> (which may or may not be embedded in the application <b>18</b>) using an abstract northbound interface <b>16</b>. The request could take the form of, by example, “real time connection between A and B and best effort connection between A and C”. The real time connection would be used for the VoIP call while the best effort connection would be used for the typically much lower speed browsing inputs (e.g., clicks). In response to the request from the application <b>18</b> the SDN controller <b>12</b> can configure one or several NEs <b>10</b> (the example of <figref idref="DRAWINGS">FIG. 3</figref> shows but a single NE <b>10</b> for simplicity).

0039Note that the NE <b>10</b> could be shared between several SDN controllers <b>12</b> and thus, to avoid conflicts, the FlowVisor <b>14</b> (or some other type of suitable subsystem) allows manipulations only of ports <b>0</b>, <b>1</b> and <b>2</b> for the SDN-controller <b>12</b> that implements the shopping portal application.

0040In operation the SDN controller <b>12</b> manipulates the accessible part of the flow table <b>10</b>D such that at the NE <b>10</b> incoming traffic on port <b>0</b> (which can comprise real time voice traffic and best effort data traffic from the user) is analyzed. If packet traffic header information contains an indication that VoIP is used (e.g., a real-time transport protocol (RTP) header is present) then the traffic is forwarded to a high performance part of the network (which in this example is accessible through port <b>1</b>), while all other traffic (e.g., the best effort traffic) is forwarded to port <b>2</b>.

0041This is one exemplary and typical deployment scenario for SDN-enabled nodes that serves to illustrate a drawback of using this approach. That is, the separation into control functions and data forwarding functions basically just converts the NE <b>10</b> into a simple forwarding engine that only allows for relaying or switching data packets from one port to another port without any further manipulation. Because of this fundamental limitation this type of NE <b>10</b> will be subsequently referred to and further denoted for convenience, and not by way of limitation, as a “class <b>0</b> NE” or as a “NE class <b>0</b>”. Note that while the class <b>0</b> NE may be well suited for pure transport packet transmission it is not well suited for relaying transport user data information that requires specific interworking on the forwarding plane. Examples of typically needed U-Plane data modifications, in the context of the mobile core network <b>4</b> of interest to this invention, include: GTP tunnel handling, policy enforcement/traffic shaping, encryption/decryption and stateful interworking, i.e., making data forwarding dependent on (user) authentication. As can be appreciated, in operation such U-Plane functions can be quite complex.

0042For example, the general packet radio system (GPRS) Tunneling Protocol User Plane (GTP-U) is a protocol used over the S1-U, X2, S4, S5 and S8 interfaces of the evolved packet system (EPS). GTP-U tunnels are used to carry encapsulated transport protocol data units (T-PDUs) and signaling messages between a pair of GTP-U tunnel endpoints. A tunnel endpoint identifier (TEID), present in the GTP header, indicates which tunnel a particular T-PDU belongs to.

0043Current SDN approaches do not properly address these types of issues, while modern edge nodes such as the S-GW <b>4</b>A and the P-GW <b>4</b>B require this type of functionality since S-GWs and P-GWs handle GTP tunnels, and the P-GWs perform policy enforcement.

0044The embodiments of this invention address and overcome these problems by allowing virtualization of mobile core domain <b>4</b> nodes such as gateway nodes (edge nodes), in such a manner that the application-related parts (control plane) may run in data center environments (e.g., in a cloud environment) while the forwarding plane is implemented using SDN principles that are enhanced such that specifically needed gateway functions can be incorporated and implemented.

0045A “cloud environment” may be considered for the purposes of describing the exemplary embodiments of this invention as including any types of suitable data processors, servers, mass storage devices, network interfaces and switches, and any other hardware/software/operating systems and the like that are needed in order to host one or more virtual machines executing one or more client applications, such as an HSS application, a MME application and a PCF application, as three non-limiting examples of applications that can be run when virtualizing the mobile core network <b>4</b>.

0046One non-limiting aspect of this invention is an expansion of the types of mobile core network <b>4</b> applications that can be instantiated in the cloud to include the application parts or partitions (after decomposition) of the gateway nodes (edge nodes) of the mobile core network <b>4</b>.

0047In accordance with the examples of the embodiments of this invention different classes of NEs are introduced, while considering as an example but not by way of limitation a current state of the art NE as being of class <b>0</b> (as shown in <figref idref="DRAWINGS">FIG. 3</figref> and discussed above).

0048Following this classification approach NEs <b>10</b> are modified to include additional functionality beyond pure packet forwarding, such as tunnel handling capabilities. <figref idref="DRAWINGS">FIG. 4</figref> shows a node that is capable of handling tunnels such as, but not limited to, the GTP-U tunnels mentioned above. Tunnels in this context mean that data packets may be enveloped or encapsulated in another frame with a tunnel header and with the tunnel having endpoints (node A, node B). Typical examples of tunnels are the GTP tunnels as used in mobile core networks and those implemented with Point-to-Point Protocol over Ethernet (PPPoE) encapsulation. PPPoE is a network protocol for encapsulating PPP frames within Ethernet frames and can be used in fixed DSL networks.

0049The extensions to an NE <b>10</b> (which converts it into the class <b>1</b> NE of <figref idref="DRAWINGS">FIG. 4</figref>) comprise additional rules <b>20</b>, i.e., tunneling rules <b>20</b>A that can be applied to tunnel types and/or ports. As an example, the class <b>1</b> NE <b>10</b> can be defined to terminate all GTP tunnels that are received through port <b>0</b>, strip off tunnel frames and forward to port <b>1</b>. In addition, the class <b>1</b> NE can be defined such that all non-tunneled packets received through port <b>0</b> are supplied with tunneling framings according to parameters given in the rules <b>20</b> (the tunnel handling rules <b>20</b>A defining tunnel end point address).

0000An exemplary set tunnel handling rules <b>20</b>A can be as follows: <br /> rules for tunnel type <b>1</b> at port <b>0</b><br /> <type info> <br /> <tunnel params> <br /> input port <b>0</b>: <ul id="ul0001" list-style="none"><li id="ul0001-0001" num="0000"><ul id="ul0002" list-style="none"><li id="ul0002-0001" num="0050">if <type info> in header, then <ul id="ul0003" list-style="none"><li id="ul0003-0001" num="0051">strip off (terminate) tunnel info and forward to port <b>1</b></li></ul></li><li id="ul0002-0002" num="0052">if no <type info> in header, then <ul id="ul0004" list-style="none"><li id="ul0004-0001" num="0053">add tunnel of <type info> with <tunnel params> and forward to port <b>2</b></li></ul></li></ul></li></ul>

0054The SDN control protocol <b>12</b> is thus extended to allow for the transfer of tunneling rules <b>20</b>A from the SDN controller to the class <b>1</b> NE <b>10</b> and to thereby enhance at least the content of the flow table <b>10</b>D and the functionality of the NE <b>10</b>.

0055<figref idref="DRAWINGS">FIG. 5</figref> shows a non-limiting example of a class <b>2</b> NE <b>10</b> which enables rules <b>20</b>B to be stored to implement traffic shaping/policy enforcement.

0056Such a traffic shaping/policy enforcement rule <b>20</b>B could comprise, by example, “all traffic on port <b>0</b> with a given source IP address in the header shall be manipulated such that a maximum throughput of 10 MBit/s is achieved. In case the data stream exceeds the limitation (or drops below a threshold), then the SDN controller <b>12</b> shall be informed”.

0000Stated more formally, the traffic shaping/policy enforcement rule <b>20</b>B can comprise: <br /> rules for traffic shaping at port <b>0</b><br /> <traffic info> <br /> <traffic params> <br /> input port <b>0</b>: <ul id="ul0005" list-style="none"><li id="ul0005-0001" num="0000"><ul id="ul0006" list-style="none"><li id="ul0006-0001" num="0057">if <traffic info> in header, then <ul id="ul0007" list-style="none"><li id="ul0007-0001" num="0058">limit packets per second to values found in <traffic params> report violations to controller</li></ul></li></ul></li></ul>

0059In this non-limiting example the <traffic info> could specify at least the IP source address.

0060A further NE class is shown <figref idref="DRAWINGS">FIG. 6</figref>. In this embodiment a class <b>3</b> NE <b>10</b> operates to cipher/decipher selected data streams with applicable encryption parameters (such as keying material) specified in rules <b>20</b>C provided by the SDN controller <b>12</b>. In the illustrated example, and still considering that the NE <b>20</b> is constrained, for example by the FlowVisor (or similar) functionality <b>14</b>, to use only ports <b>0</b>-<b>2</b>, ciphered data received on port <b>0</b> is deciphered and forwarded to port <b>1</b> while unciphered (e.g., plain text) data received on port <b>0</b> is ciphered using <keys> parameters and sent to port <b>2</b>.

0000Stated more formally, the encryption handling rule <b>20</b>C can comprise: <br /> rules encryption handling <b>1</b> at port <b>0</b><br /> <encrypt info> <br /> <keys> <br /> input port <b>0</b>: <ul id="ul0008" list-style="none"><li id="ul0008-0001" num="0000"><ul id="ul0009" list-style="none"><li id="ul0009-0001" num="0061">if <encrypt info> in header, then <ul id="ul0010" list-style="none"><li id="ul0010-0001" num="0062">decipher using <keys> and forward to port <b>1</b></li></ul></li><li id="ul0009-0002" num="0063">if no <encrypt info> in header, <ul id="ul0011" list-style="none"><li id="ul0011-0001" num="0064">then cypher using <keys> and forward to port <b>2</b></li></ul></li></ul></li></ul>

0065It can be noted with regard to this exemplary embodiment that the class <b>3</b> NE <b>10</b> can be constructed to include certain hardware, e.g., an encryption engine chip, a higher performance data processor, additional memory/faster memory, than would be present in a different type of NE <b>10</b>, e.g., in a class <b>1</b> NE <b>10</b>. In like manner the class <b>1</b> NE can be constructed to include hardware resources specifically tailored to perform high speed tunnel handling operations. In general the NEs <b>10</b> can be designed to support some desired maximum level of functionality, although in some cases they could also be re-programmed/re-purposed to support some lower level of functionality if needed.

0066<figref idref="DRAWINGS">FIG. 7</figref> illustrates a class <b>4</b> NE <b>10</b> that can allow a further significant extension of functionality through downloading rules comprised of specific code <b>20</b>D. Here no specific NE functionality needs be determined beforehand. If specific functionality is required then executable code <b>20</b>D providing the desired capabilities is downloaded to the NE <b>10</b>, such as from the SDN controller <b>12</b>. This may be accomplished via an OpenFlow Control Path and/or by downloading directly through a given port <b>10</b>E with an indication (e.g., in the header) that the associated data is executable code meant for the NE <b>10</b>. In this case the NE <b>10</b> includes some data processor (DP) functionality <b>10</b>F having the capability to execute the code <b>20</b>D. If the specific functionality is no longer required then the code <b>20</b>D can be erased and/or it can be overwritten with different code to provide the NE <b>10</b> with a different capability or capabilities as needed. In general the executable code <b>20</b>D can be stored in any suitable type of non-transitory computer readable medium such as semiconductor memory including Flash-type memory. The executable code <b>20</b>D could be stored in the same memory device as the software <b>10</b>B or in a separate memory device.

0000In this example the rules can be expressed as follows: <br /> rules for processing at port <b>0</b><br /> <code> <br /> use code for processing of input data (port <b>0</b>) <br /> report any incidents to controller <br /> forward to ports (<b>1</b> or <b>2</b>) according to flow table entries and processing results

0067It should be noted that the class <b>4</b> NE <b>10</b> could be used to implement, for example, one or more of the class <b>0</b>, class <b>1</b>, class <b>2</b> and class <b>3</b> NEs by providing appropriate code <b>20</b>D, such as code that includes encryption/decryption code in the case of the class <b>3</b> NE. Further in this regard, and while all NE classes (e.g., <b>0</b>, <b>1</b>, <b>2</b> and <b>3</b>) could potentially be implemented with one NE class <b>4</b> containing appropriate code <b>20</b>D, in this case it may be necessary for the network operator to store code images for different purposes for all possible network operator NE vendors. The network operator may further then need to manage which code image matches which NE vendor for which desired functionality. One result is that the network operator could require detailed knowledge of the hardware of the various NEs sourced by various vendors. Furthermore, using one generic NE capable of supporting all NE classes could result in some cases where the NE has more inherent capability (e.g., processing power, memory speed and/or capacity), and hence more cost, that what is actually required for a particular task. Being able to acquire already classed NE hardware (e.g., a class <b>2</b> or a class <b>3</b> NE) can thus potentially allow the network operator to better trade off needed functionality versus cost.

0068Using the teachings of this invention it becomes possible for vendors and others to specialize in specific classes of data forwarding NEs <b>10</b> and provide solutions suitable for different environments. Additionally it becomes possible to combine different classes of NEs <b>10</b> such that a network operator can easily and quickly reconfigure at least the transport network and assign functionality according to given circumstances.

0069<figref idref="DRAWINGS">FIG. 8</figref> shows a non-limiting example of an embodiment where an SDN controller <b>12</b> can take maximum advantage of combining different classes of network forwarding elements. In this embodiment the SDN controller <b>12</b> contains or otherwise has access to a database <b>12</b>A that stores the capabilities of different NEs <b>10</b>. In order to keep the database <b>12</b>A updated the SDN controller <b>12</b> can issue a “capabilities enquiry” request to a population of available NEs <b>10</b> which in turn report the type or types of classes that they support. Thus there could be present NEs <b>10</b> which support only pure forwarding (class <b>0</b>) while others may support one or a combination of additional functionality as described above (e.g., class <b>3</b>, classes <b>2</b>, <b>3</b> and <b>4</b>, classes <b>1</b> and <b>2</b>, etc.) In this case the SDN client <b>10</b>A can be configured to at least respond to the capabilities enquiry and return a response.

0070Upon a request of an application <b>18</b> through the northbound interface <b>16</b> the SDN controller <b>12</b> can configure appropriate NEs <b>10</b> and gain further flexibility by taking into account the additional functionality of the different classes that are present.

0071<figref idref="DRAWINGS">FIG. 9</figref> describes a non-limiting example of an implementation for the network example shown in <figref idref="DRAWINGS">FIG. 1</figref> that solves the problems described above by taking advantage of the embodiments shown in <figref idref="DRAWINGS">FIGS. 3-8</figref>.

0072In the upper portion of <figref idref="DRAWINGS">FIG. 9</figref> there is shown a simplified architecture of a conventional mobile network <b>1</b>, without virtualization or software-defined networking (SDN). The UE <b>2</b> sends and receives payload data (U-plane, solid line through base station (eNB <b>3</b>A) and gateways (S-GW <b>4</b>A, P-GW <b>4</b>B)) to and from destinations which may be, for example, a server in the web (e.g., in the WWW domain <b>8</b> shown in <figref idref="DRAWINGS">FIG. 1</figref>). In order to provide proper service delivery some control information is exchanged over the C-Plane (dotted lines) between various network elements as specified for a 3G/4G cellular network (or other types of cellular and non-cellular networks). The EPC <b>4</b> in this non-limiting example is shown to include just the MME <b>11</b>F, the HSS <b>11</b>C and the PCF <b>11</b>B, although other EPC <b>4</b> elements and functions would be present as well.

0073The lower portion of <figref idref="DRAWINGS">FIG. 9</figref> shows the mobile network <b>1</b> with the same functionality as in the upper portion, but wherein parts of the core network (EPC <b>4</b>) are virtualized. For example, control functions such as the HSS <b>11</b>C, MME <b>11</b>F and the PCF <b>11</b>B run as applications HSS-App, MME-App and PCF-App in a cloud computing environment <b>30</b>.

0074In accordance with embodiments of this invention, and in order to achieve full virtualization, the edge nodes are decomposed and re-arranged following the principles described above.

0075It can be noted that, by way of example, in LTE the S-GW <b>4</b>A protocols and interfaces may include: an S11 control plane stack to support an S11 interface with the MME <b>11</b>F, S5/S8 control and data plane stacks to support an S5/S8 interface with the P-GW <b>4</b>B, an S1 data plane stack to support an S1 user plane interface with the eNB <b>3</b>A, and an S4 data plane stack to support an S4 user plane interface between the RNC of UMTS and S-GW of the eNB. Further by way of example, in LTE the P-GW <b>4</b>B protocols and interfaces may include the S5/S8 control and data plane stacks to support the S5/S8 interface with the S-GW <b>4</b>A.

0076As an intermediate step (shown in the central portion of <figref idref="DRAWINGS">FIG. 9</figref>) these functionalities of the S-GW <b>4</b>A and the P-GW <b>4</b>B are each de-composed into three sections or partitions. These partitions include an Application part (S-GW-App, P-GW-App) that comprises, for example, all GW signaling functions (typically those specified in the applicable wireless network standard(s)). Generally speaking the Application part includes the software that interfaces logically to other signaling entities. For example, the S-GW-App communicates logically with the MME <b>11</b>F, the eNB <b>3</b>A and the HSS <b>11</b>C. Most preferably, the Application part is designed in a way that it can run on top of a virtual machine in the cloud <b>30</b>, and can then be added as another application running in the cloud <b>30</b> along with, for example, the MME-App and HSS-App (and any other Apps that may be virtualized and instantiated in the cloud <b>30</b>).

0077Another result of the edge node decomposition is the creation of the Control part (partition) (S-GW-Ctrl, P-GW-Ctrl). These represent the control functionality (SDN control <b>12</b>) that has been removed from the existing S-GW/P-GW edge nodes. The Control part is used to steer the transport resources as described above. Most preferably, the Control part can be designed as an extension of an existing overall network controller and then it may be added to the SDN controller <b>12</b>, or more simply it is the SDN controller <b>12</b>.

0078Another result of the edge node decomposition is the Network Element <b>10</b> part (S-GW-NE, P-GW-NE). This represents the data forwarding part of each formerly monolithic gateway node. The S-GW-NE and P-GW-NE follow the principles of SDN (i.e., it can be steered/controlled by the SDN controller <b>12</b>). Most preferably it is equipped with the various examples of the extensions described in <figref idref="DRAWINGS">FIGS. 3-7</figref>.

0079In accordance with these embodiments of the invention, and when for example a S-GW <b>4</b>A is to be put into service, the network operator, using one or more data processing systems, launches the necessary number of applications (S-GW-Apps) on virtual machines in the cloud <b>30</b>. The necessary number of launched applications is the number needed to match the needed performance requirements (e.g., three instances of the S-GW-App to achieve a maximum of 5000 transactions per minute). Through the northbound interface <b>16</b> the S-GW-App makes a request for appropriate connectivity and functionality to the SDN controller <b>12</b>. For example a request is made for connectivity from the eNB <b>3</b>A to the S-GW with 10 GBit/s and GTP tunnel capability. The SDN controller <b>12</b> has an overview of the network topology (which NE <b>10</b> is connected where and to whom) and the capabilities of each NE <b>10</b> (e.g., class <b>0</b> to class <b>4</b>) and can assign appropriate functionality to the NE <b>10</b>. Note that a certain NE <b>10</b> that was being used solely for pure packet forwarding (e.g., as a router) may on demand be at least partially configured as an S-GW-NE to provide class <b>1</b> capabilities which can be used to fulfill a given request. At another time the full S-GW capacity might not be needed and so the SDN controller <b>12</b> may reconfigure the NE <b>10</b> such that it is used again simply as a router, possibly enabling it to be shared with another operator.

0080In this type of scenario the cloud <b>30</b> environment could be one hosted by a third party cloud provider, and the SDN controller <b>12</b> and NEs <b>10</b> can be hardware/software entities hosted by one or more data processing and other systems of the network provider (or by some other party in some cases).

0081One non-limiting example of the use of this functionality could be an occurrence of an event at a certain location (e.g., a stadium hosting a sporting match) where there is a temporary high demand for additional gateway functionality. In this case the operator can flexibly react by converting a simple router NE into a gateway NE, and when the event is over, re-assign the NE back to the router function.

0082A data processing system <b>100</b> that is owned by or at least otherwise usable by a network operator is shown in <figref idref="DRAWINGS">FIG. 10</figref>. The data processing system <b>100</b> includes at least one data processor <b>102</b>, at least one memory <b>104</b> storing program code <b>106</b> that is executable by the data processor <b>102</b>, and at least one interface <b>108</b> enabling communication with, for example, the cloud computing environment <b>30</b> shown in <figref idref="DRAWINGS">FIG. 9</figref>. By the use of the data processing system <b>100</b> the network operator, or some agent of the network operator, is enabled to decompose functionalities of a node, such as an edge node, of the mobile core domain <b>4</b> of the wireless communications system <b>1</b> into a plurality of partitions comprised of at least one application part for executing at least signaling plane (C-Plane) functions. The application part can be considered simply for purposes of explanation as that part of the edge node that interfaces logically to other signaling entities. Another partition that results from the decomposition of the edge node by the data processing system <b>100</b> (e.g., by operation of the program code <b>106</b>) is at least one control part for executing at least transport functions. Another partition that results from the decomposition of the edge node by the data processing system <b>100</b> is the at least one network element part (NE <b>10</b>). The combination of the control part and the network element part are configurable for jointly executing at least data forwarding (U-Plane) functions of the edge node. Once the partitions have been created the data processing system <b>100</b> can interact with the provider of the cloud <b>30</b> (if the cloud environment is not hosted by the network operator) to virtualize the at least one application part. The data processing system <b>100</b> is also programmable to configure at least one NE <b>10</b> to perform at least one data forwarding function under the direction of the at least one control part. The control part can be instantiated by the data processing system <b>100</b> as at least one software-defined networking (SDN) controller <b>12</b>. The at least one NE <b>10</b> includes the plurality of SDN controller configurable ports <b>10</b>E to receive data and to send data, and is also preconfigured or configurable (e.g., via downloaded code) to perform operations on received data such as tunnel termination/origination, encryption/decryption, traffic shaping and other needed user plane transport functions.

0083During operation of the wireless communications system <b>1</b> the data processing system <b>100</b> is enabled to reconfigure/repurpose if needed one or more of the NEs <b>10</b> via the virtualized application parts (e.g., S-GW-Ctrl-App) and the SD controller(s) <b>12</b>. One or more of the NEs <b>10</b> could also possibly be shared with one or more other network operators.

0084The data processing system <b>100</b> could itself be instantiated in the same cloud <b>30</b> shown in <figref idref="DRAWINGS">FIG. 9</figref> or in a different cloud environment. In this case the data processor <b>102</b>, memory <b>104</b>, etc. can all be logically resident in some cloud computing environment and the program code <b>106</b> can be an application or applications executed by one or more virtual machines (VMs) that are instantiated in the cloud computing environment.

0085It should be appreciated that there are a number of advantages and benefits that can be realized from the use of the embodiments of this invention. For example, the use of the embodiments of this invention, including the decomposition/partitioning aspects thereof, enables a complete virtualization of the mobile core network to be accomplished, including the edge nodes. Further by example, the use of the embodiments of this invention enables a very flexible re-assignment of resources to be accomplished and further allows for compensations to be made for fluctuations in wireless network loading. Further by example, the use of the embodiments of this invention enables proven technologies such as SDN and virtualization to be extended and exploited to provide new and important functionalities in the wireless network area. Further by example, the use of the embodiments of this invention enables sharing of resources between operators to occur and facilitates the hosting of core networks. Still further by example, the use of the embodiments of this invention enables large TCO reductions to be realized since resources can be used (and re-used) in an optimized manner.

0086Based on the foregoing it should be apparent that the exemplary embodiments of this invention provide a method, apparatus and computer program(s) to accomplish a virtualization of a mobile core network including, but not limited to, those nodes having dual signaling plane and user data plane functionalities, such as the signaling gateway and the packet data gateway.

0087<figref idref="DRAWINGS">FIG. 11</figref> is a logic flow diagram that illustrates the operation of a method, and a result of execution of computer program instructions, in accordance with the exemplary embodiments of this invention. In accordance with these exemplary embodiments a method performs, at Block <b>11</b>A, a step of decomposing functionalities of a node of a wireless communications system into a plurality of partitions comprised of at least one application part for executing at least signaling plane functions and that interfaces logically to other signaling entities, at least one control part for executing at least transport functions and at least one network element part for executing at least data forwarding functions. At Block <b>11</b>B there is a step of virtualizing the at least one application part. At Block <b>110</b> there is a step of configuring at least one network element to perform at least one data forwarding function under the direction of the at least one control part.

0088In accordance with the method shown in <figref idref="DRAWINGS">FIG. 11</figref>, the step of configuring can be performed in response to a request made to the at least one control part by the at least one application part.

0089In accordance with the method shown in <figref idref="DRAWINGS">FIG. 11</figref>, the step of virtualizing operates to instantiate the at least one application part on a virtual machine in a cloud environment.

0090In accordance with the method shown in <figref idref="DRAWINGS">FIG. 11</figref> and discussed above, the method further comprises instantiating the control part as at least one software-defined networking (SDN) controller, and where the at least one network element can be comprised of a plurality of SDN controller configurable ports to receive data and to send data.

0091In accordance with the method shown in <figref idref="DRAWINGS">FIG. 11</figref> and discussed above, the at least one network element can be further configured to receive data from a tunnel at a first port, to terminate the tunnel and to send data that was encapsulated in the tunnel from a second port.

0092In accordance with the method shown in <figref idref="DRAWINGS">FIG. 11</figref> and discussed above, the at least one network element can be further configured to receive data at a first port, to encapsulate the data in accordance with predetermined tunnel parameters, and to send the encapsulated data as tunneled data from a second port.

0093In accordance with the method shown in <figref idref="DRAWINGS">FIG. 11</figref> and discussed above, the at least one network element can be further configured to perform traffic shaping for data received at a first port in accordance with predetermined traffic shaping parameters, and to send the traffic shaped data from a second port.

0094In accordance with the method shown in <figref idref="DRAWINGS">FIG. 11</figref> and discussed above, the at least one network element can be further configured to perform encryption of data received at a first port in accordance with predetermined encryption parameters, and to send the encrypted data from a second port.

0095In accordance with the method shown in <figref idref="DRAWINGS">FIG. 11</figref> and discussed above, the at least one network element can be further configured to perform decryption of encrypted data received at a first port in accordance with predetermined decryption parameters, and to send the decrypted data from a second port.

0096In accordance with the method shown in <figref idref="DRAWINGS">FIG. 11</figref> and discussed above, the at least one network element can be further configured to store executable code defining processing rules, to perform processing of data received at a first port in accordance with the processing rules, and to send the processed data from a second port.

0097In accordance with the method shown in <figref idref="DRAWINGS">FIG. 11</figref> and discussed above, the at least one network element can be further configured to receive data at a first port, to examine the received data, and to route the data to either a second port or a third port based on information stored in a flow table.

0098In accordance with the method shown in <figref idref="DRAWINGS">FIG. 11</figref> and discussed above, the at least one network element is comprised of n ports and where the SDN controller is constrained to use m ports, where m&lt;n.

0099In accordance with the method shown in <figref idref="DRAWINGS">FIG. 11</figref> and discussed above, the SDN controller is a first SDN controller coupled with a first virtualized application, and where there can be a second SDN controller that is coupled with a second virtualized application and that uses at least one port not used by the first SDN controller.

0100In accordance with the method shown in <figref idref="DRAWINGS">FIG. 11</figref> and discussed above, the SDN controller can be coupled with a network element capabilities database. In this case the method further comprises sending a capabilities enquiry to one or more network elements, receiving a capabilities response from the one or more network elements, and storing network element capabilities information in the capabilities database.

0101In accordance with the method shown in <figref idref="DRAWINGS">FIG. 11</figref> and discussed above, where a capabilities response received from a particular network element indicates that the network element has at least one or more of a routing capability, a tunnel handling capability, an encryption/decryption capability, a traffic shaping capability, and an ability to be programmed to have a particular capability or capabilities.

0102In accordance with the method shown in <figref idref="DRAWINGS">FIG. 11</figref> and discussed above, the node is an edge node of a mobile core domain of a wireless communications system and can be comprised of one of a signaling gateway (S-GW) coupled with a radio access network or a packet data gateway (P-GW) coupled with at least one packet data network.

0103In accordance with the method shown in <figref idref="DRAWINGS">FIG. 11</figref> and discussed above, the step of virtualizing can instantiate at least one of an application part of the S-GW and an application part of the P-GW on a virtual machine in a cloud environment. In this embodiment at least one signaling functionality of the mobile core domain, such as the MME, can also be instantiated in the same or in a different cloud environment.

0104The embodiments of this invention also pertain to a non-transitory computer-readable medium that contains software program instructions, where execution of the software program instructions by at least one data processor results in performance of operations that comprise execution of the method shown in <figref idref="DRAWINGS">FIG. 11</figref> and discussed above.

0105The various blocks shown in <figref idref="DRAWINGS">FIG. 11</figref> may be viewed as method steps, and/or as operations that result from operation of computer program code, and/or as a plurality of coupled logic circuit elements constructed to carry out the associated function(s).

0106In general, the various exemplary embodiments may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device, although the invention is not limited thereto. While various aspects of the exemplary embodiments of this invention may be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

0107It should thus be appreciated that at least some aspects of the exemplary embodiments of the inventions may be practiced in various components such as integrated circuit chips and modules, and that the exemplary embodiments of this invention may be realized in an apparatus that is embodied as an integrated circuit. The integrated circuit, or circuits, may comprise circuitry (as well as possibly firmware) for embodying at least one or more of a data processor or data processors, a digital signal processor or processors, baseband circuitry and radio frequency circuitry that are configurable so as to operate in accordance with the exemplary embodiments of this invention.

0108Various modifications and adaptations to the foregoing exemplary embodiments of this invention may become apparent to those skilled in the relevant arts in view of the foregoing description, when read in conjunction with the accompanying drawings. However, any and all modifications will still fall within the scope of the non-limiting and exemplary embodiments of this invention.

0109For example, and referring to the above-mentioned PPPoE embodiment, the examples of the embodiments of this invention can also be employed to advantage in a fixed-mobile convergence (FMC) network type of deployment where one operator has both a fixed network and a mobile network. In this exemplary deployment scenario the operator could deploy the NEs <b>10</b> plus relevant applications and software to configure and instantiate a broadband remote access server (BRAS) or a S-GW, or both simultaneously as needed, by using and re-using (configuring and re-configuring) one or more of the NEs <b>10</b>.

0110Further by example, while the exemplary embodiments have been described above at least partially in the context of the LTE system it should be appreciated that the exemplary embodiments of this invention are not limited for use with only this one particular type of wireless communication system, and that they may be used to advantage in other wireless communication systems, such as in WLAN or in GSM systems as two non-limiting examples.

0111It should be noted that the terms “connected”, “coupled”, or any variant thereof, mean any connection or coupling, either direct or indirect, between two or more elements, and may encompass the presence of one or more intermediate elements between two elements that are “connected” or “coupled” together. The coupling or connection between the elements can be physical, logical, or a combination thereof. As employed herein two elements may be considered to be “connected” or “coupled” together by the use of one or more wires, cables and/or printed electrical connections, as well as by the use of electromagnetic energy, such as electromagnetic energy having wavelengths in the radio frequency region, the microwave region and the optical (both visible and invisible) region, as several non-limiting and non-exhaustive examples.

0112Further, the various names used for the described network node, elements and functionalities (e.g., S-GW, P-GW, MME, HSS, eNB, etc.) are not intended to be limiting in any respect, as these various network node, elements and functionalities could be identified by any suitable names.

0113Furthermore, some of the features of the various non-limiting and exemplary embodiments of this invention may be used to advantage without the corresponding use of other features. As such, the foregoing description should be considered as merely illustrative of the principles, teachings and exemplary embodiments of this invention, and not in limitation thereof.