System and method for adjusting codec speed in a transmission path during call set-up due to reduced transmission performance

rmwpn οιοώ ymvz myoz/mpQ nn-ra ηΏκηη^ι περϊςπ ro-ra trrnrra ίπ^ πρν

SYSTEM AND METHOD FOR ADJUSTING CODEC SPEED IN ATRANSMISSION PATH DURING CALL SET-UP DUE TOREDUCED TRANSMISSION PERFORMANCE

5 BACKGROUND OF THE INVENTION

Telephony has advanced dramatically with the advancement of technology. Telephone communication was once limited to an analog public switched telephone network (PSTN), where the PSTN has been traditionally formed of two types of telephone carriers, local and long distance. The local carriers established local networks for subscribers to communicate 10 within local regions, and the long distance carriers created networks between the localnetworks to enable subscribers of different local earners to communicate with one another.

Over time, mobile telephone networks were developed to enable subscribers to usemobile telephones. At first, the wireless networks and handsets were analog. Technology forthe wireless networks was developed to provide digital wireless communications, which 15 provided a clearer signal than analog wireless communications.

About the same time that the digital wireless networks were developing, the Internetwas also developing. The International Standards Organization (ISO) developed an OpenSystems Interconnection (OSI) basic reference model in 1977 that currently includes seven. different layers. Each of the layers provides protocols for certain types of operations. More 20 specifically, the seven layers include: physical layer (Layer 1), data link layer (Layer 2),network layer (Layer 3), transport layer (Layer 4), session layer (Layer 5), presentation layer(Layer 6), and application layer (Layer 7). Each entity interacts directly with the layerimmediately beneath it and provides facilities for use by the layer above it. The protocols oneach layer enable entities to communicate with other entities on the same layer. The Internet 25 initially provided for simple digital data to be communicated between users. One of the earlycommunication application included email. However, as communications standards andprotocols developed, the Internet matured to include more advanced communicationapplications, including voice over Internet protocol (VoIP). FIG. 1 is an illustration of a legacy telecommunications network that includes class 4 30 and 5 switches 102a-102n (collectively 102) and 104a-104n (collectively 104), respectively, connected to a signaling system #7 (SS7) network 106 (indicated as dashed lines).

Historically, the class 5 switches 104 were generally configured to communicate via in-band signaling verses the use of SS7 signaling and operate to form a local network of subscribers 1 within the network to place telephone calls to one another. The class 4 switches 102 weredeveloped for long distance connections between the class 5 switches 104 at end offices (notshown). The class 4 switches 102, which are monolithic, are generally formed of multiplecomponents, including a port, port cross-connect matrix, switch messaging bus with external 5 signaling units, and call processing unit, as understood in the art. Class 4 switches are circuitbased and utilize time division multiplexing (TOM) and are capable of terminating higherhigh-speed communications, including ΤΙ, T3, OC-3, and other four-wire circuit connections.As understood in the art, TDM is a synchronous communications protocol.

The SS7 network 106, which includes signal transfer points (STPs) 108a-108n 10 (collectively 108), service switching points (SSs) on the class 4 and class 5 switches, andservice control points (SCPs - not shown). The SS7 network is connected to the class 4 and 5switches for providing and maintaining inter-switch call services between the switches. TheSS7 network is used to signal out-of-band call setup, as out-of-band signaling is more secureand faster than in-band signaling. The call state changes of the inter-switch trunks of the class 15 4 switch are communicated to the adjoining switches via the SS7 network via a connection to the STP. To manage and route calls, the STPs 108 are used as an Inter-Switch messagingnetwork, whereby two switches control the trunking between the switches via messaging overthe SS7 network provided by the STP switches that act as the inter-switch message bus. A callstate machine of the class 4 switch provides control for routing traffic within the cross-connect 20 matrix of the monolithic switch. The call state machine also provides call control signalinginformation to other switches via the connections to the STPs. The call control signalinginformation is routed via the STPs to other switches for call setup and tear-down. The callcontrol information routed by the STPs contains pertinent information about the call to allowthe terminating switch to complete various calls. 25 Telephony has benefited from the development of the OSI model in a vast number of

ways. One way has been through separating the call controller into a distributed cross-connecton an asynchronous network, such as asynchronous transfer mode (ATM) or a InternetProtocol (IP) network. FIG. 2 is an illustration of a conventional telephony network 200 thatincludes a packet network 202. In one embodiment, the packet network 202 is an ATM 30 network. Media gateways (MGs) 204a-204n (collectively 204) are media translation orconversion devices that modify and convert protocols between disparate communicationnetworks. The media gateways 204, which are in communication with class 5 switches 206a- 2 206n (collectively 206) are located at the edge of the packet network 202. The media gateways 204 convert TDM packets or streams 208 into packets, frames, or cells (collectively referred to hereinafter as “packets”) 210 and vice versa.

The packet network 202 operates independently as a distributed virtual media gateway 5 port cross-connect for voice calls primarily due to one or more call control managers (CCMs) 212 located on the packet network 202. The call control manager 212 is in communicationwith the media gateways 204 and operates to control the media gateways 204 and provideinstructions on how to rotate the packets 210 via far-end address allocations. By separating thecall controller fiom the class 4 switches, the packet network 202 becomes, in effect, the virtual 10 cross-connect of the switching system. The packet network 202 enables packets 210 thatinclude voice data, commonly known as bearer packets, to be tagged with a destination address214a and origination address 214b for enabling content data 214c to be properly routed fromthe origination media gateway 204b to the destination media gateway 204a. The mediagateways 204 use of the packet network 202 is controlled by the CCM 212 and may 15 communicate the packets 210 over the packet network 202 via IP addresses and virtual circuit(VC) or virtual path (VP) between the media gateways 204 to appropriately route the packetsto the correct destination network node through the packet network 202. CCM 212 receivescall state processing information from the media gateways 204 and signaling points, andprocesses the call state changes by using look up tables (not shown). The CCM 212 thereafter 20 communicates packet addressing and state changes to the media gateways 204 to process thecall.

Ethernet protocol was developed to provide for a computer network that enablesmultiple computers to share a common external inter-communication bus. Ethernet isgenerally used to provide for local area networks (LANs). Ethernet operates by 25 communicating frames of data. While Ethernet operates well within a local environment (e.g., within a building) because Ethernet assumes that there is an known capacity of bandwidthassociated with the bus standards set forth in the IEEE 802.3 standard that defines Ethernet.Ethernet is a shared environment, where co-utilization creates transmission errors calledcollisions. These collisions are detected by Ethernet cards in computers and a random re- 30 transmission timer is used to avoid the next collision. Ethernet poses special problems for use in communications systems given it lacks dedicated bandwidth and time slots. The shared 3 nature of an Ethernet network creates additional complexities in that the amount of available bandwidth can vary when used with wireless technologies.

Communication protocols transmitted over packet networks, such as ATM or IP networks, may utilize TDM based transmission facilities, which are synchronous as compared 5 to Ethernet transmission facilities, which are asynchronous. Synchronous transmissionprotocols utilize a common clock and channel schema so that each device on the networkoperates synchronously with a dedicated path. Two types of “connection” state knowledge arepresent in a dedicated system, such as a TDM. Each channel has a dedicated amount ofbandwidth and an error rate that is calculated from a common clock to determine path errors. 10 The two types of connection state awareness functionality are provided by the channel itselfand the common clock and data within a TDM header. The common clock provides for adetermination of (i) a communications data rate from one end-point to another end-point and(ii) the data quality. Additionally, the TDM protocol includes “far end state” data in a headerof a TDM frame to indicate whether there is a connection at the far end, thereby providing an 15 indication of continuity along the communications path. Specifically, in-band end-to-endalarming allows the cross-connect devices to receive indications of continuity problems withother end-points. The in-band alarming is also provided for connection quality, where BitError Rate (BER) allows each end-point to know the quality of the data being received.Furthermore, bandwidth is always in use meaning that packets are synchronous, which that the 20 far end knows exactly how many packets are to be sent and received in a given time period(e.g., one second). Computation of utilization is easily made by using the known bandwidthand multiplying it by the “seizure” time or the amount of time in use.

Packet-based communications sessions lack circuit based connection state awarenessindicators and clocking functionality to provide a session controller the ability to know the path 25 connectivity state to efficiently manage making call handling decisions with anything otherthan ample bandwidth to setup and use sessions. This lack of connection path state awarenesswith the communications protocols, such as Ethernet and Internet protocol (IP) technologies,result in “gaps” in terms of being able to react to decaying transmission path quality and besensitive to shared use of bandwidth. Most IP call controller solutions are founded on 30 enterprise applications, where a single entity owns the network and scale of the network isrelatively small. IP and Ethernet protocols lack the in-band path signaling, quality and usemetrics to allow for this scale, or the ability to perform enhanced call handling with paths 4 outside the governance of the packet network. Because packet communications are asynchronous, there is no common clock, and, thus, there is no way to know how many packets were transmitted, which, in turn, removes the ability to characterize transmission quality of the entire path, the amount of bandwidth available, or the amount in use. Further, packet networks 5 are “converged” meaning they have both real-time and non-real-time bandwidth use.Currently, there is no in-band mechanism for determining real-time and non-real-timebandwidth use; having such information would tillow for handling calls. It is commonlyunderstood that proper connection operational assumptions are made by call control engineswhen the SS7 signaling path is properly operating (e.g., provisioned bandwidth is available) 10 between end-points within the SS7 network. These operational assumptions are problematic asEthernet, IP, and other data networks become oversubscribed and cause the packet network tobecome congested and prevent throughput. In cases where an end-point, such as a WiFitelephone, is mobile and bandwidth changes with signal strength (e.g., a WiFi telephone losingbandwidth as an individual walks away from a connection point antenna), the connection 15 operation assumptions also fail to provide graceful call handling.

One available technique in packet networks to prevent oversubscription of real-time media traffic is through the use of call admission control (CAC) or the IP equivalent known asResource Reservation Protocol (RSVP). CAC is primarily used to prevent congestion in voicetraffic and is applied in the call setup phase to ensure there is enough bandwidth for data flow 20 by reserving resources. To reserve bandwidth through the entire packet network, a CACrequires that the CAC procedure be performed at each point along a virtual circuit between twomedia gateways on which a call is to be routed, and often in a bi-directional fashion. WhileCAC functionality exists, the use of such CAC functionality is almost never applied because ofthe amount of time needed by the CAC procedure during call set up. For example, currently, 25 CAC typically cannot operate over 40 calls per second and typical call set-ups on mediagateways or class 4 switches may be 200 calls per second or higher.

One technique used to monitor the performance of IP session performance (i.e., after acall session has been established) is the use of the real-time control protocol (RTCP) as definedin IETF RFC 3550. RTCP collects statistics on a media connection, including bytes sent, 30 packets sent, lost packets, jitter, feedback, and round trip delay. Other information may beprovided in the RTCP packet using profile specific extensions. RTCP, which operates on a persession basis, is used for quality of service (QoS) reporting after termination of a session. The 5 statistics information may be used, for example, to improve the quality of service by limiting data flow or changing CODEC compression. Utilization of the real-time QoS statistics, however, is limited to the specific session associated with the RTCP stream.

An emerging standard that is being developed for Ethernet performance measures is 5 802.1AG. This standard operates by generating and communicating an 802.1AG packet or “heart beat” over an Ethernet network segment. The 802.1AG packets are communicated via aLayer 2 Ethernet Virtual Circuit, such as a VLAN or Ethernet tunnel. At the ends and mid-points in Ethernet tunnels, 802.1AG packets are transmitted periodically over the Ethernetnetwork to the far end. The Y.1731 protocol is utilized to calculate the number of data frames 10 communicated between the 802.1AG packets. This configuration enables a performancemeasures (PM) to compute certain information about the performance of the path between theend-points on an Ethernet network. This combination of 802.1AG and Y.1731 enables the endpoints to be knowledgeable about the Frame Loss Rate (FLR), packet delay, and jitter in thepath. This configuration is helpful to assist in monitoring performance of an Ethernet network 15 path and diagnosing connectivity faults. However, the configuration falls short of providingthe amount of real-time bandwidth in use or the total bandwidth in use. This information isuseful to the proper management by a session controller handling calls during periods of fluxin the packet transmission path, or the management of the real-time traffic.

Service providers often have trouble isolating and diagnosing network problems. To 20 attempt to locate a packet loss problem along a node segment (i.e., a path between two networkcommunications devices) over a network, a probe that may be used to trace data packets beingcommunicated over the node segment. This probe, however, is typically an external devicefrom the network communications devices and operates to run a trace over an instant of time todetermine network performance information, such as packet loss, jitter, and delay. An operator 25 using the external probe may view results of a trace to diagnose the network communicationsproblem. These results are not accessible to the network communications devices and cannothe accessed by network communications devices to alter network communications.

Telecommunications switching systems today provide for Internet protocol (IP)communications between two end-points within a network or a different network to be 30 terminated to a far end-point. Calls between two end-points are routed to the terminating end-point based on the address input at the originator. This address information is then relayed to aCall Control Manager (CCM) that screens, translates, and routes the call to the terminating 6 subscriber or to another network to be terminated at a far end subscriber’s end-point. The basic functionality of this process is widely known within the art and is used throughout telecommunications networks for voice calling.

Within the architecture of this switching system, calls to and from end-points are 5 controlled by the CCM. The CCM may be located within a monolithic device in a TDMswitch architecture or provided by an outboard computing device that controls the calls byusing signaling that controls network based routing and switching devices located within thenetwork. The latter device is known as soft-switch architecture.

The soft-switch architecture within an IP network controls call processing through use 10 of signaling to and from the end devices and media gateways. One example of a protocol usedfor this IP signaling is Session Initiation Protocol (SIP). This protocol is currently used mainlywith IP telephony, such as VoIP, and can be used as an access protocol between the end-userand the CCM and/or between the CCM of one network and the CCM of another network.

Another protocol used mainly between the CCM and a media gateway is the ITU-T 15 H.248 protocol, commonly known as Megaco. This protocol is a control protocol that allows the CCM to control ingress and egress from/to the media gateway as calls are set up using amedia gateway. Within a packet network framework, IP communications between two end-points (both access end-points and media gateways) are controlled by the signaling of the end-point to/from the CCM. The CCM provides authentication, screening, translations and routing 20 based on information that is stored in the CCM and from the state of the end-points that theCCM controls.

Within the soft-switch architecture, call control can only be accomplished based oninformation possessed by the CCM or the on/off state of the devices that has an associationwith the CCM. While this configuration is fine in a static environment, packet networks are in 25 a state of change at all times since the network itself can carry different types of informationbesides voice calls. One skilled in the art knows that a packet network is a converged networkthat can carry voice, data, and video all in a single path, and routing of calls within a packetnetwork is not static and can vary significantly from call to call.

Because of packet network content communications variables, calls may encounter 30 congestion and loss of voice quality based on latency, jitter, and packet loss. These contentcommunications variables can affect any portion of a call at any time based on the networkelements usage at the time of the call. Unlike a TDM system where dedicated channels and 7 circuits are provided, the CCM only has control of it own end-points. Other end units may attempt calls, computers may send/receive data without talking to the CCM, and other devices may require bandwidth while the original call is progressing, thus causing voice quality problems for the participants. In addition to these basic gaps, many physical layer 1 systems 5 that are poor in regulating bandwidth, are being used for transmission facilities. WiFi, EVDO,4G (WiMax), DSL, and cable’ systems are all physical layer 1 technologies that demonstratedifferent bandwidth rates and management of their ability to modify available bandwidth as theSignal-to-Noise (SNR) ratios change.

Conventional soft-switching has not been designed to provide relief for callers when 10 congestion, jitter or delay problems, such as those described above, are encountered. Sinceconventional CCMs can only determine call success based on connectivity to and from thecalling parties, voice quality between two parties is not taken into consideration for callsuccess.

Communication problems of in-band signals over packet networks are difficult to 15 isolate. Currently, if a communication problem exists over a transmission path, there are fewtechniques to isolate the problem. One technique includes using an external probe to captureand decode packets, commonly know as a trace, traversing over a communications path to helpisolate the problem. However, technicians generally only run the trace in response to acustomer notifying a communications carrier of a communications problem. If a problem 20 exists across packet networks operated by different carriers, a typical response by a carrier is tocontact the other to determine if the other carrier can locate a problem in its network. In otherwords, locating an in-band communications problem over one or more packet networks isdifficult as troubleshooting tools for such problems are limited to out-of-band performancemetrics (PM) and are not available as in-band information via control or signaling paths. 25 A problem that exists with current implementations of telephony over packet networks is that a call control manager does not have information about the bearer path. Traditionally,there was a linkage between transmission path state and the monolithic switch that essentiallyowned one end of that path where the in-band signaling and line characteristics were availableand was an integral part of the information used by the CCM for call processing. As 30 demonstrated in current implementations of VoIP, without knowledge of the bearer path, thecall control manager may establish calls that result in poor voice quality or call setup failure. 8

In addition, IP Service gateways, such as a broadband remote access server (BRAS),functions to limit, commonly known as traffic rate shaping, each customer’s DSL traffic totheir purchased speed. There is no end-to-end signaling, outside of the embedded TCP flowcontrol mechanism, used to adjust the bursting to eliminate packet loss. Rate shaping is a 5 statically forced bandwidth constraint that alters the nature of a transmission path in the packetnetworks. This shaping coupled with commonly shared or “over-subscribed” bandwidthnormally associated with trunking facilities between networks results in unknown transmissionpath states between media gateways servicing VoIP and other real-time services, such as Videoon Demand (VOD). 10 Traffic Quality of Service (QoS) management of packets is performed, where multiple flows aggregate into a smaller flows or channels. The application of Internet Protocol QoS isperformed at the egress point where traffic is transmitted over a single link. Current trafficengines use the following information to make QOS traffic decisions. The decisions areassigning a Class of Service (CoS) and then acting upon that service to shape, restrict, or pass 15 traffic to an egress point. The variables used to assign priority to traffic flows can be based on:entrance port (assign a whole port a CoS), virtual circuit in a port (assign a CoS to an EthernetVirtual Circuit, LSP, etc.), priority bit marking of each packet (P bit), protocol type (assigninga CoS to specific types of packets or traffic), IP address and port (assigning a CoS to a wholeIP address, or its port addresses), session identification (a HTTP, UDP, or other session 20 addressed call), or otherwise. This priority marking information is used by service points, andshared links to implement QoS for the shared traffic flows. QoS and CoS types of informationare made available at the point of aggregation where traffic management or QoS functionsoccur. However, the number of packets transmitted or lost in the packet stream elsewhere inthe network is currently unavailable without the use of a session or path based protocol. These 25 packet loss functions are generally not tracked by QoS mechanisms.

In current traffic rate shaping designs, the Internet may burst a packet stream to a DSL user when the packet network or Digital Subscribe]: Line Access Multiplexer (DSLAM) itselfmay not have sufficient bandwidth to accommodate the packet session. In a TCP-basedsession, the transmission rate is throttled down after packet loss is detected in the session. In 30 VoIP, the packet loss is not counted by the use of Real Time Control Protocol (RTCP)signaling, but it is captured as the call progresses by the end points. RTCP, however, onlyconsiders performance of its own sessions and not the transmission path performance as a 9 whole. In both cases, packets are sent over the packet network that get dropped in mid-path and will not make it to the customer premises equipment (CPE) and user. More importantly, there is no cross-session information about the packet loss and no whole path information available in-band. 5 Also, packet loss can be due to available bandwidth transmission rate fall-off, such as when a WiFi user walks away from a WiFi Access Point (AP) and loses RF signal strength,signal-to-noise ratio increases, or congestion increases due to many users concurrentlyaccessing the AP. All of these types of conditions in the transmission paths can have severeimpacts upon the ability to accomplish call processing and call management. 10 An Internet Service Provider (ISP) may provide different Internet connectivity speeds or data transfer rates based upon their service plans. For example, a user may purchase 1.5Mbits/sec data transfer rate for a predetermined amount, such as lOMbits/sec data transfer rateor higher. In general, the transmission path is between the shared (trunked) BRAS resourceand the DSLAM that is supplying. The normal amount of bandwidth consumption in the

15 download direction from the network to the user is high as compared to the upstream direction.However, there is no correlated throttling mechanism in the IP web-server linked to user’s ISPservice plan that can be used to shape the packet transmissions. So, all of the network trafficis shaped at the BRAS typically based on the user’s purchased data transfer rates. Dependingupon network conditions, some of this traffic may not make it to the DSL user since the BRAS 20 has no knowledge of the IP service path from itself to the customer. A problem occurs when the BRAS does realize congestion on a packet network, where packets are being dropped due to insufficient bandwidth. Some packets could be dropped inthe packet network or at an aggregation device somewhere in the packet network. Currently,there is little intelligence that recognizes the dropped packets in the packet network. In fact, 25 packet networks are designed to discard traffic based on QoS markings. This problem is madeworse because transporting packets that will ultimately be dropped adds to congesting thenetwork. The packets consume bandwidth until dropped.

Transmission Control Protocol (TCP) was designed to work in a best-effort, packetstore-and-forward environment characterized by the possibility of packet loss, packet 30 disordering, and packet duplication. Packet loss can occur, for example, by a congestednetwork element discarding a packet. Here, a microprocessor or memory of a network elementmay not have adequate capacity to address all packets routing into and out of the element. 10

Packet disordering can occur, for example, by routing changes occurring during a transmission.Here, packets of TCP connection may be being arbitrarily transmitted partially over a lowbandwidth terrestrial path and as routing table changes occur partially over a high bandwidthsatellite path. Packet duplication can occur, for example, when two directly-connected 5 network elements use a reliable link protocol and the link goes down after the receivercorrectly receives a packet but before the transmitter receives an acknowledgement for thepacket.

An embedded capability within TCP protocol is the TCP sliding window technique.The sliding window was developed and deployed as a flow control mechanism used to 10 minimize the inefficiencies of packet-by-packet transmission. The sending of data betweenTCP enabled end-devices on a connection is accomplished using the sliding window technique.TCP requires that all transmitted data be acknowledged by the receiving host. The slidingwindows method is a process by which multiple packets of data can be affirmed with a singleacknowledgement.

15 SUMMARY OF THE INVENTION

To overcome the problem of the call control manager not knowing information about a bearer path between an originating call device and a termination call device, the principles ofthe present invention provide for a network segment status table that the call control managermay access to determine if node segments along the transmission path for the call are operating 20 properly. One embodiment includes a system and method for establishing a phone call over apacket network. The process may receive a call request from an originating call device to atermination call device. A determination may be made to determine whether the terminatingcall device is available. If the terminating call device is determined to be available,transmission path status information between the originating and terminating call devices may 25 be retrieved. Status of the transmission path on the packet network between the originatingand terminating call devices may be determined. If the status of the transmission path isdetermined to be within a first range, a call may be established between the originating andterminating call devices via an encoder/decoder (CODEC) having a first data rate. Otherwise,if the status of the transmission path is detennined to be within a second range, the call may be 30 established between the originating and terminating call devices via a CODEC having asecond, lower data rate. 11

BRIEF DESCRIPTION OF THE DRAWINGS

Illustrative embodiments of the present invention are described in detail below with reference to the attached drawing figures, which are incorporated by reference herein and wherein: 5 FIG. 1 is an illustration of a legacy telecommunications network that includes class 4 and 5 switches connected to a signaling system #7 (SS7) network; FIG. 2 is an illustration of a conventional telephony network that includes a packetnetwork; FIG. 3 is an illustration of an exemplary packet network that utilizes performance10 information packets to determine network performance information; FIG. 4 is an illustration of an exemplary data packet stream including PIP data packetsand data packets including real-time and non-real-time content; FIG. 5 is a block diagram of an exemplary network node configured to performfunctionality in accordance with the principles of the present invention; 15 FIG. 6 is a block diagram of exemplary modules configured to determine and collect network performance information in accordance with the principles of the present invention; FIG. 7 is an illustration of multiple exemplary data packet networks having exemplarynetwork nodes configured to determine and collect network performance information; FIG. 8 is a block diagram of an exemplary end or mid-point device showing structural20 and functional operations used for employing the principles of the present invention; FIG. 9 is a block diagram of an exemplary pin-hole firewall device showing structuraland functional operations used for employing the principles of the present invention; FIG. 10 is a block diagram of an exemplary head-end device showing structural andfunctional operations used for employing the principles of the present invention; 25 FIG. 11 is a block diagram of exemplary modules configured to determine network performance information associated with data packets communicated with networkcommunication devices described in FIGS. 8-10; FIG. 12 is an illustration of exemplary processes performed on network nodes in a datapacket network; 30 FIG. 13 is an illustration of an exemplary network node configured to perform functionality and communications over a packet network in accordance with the principles ofthe present invention; 12 FIG. 14 is a flow chart of an exemplary process for managing network communications; FIG. 15 is an illustration of an exemplary packet network having a call control manager with a centralized table of network performance information for use in managing call 5 communications over the packet network; FIG. 16 is a flow chart of an exemplar/ process for using network performanceinformation stored in a centralized table for controlling calls by a call control manager; FIG. 17 A is a flow chart of a high-level process for controlling communications on apacket network; 10 FIG. 17B is one embodiment of a permission table that may be utilized to establish permission or access levels by various network participants to network performanceinformation that has been collected over one or more networks; FIG. 18 is a block diagram of exemplify multi-node packet networks used tocommunicate data packets including network performance information generated by each node 15 in a transmission path; FIG. 19 is a flow diagram of an exemplary process for generating and communicatingnetwork performance information in data packets in accordance with the principles of thepresent invention; FIG. 20 is a flow diagram of an exemplary process for isolating a node within a packet20 network that generated network performance information indicating a transmission performance problem; FIG. 21 is an exemplary process for identifying communication problems within one or more packet networks; * FIG. 22 is an illustration of an exemplary packet network with one service provider and25 two operators; FIGS. 23A and 23B are illustrations of a multi-carrier network having multipleEthernet service providers (ESPs) and a multi-point network having a multi-point device incommunication with network interface devices; FIGS. 24A-24C (collectively FIG. 24) are flow charts of an exemplary process for30 performing line-to-line call flow; FIGS. 25A-25C (collectively FIG. 25) are flow diagrams of an exemplary process forproviding call processing for rerouting a call between an originating line and terminating trunk; 13 FIGS. 26A-26C (collectively FIG. 26) is a flow chart of an exemplary process for performing congestion control for calls coming through an IP trunk to a line; FIG. 27A is an illustration of an exemplary network system that includes two networks operated by different communications carriers; 5 FIG. 27B is an illustration of an exemplary billing entity for use in determining billing for customers and partners of a communications carrier; FIGS. 28A and 28B (collectively FIG. 28) are screenshots of exemplary web browserinterfaces; FIG. 29 is an illustration of an exemplary graphical user interface (GUI) that displays a10 schematic of a packet network and performance monitoring devices; FIG. 30 is a screenshot of another exemplary graphical user interface that is displayinga chart of node segments status usage for a particular node on a network; FIG. 31 is an illustration of the OSI 7-layer basic reference model; FIG. 32 is an illustration of an example of an Operations, Administration, and 15 Maintenance Entities depicting multiple administrative domains; FIG. 33 illustrates a block diagram of a network entity according to an embodiment of the present invention; FIG. 34 illustrates a plurality of Vector Performance Tables according to anembodiment of the present invention; 20 FIG. 35 illustrates a flow diagram of the MEF Maintenance Entity data flow according to an embodiment of the present invention; FIGS. 36 - 39 illustrate exemplary MEF Maintenance Entity payload ingress andegress data flows according to an embodiment of the present invention; FIG. 40 illustrates an end station ME payload data flow according to an embodiment of25 the invention; FIG. 41 illustrates a network diagram of a Vector Performance Correlation Engine(VPCE) according to an embodiment of the present invention; FIGS. 42a - 42c illustrate a Graphical User Interface (GUI) according to anembodiment of the present invention; 30 FIG. 43 illustrates a MEF network implementation according to an embodiment of the present invention; 14 FIG. 44 illustrates a MEF network implementation of inter-layer communicationbetween Data Link Layer devices and Physical Layer devices according to an embodiment ofthe present invention; FIG. 45 illustrates a wireline digital subscriber loop network according to anembodiment of the present invention; FIG. 46 illustrates a bit swapping table according to an embodiment of the presentinvention; FIG. 47 illustrates a wireless network according to an embodiment of the presentinvention; FIG. 48 illustrates a MEF network implementation of inter-layer communicationbetween Data Link Layer devices and Network Layer devices according to anotherembodiment of the present invention; FIG. 49 illustrates a MEF network implementation of inter-layer communicationbetween Data Link Layer devices and Transport Layer devices according to anotherembodiment of the present invention; FIG. 50 illustrates a TCP packet according to an embodiment of the present invention; FIG. 51 illustrates a MEF network implementation of inter-layer communicationbetween Data Link Layer devices and Session Layer devices according to another embodimentof the present invention; FIG. 52 illustrates a MEF network implementation of inter-layer communicationbetween Data Link Layer devices and Presentation Layer devices according to anotherembodiment of the present invention; FIG. 53 illustrates a MEF network implementation of inter-layer communicationbetween Data Link Layer devices and Application Layer devices according to anotherembodiment of the present invention; FIG. 54 illustrates a block flow diagram for a TCP Window sizing method according toan embodiment of the present invention; FIG. 55 illustrates a block flow diagram for a TCP Window sizing method according toanother embodiment of the present invention; FIG. 56 illustrates a network diagram including for traffic shaping a network includinga BRAS and a DSLAM according to an embodiment of the present invention; 15 FIG. 57 illustrates a user interface for network traffic shaping method according to an embodiment of the present invention; FIG. 58 illustrates and embodiment of a Data Link Layer device and an ASIC device that is associated with an incoming network interface for communicating to an outgoing 5 network interface; FIG. 59 illustrates a block flow diagram for the traffic shaping method according to anembodiment of the present invention; FIG. 60 illustrates a block flow diagram for a method of for using informationcontained in a PIP packet to control packet traffic flow with UDP; 10 FIG. 61 is an example of an Ethernet network in accordance with an illustrative embodiment of the present invention; FIG. 62 is an example of an Ethernet network in accordance with an illustrativeembodiment of the present invention; FIG. 63 is an example of a CAC engine conf iguration in accordance with an illustrative 15 embodiment of the present invention; FIG. 64 is an example of PIP packet flow of network perfonnance information inaccordance with an illustrative embodiment of the present invention; FIG. 65 is an example of stored network performance information associated withaccess nodes in accordance with an illustrative embodiment of the present invention; 20 FIG. 66 is a flowchart of a process for allocating network resources in accordance with an illustrative embodiment of the present invention; and FIG. 67 is a flowchart of a process for correcting failure of network resources inaccordance with an illustrative embodiment of the present invention.

DETAILED DESCRIPTION OF THE DRAWINGS 25 FIG. 3 is an illustration of an exemplary packet network 300 that utilizes performance information packets 302a, 302b (collectively 302) communicated in-band, and along virtualpacket paths between represented as node links 303a, 303b, and 303c (collectively 303)between network nodes 304a, 304b, and 304c (collectively 304). For purposes of thisapplication, a performance information packet (a “PIP packet” or “PIP data packet”) shall 30 mean a packet communicated over data paths of a data packet network that is used by the datapacket network to obtain performance information associated with path transmission states ofthe data packet network. In one embodiment, such PIP packets are communicated in-band 16 along the data or bearer path of a packet network. However, such PIP packet information may also be communicated out-of-band between network elements of the packet network to provide utilization performance measures to other switching and control systems via control signaling over an Operational Support Network or other operations or maintenance network. 5 A PIP packet may be communicated between the nodes of a network to establish

windows of time in which a node collects or determines network performance information,which may be any information that describes packet utilization and performance of a node,node segment, transmission path, or network element. More particularly, a PIP packet mayhave a timestamp, counter, sequence number or other identifiers to enable the use of the PIP 10 packet by a network node to establish a sampling window of time for collecting or determiningsuch network performance information. Alternatively, a PIP packet may not include suchidentifier and may instead be generated at regular intervals between nodes of the network.Each network node or path transmission point may transmit PIP packets to a far-end elementvia the packet transmission path and the far-end element may receive, calculate performance 15 and store the information for use as a utilization and performance measurement or networkperformance information. Given each communication path may contain information from itstransmission to receive paths, the end-points may exchange and track the measures of the bi-directional path via relaying those measures at either given intervals or any other mechanismso that at least one end of the communication path has both the transmit and receive path 20 utilization and performance measurements. The PIP packet may provide a “heartbeat”between network nodes by which the network nodes may use to determine the networkperformance. A PIP packet may also be used to communicate the collected or determinednetwork performance information between nodes of the network by, for example, including thenetwork performance information in the header or payload portion of the PIP packet In one 25 embodiment, a PIP packet is an Ethernet Connectivity Fault Management (CFM) packet, suchas an 802.1 AG packet, and a receiving utilization and performance tracking mechanism maybe a ITU Y.1731 protocol stack. However, any packet of data may be utilized under anysuitable protocol, as well as calculation methodology to track and store the networkperformance information. 30 The PIP packets 302 may be formatted to include any information capable of providing information to network nodes to determine network performance information, where thenetwork performance information may include transmission rate, transmission quality, and/or 17 transmission connectivity. Other network performance information, such as communication path utilization, may additionally be determined.

The PIP data packets 302 provide a “heartbeat” to enable the network nodes 304 at the far-end (i.e., receiving end) to generate network performance information. The PIP data 5 packets 302 may be communicated every Tpn> seconds, where ΤΡίρ may be less than, equal to,or higher than one second and include a timestamp of the time of communication and a countervalue indicating the number of packets previously sent to enable a receiving network node todetermine whether any data packets were lost between successive PIP data packets.Transmission rate is an indication of the number of data packets communicated over a time 10 period and can be determined by counting data packets communicated over a network segmentduring a time period, for example. The PIP data packets may be used in determining a timeperiod over which to measure the transmission rate. Transmission quality is a measure of linkstate and may include various link state parameters, including packet loss, jitter, and delay, forexample. In one embodiment, the PIP data packets 302 may be communicated over Layer 2 of 15 the OSI model and the network performance information may be determined in Layer 2. Thenetwork nodes 304 may include transmission performance collection units 306a, 306b, and306c (collectively 306), respectively, to generate and collect network performance information.Transmission connectivity is an indication of communications between two devices on anetwork. The connectivity may be indicative of signal strength of communications between 20 the devices, an on/off indication of a device being off or otherwise incapable ofcommunicating, or other suitable performance measures. The transmission performance units306 may generate network performance information in association with the PIP data packets302.

Network performance information may also include information associated with non- 25 packet networks, such as cable DOCSIS, wireless CDMA, TDMA, WiMax, or circuit basednetworks. Without limiting the foregoing, network performance information may include anydata that is associated with any wired or wireless network that may be indicative of orotherwise suitable for use in determining the operation or general health of the network.Network performance information may also include information that is not associated with the 30 communication of data between network elements along connection paths, but is insteadassociated with the performance of network devices themselves located at a particular node ofthe network. For example, network performance information may indicate buffer utilization 18 levels, buffer overflows, errors experienced in caching or queuing data, latency introduced by alack of processing, packet loss across a switching fabric of a particular network device such asa switch and router, or any other performance issue associated with a particular networkdevice. It should be understood that different network types and different connection paths 5 may have different indicia of network performance issues. For example, a Tl line may nothave data associated with packet loss, jitter, or latency but may instead only present alarms ofred, yellow, or green associated with the perceived performance along a Tl connection path.Similarly, there may be a large amount of data associated with wireless networks that areindicative of network performance such as signal to noise ratio, levels of interference, signal 10 strength, or any other suitable data regarding the performance of the wireless network that maybe useful and utilized as network performance information.

Continuing with FIG. 3, in addition to generating network performance informationbased on PIP data packets, the principles of the present invention provide for the networknodes 304 to determine real-time transmission rate or real-time traffic utilization (i.e., a 15 number of data packets including real-time content being communicated over networksegments during a time period or, mathematically described, real-time bandwidth use may bedetermined by tracking the summation of the size of each real-time packet that is transmitted ina given time period, collectively. Alternatively, tracking the real-time packets transmissionrate = number of real-time packets * average packet size / given time period). Real-time 20 content is data produced by applications that use real-time and near real-time data packetcommunications (e.g., VoIP telephone calls). Data packets including real-time content (i.e.,real-time data packets) may include payload dam 308 representative of speech during atelephone call, video data stream during a live event, music stream during a live concert or liveradio broadcast, or gaming data, possibly including embedded voice, during a competitive live 25 gaming session, for example. Non-real-time data packets may include payload datarepresentative of content that does not need to be communicated in real-time (e.g., musicdownload, webpage content, program update download, etc.). Total bandwidth transmissionrate or total transmission rate may also be determined so that if the real-time transmission rateis known, then the non-real-time transmission rate is also known. 30 Determining bandwidth usage, both real-time and total bandwidth usage, can be accomplished by tracking either individual data packets and packet flows or internal networkelement traffic statistics. Collecting the amount of real-time or total bandwidth usage may be 19 performed in a number of ways, including examining a priority bit marking (‘P’ bit), type ofservice (TOS) bit marking, virtual local area network Class of Service (COS) marking, IPaddress and/or port. Additionally, probes, queuing, scheduler, bus, or path metrics may also beused with any other information associated with a data packet that is capable of indicating 5 whether one or more data packets are real-time or non-real-time may be used in collecting real-time and non-real-time bandwidth usage. For example, accessing other in-use protocol stacksvia probes or “cross stack” communication can provide information from real-time controlprotocols, such as Real Time Protocol (RTP) and Real Time Control Protocol (RTCP). Real-time protocol packets may be used to identify real-time bandwidth rate of communication 10 sessions data packets. By determining bandwidth of real-time and total data packets, andoptionally other PIP information, a call control manager 310 may manage networkcommunications sessions in a more intelligent manner. Determining transmission rates may beperformed at Layer 1 or Layer 2 of the OSI model. However, it should be understood thatdetermining the network performance information may be performed on a different layer if 15 information is available on the different layers to provide enough information to determinebandwidth utilization of real-time and total data packets being communicated over a nodesegment. These segments may be a shared path resource as in a media gateway to mediagateway path, pin-hole firewall access node path, or they could be to a single subscriber end-point or intermediate trunking point. It is understood that multiple communications devices 20 share the same transmission path and no single session controller may have knowledge of thein-use real-time data packet counts or bandwidth suite without this information being derivedfrom the use of PIP packets.

Continuing with FIG. 3, the transmission performance collection units 306 may includeone or more modules to determine the network performance information, both with respect to 25 the communication of real-time content and non-real-time content, number of real-timesessions, packet loss rate, jitter, delay, etc. The module(s) may be in the form of softwareexecuted by one or more processors, hardware (e.g., ASIC chips), external probes, firmware, ora combination of hardware and software, as understood in the art. The modules may beconfigured to count the number and size of total data packets and bandwidth of real-time data

30 packets and non-real-time data packets that are being communicated over a node segment (i.e.,one or more communications links between two network nodes or connections, processes, orcomponents within a network node), also referred to herein as a network segment. A 20 communications path may include one or more node segments. Counts may include data packets and real-time packets with and/or without error rate. It should be understood that counting non-real-time data packets is equivalent to counting real-time data packets and total bandwidth because real-time data packets may be determined by subtracting non-real-time data 5 packets from total data packets and bandwidth. In one embodiment, multiple channels may beutilized to communicate real-time data packets and non-real-time data packets along differentpaths through one or more devices and communications paths. Channels may include virtualor physical paths constructed of ports, buses, schedulers, shift registers, cards, and chips usedto transfer or move packets through the device. Real-time packet flows may be separated by 10 assigning ports, markings, size, type, and/or other sorting and scheduling methods to mapspecific traffic to a specific route or path. Using different channels for real-time and non-real-time data packets may enable counting real-time data packets and non-real-time data packetsfaster than having to analyze information contained within the data packets (e.g., P-bit in datapacket header). Alternatively, real-time and non-real-time ports may be configured at a 15 network node to monitor and measure real-time and non-real-time data packets, transmissiontimes, or given path or resource utilization. Real-time and non-real-time bandwidth utilizationcan also be measured by the amount of time the resource is not in use by multiplying thetransmission rate of the path or resource. In addition to measuring the amount of real-time andnon-real-time bandwidth utilization, a second measurement to .characterize the burst nature 20 (burstihess) of the data flows may be performed. When multiple packet flows of differentpacketization rates and different bandwidth utilization rates are combined, an average and peakutilization occurs. One example of a measurement to characterize the burstiness of combinedreal-time flows includes using standard deviation of the peak from the average calculation.Other mathematical methods may be applied to characterize this ability to over-subscribe the 25 real-time flows based on fluxuation in real-time bandwidth usage during the sampling windowthat calculates average bandwidth use. This added measure of burstiness can be usedoptionally with the real-time bandwidth usage. Because PIP data packets 302 effectivelyoperate as a clock on an asynchronous network, the transmission performance collection units306 may monitor and/or inspect the PIP data packets 302 to determine rates of total data 30 packets and bandwidth, real-time data packets and bandwidth, and non-real-time data packetsbeing communicated over a network segment. 21 FIG. 4 is an illustration of an exemplary data packet stream 400 including PIP data packets 402 and data packets 404a-404n (collectively 404), the latter including a packet payload of either real-time or non-real-time content. Each of the data packets 404 may include a header portion including a destination address 406a and origination address 406b, among 5 other header information, and a content or packet payload portion 406c that includes either thereal-time or non-real-time data along with other transmission characteristics. Although only asingle data packet 404a is shown between successive PIP data packets 402a and 402b, asunderstood in the art, there may be many data packets 404 that are communicated betweensuccessive PIP data packets. By determining a total number of data packets and packet size 10 both real-time and non-real-time, communicated during a time duration (e.g., 1 second) and anumber of real-time data packets communicated during that time duration, bandwidth of thetotal number of data packets and real-time data packets communicated over a node link may bedetermined. Alternatively, the amount of time a communications path or resource is not in atransmission state may be utilized to determine bandwidth and data packets communicated. 15 Additional information, such as the distribution, burstiness, or timing of the real-time flows,may also be made available within the PIP packets 402. Together, the real-time and non-real-time packets may be used in conjunction with the link capacity to calculate the averageutilization over the interval. The bandwidth determination may be performed by monitoringthe PIP data packets 402 that are collected to indicate that the time period has completed or 20 inspecting the timestamps contained within each of the PIP data packets 402, which may bemore accurate. Monitoring the PIP packets 402 may include monitoring one or more PIPpackets 402. Performance calculation modules may track utilization and performance ofcommunications paths and node segments, and create historical performance and utilizationmeasure logs. Collected performance information may be used to detect threshold crossings to 25 be communicated to session controllers, as further described herein. Other networkperformance information may be determined by monitoring the PIP data packets 402,including jitter, packet loss, and delay, as the PIP data packets 402 may include informationindicative of time sent and counter value indicative of number of data packets sent between theprevious and current PIP packet. The network performance information may further be 30 categorized as real-time, non-real-time, and/or total network performance information (seeTABLE I). In one embodiment, intermediate levels may also be established, such as near-real-time, higher priority non-real-time, etc. 22 FIG. 5 is a block diagram of an exemplary network node 500 configured to perform functionality in accordance with the principles of the present invention. The network node may include a processing unit 502 that includes one or more processors that execute software 504. In one embodiment, the software 504 may include module(s) that operate as a 5 transmission performance collection function to collect network performance information. Theprocessors 502 may be in communication with a memory 506 configured to store information,such as network performance information, in registers or one or more tables in memory, asunderstood in the art. The processors 502 may further be in communication with aninput/output (I/O) unit 508 that is configured to communicate over one or more 10 communications networks. I/O unit 508 may include: one or more access ports (not shown).The processors 502 may also be in communication with a storage unit that is configured tostore one or more data repositories (e.g., databiises) that store network performanceinformation. The storage unit may work in conjunction with the memory 506. These memoryregisters are sometimes referred to as bins. The network node 500 may be one of a wide 15 variety of network nodes, including the Maintenance-End-Points (MEPs) and Maintenance-Intermediate-Points (MIPs) of a Maintenance Entity Group (MEG). The MEPs may includeaccess node devices, such as a digital subscriber line (DSL) modem, or Cable Modem and itscorresponding Access node DSLAM or Cable Management Termination System (CMTS).Mobile data element, SIP phone, Video On Demand (VOD) Server or a Media Gateway (MG) 20 device, and/or network-to-network interface (NNI), for example. Additionally, the MEPs mayinclude user network interfaces integrated access devices (IADs), session initiation protocol(SIP) devices, or other end-user devices or customer premises equipment (CPE). The MIPsmay include bridges, switches, and routers, for example.

In one embodiment, the memory 506 may store network performance information in 25 bins over a short period of time, such as seconds or minutes, and the storage unit 510 may storehistorical network performance information for longer periods of time, such as hours, days,weeks, or longer periods of time. By storing recent network performance information, remotenetwork nodes (e.g., call control manager, and resource allocation systems and software) maypoll the network node 500 for the network performance information and receive the network 30 performance information in a relatively short period of time as compared to the network node500 having to access the network performance information from the storage unit 510. Periodicupdates may be retrieved via polling, event driven on a regular time basis or during unit 23 initiation or power off, or trigger driven events may also be utilized to transmit the network performance information. The network performance information may include network performance information indicative of data packets including real-time and non-real-time content. TABLE I is an exemplary table that describes an example of network performance 5 information as associated with real-time and non-real-time data packets. Although notillustrated, such data may be identified for communication in each direction over a particularnode segment.

Characteristic Real-Time Measure To tai/Average Max Packet Count 877 34749 Avg Packet Count 852 32833 Active use time .87 30005 Peakedness characteristic 128 200 Time Period Is Is Number ofRT sessions 156 187 Max Bandwidth (Kbps) 877 34.75 Avg Bandwidth (Kbps) 852 32.83 Packet Loss 37 241 Jitter .004s .006s Delay .020s .028s TABLE I. Real-Time and Non-Real-Time Network performance information 10 Although the time period is shown as 1 second, it should be understood that any time period may be utilized to collect the network performance information. Multiple tables or binsmay be used to tabulate different time periods, such as 15 minute, 1 hour, 1 day, and so forth,may be managed for storing the same, different, or additional network performanceinformation and, optionally, over different periods of time. In one embodiment, historical 15 network performance information may be stored in a database to enable a call control managerthe ability to predict usage of the network node 500 during an upcoming time period (e.g., next5 second, next 2 minutes, next day, etc.). For example, if the network node is over-subscribedwith users who perform real-time data downloads during the 7pm - 9pm timeframe, then thecall control manager may utilize that information to route certain calls to other network nodes 20 that have less utilization during that timeframe. In another example, real-time and non-real-time files may be stored on a Video On Demand (VOD) server, in which case actual real-timeuse information could be used to load balance system requests. Many other call and networkcontrol functions may be employed by knowing the real-time and total data packet network 24 performance information. Other statistical analysis uses of this data are possible. Another near real-time use is a graphical presentation of this data in Operator Network Management

Systems (NMS). FIG. 6 is a block diagram of the software 504 of FIG. 5 and exemplary modules 5 configured to determine and collect network performance information in accordance with theprinciples of the present invention. In collecting the network performance information, oneembodiment of the network node 500 may include IEEE 802.1AG and ITU-T Y.173I modules602 and 604, respectively, to generate and receive IEEE 802.1AG data packets and determinenetwork performance information associated therewith. The ITU-T Y.1731 module 604 may 10 be a modified ITU-T Y.1731 function that is configured to collect network performanceinformation associated with data packets containing both real-time and non-real-time content(see, for example, TABLE I). The modified ITU-T Y.1731 module 604 may be configured tocollect performance information, such as maximum and average bandwidth for both real-timeand total data packets along with other transmission characteristics that are being received 15 and/or transmitted from the network node. One or more modules 606 may be configured tostore and communicate collected transmission performiince data. As described with respect toFIG. 5, the transmission performance data may be stored in memory and/or storage unit in oneor more data repositories, such as database(s) or table(s). Communication of the collectedtransmission performance data may be triggered by event threshold crossings or pulled by 20 another network system, network node, Element Management Systems, or call controlmanager, for example, and be performed on a routine basis or in response to a poll, audit, orevent (e.g., dropping below a transmission quality threshold). In addition, although 802.1AGand ITU-T Y.1731 standards are presented for generating PIP data packets and collectingnetwork performance information, the principles of the present invention may use other 25 standards and protocols for collecting network performance information for real-time and totaldata packets being communicated over a node segment. FIG. 7 is an illustration of multiple exemplar}/ data packet networks 700a and 700b(collectively 700) having exemplary network nodes configured within the networks 700 todetermine and collect network performance information. The data packet network 700a 30 includes a media gateway 702, router 704, bridge 706, and network-to-network interface (NN1)708. Other network nodes, such as a session border controller, switch, firewall, computer,satellite, service point or broadband node gateway (VOD server, IP service point, end-point), 25 CPE (customer premises equipment), wireless handset, or any other packet service network node may be configured in accordance with the principles of the present invention. More specifically, each of these network nodes may be configured with modules, such as the modules described with respect to FIG. 6, which produce PIP data packets and collect network 5 performance information for both real-time and total data packets communicated over thenetwork. In one embodiment, a network node, such as router 704, may collect andcommunicate the network performance information in the form of data packets 710 viacommunication link 712 to a call control manager 714. The communication of the networkperformance information may be communicated to the call control manager 714 in response to 10 a poll from the CCM 714, periodically, or in response to an event (e.g., packet loss droppingbelow a predetermined percentage). Because the CCM 714 communicates with multiplenetwork nodes, the CCM 714 may be configured to route calls based on the networkperformance information collected from the network nodes.

The PIP data packets that are communicated present an opportunity for network 15 performance information to be communicated along a network path. In one embodiment,network performance information may be collected and. appended to or otherwise inserted intoPIP data packets so that other network nodes can monitor performance along a virtual path.For example, the network performance information may be collected by a network node, stored ' in a database, and a summary may be appended with a PIP data packet and communicated to 20 another network node to which the PIP data packet is communicated. This concatenationprocess may be performed at regular intervals such as every 5 minutes, hourly, or daily, tominimize the amount of data communicated over the data packet network and stored atnetwork nodes. The CCM 714 may collect and store historical network performanceinformation and utilize such information to monilor trends in the network 700a and 25 automatically alter network operation or enable network operators to reconfigure the network.For example, if it is detennined that the paths to a network node or the network node itself isnot operating properly, the CCM 714 may choose or establish a virtual path through differentnetwork nodes or different paths through the same node than would otherwise not have beenestablished. As another example, if data packets are being lost, the CCM may choose to force 30 existing and new call sessions to use CODECs of a lower compression rate on the nodesegment to alleviate congestion and improve call connectivity. 26 FIGS. 8-9 are block diagrams that more specifically describe structural and functionaloperations of network communications devices, (i) end or mid-point devices (FIG. 8), (ii)firewall device (FIG. 9), and head-end device (FIG. 10). FIG. 8 is a block diagram of an exemplary end or mid-point device or network5 communications device 800 showing structural and functional operations used for employingthe principles of the present invention. The device 800 includes a network packet port or end-point IP trunk 802 that is configured to transmit and receive real-time traffic, such as VoIP,video, RTCP and other real-time data packets 804, via one or more data ports 806a-806c(collectively 806). Each of the data ports 806 may have one or more communications lines 10 808a-808c (collectively 808). Network counters 810 may operate at the network packet port 802 to count data packets including real-time content and total data packets. It should beunderstood that the network packet port 802 may receive data packets including both real-timeand non-real-time content and that data packets containing real-time content should not bedelayed any more than is proper to allow for real-time communications (e.g., telephone calls) 15 to occur without an end-user noticing the delay. A remote monitoring function 812 that isassociated with the network packet port 802 enables various network monitors and consolesystems to exchange network-monitoring data, in the exemplary case, 802.1AG data. A set ofnetwork-side counter functions 814, which may be executed on one or more processors or be ahardware device (modules), operate to count and determine a number of real-time data packets 20 and total data packets (i.e., number of data packet having real-time and non-real-time content),and determine bandwidth for the data packets including real-time content and bandwidth oftotal data packets over a time period. The network side-counter functions 814 may beperformed on each port, line, and/or entity and generate the network performance information(e.g., packet count and bandwidth) for both transmitted and received data packets. 25 Performance collection engines are understood to be located at the same end-point 812 thattransmits the PIP information to the far-end. However, the performance collection enginesreceive a PIP stream from the far-end for the purpose of measurement and collection. A call control module 816 connected between the network packet port 802 and a plainold telephone system (POTS) port 818 on a line side of the network communications device 30 800 may operate to control and manage call connections for a network communications device 800. The call control module 816 may include one or more CODECs using real-time transportprotocol (RTP) and/or real time control protocol (RTCP). The network-side counter functions 27 814 may be in communication with the call control module 816 and generate packet count andbandwidth information based on the data packets being handled by the call control module816. As understood in the art, CODEC stacks may be any session control protocol based oroperate in-band, such as Internet Group Management Protocol (IMGP) for video multicasting, 5 and may have knowledge of the type of call being set-up. Such a configuration also applies topin-hole firewalls, media gateways, call controllers, and bandwidth reservation protocolsystems that use control stacks, bandwidth reservation or resource reservation functionality. Itis understood that a network-side counter function may also contain a pseudo utilizationcounter that indicates bandwidth reservation use information to be passed along with real 10 bandwidth utilization performance and utilization information. The pseudo or reserved useprovides an indication of bandwidth use for calls that are being setup, on hold, or simply beingreserved. The reservation information may be used in providing the session controllers withthe information collected or determined at the network segment. The CODEC stacks and real-time protocols can be used to track correlated per session bandwidth use and report total use to 15 the network side counter function. A few example techniques for counting data packets anddetermining bandwidth of data packets including real-1ime data packets and total data packetsare provided below: (i) bandwidth use via reading CODEC settings and packetization rates, (a) in use,and (b) reserved 20 (ii) if IGMP is used for video, (a) track total number of streams - real time, and (b) track individual stream use (bandwidth varies) via CODEC and packetization rates.

Additionally, any network EMS or protocol stack provisioning engine maycommunicate with the counters 814 and 826 so as to monitor call shaping at the networkcommunications device 800. Real-time performance engines and probes may also 25 communicate with control stacks. Client interacting control stacks may enable a user to selectfunctions to perform in user devices. In summary, these applications may be used on (i)virtual, logical, and physical ports, (ii) firewalls, (iii) traffic shaping instances, (iv) serviceagents, (v) network elements, and element schedulers, (vi) and/or service points, such asgateways, VOD servers, and conference bridges for example. 30 The network communications device 800 may further include one or more line-side packet ports 820a-820n (collectively 820) that are configured to communicate data packets.

The line-side packet ports 820 may include respective remote monitors (RMONs) 822a-822n 28 (collectively 822), PIP generators, and performance collection elements (not shown), that areused for exchanging network monitoring data with other network devices and counters 824a-824n (collectively 824). Similar to the network side, the line-side may include a set of line-side counter functions 826 that count packets and determine bandwidth of data packets 5 containing real-time content and total data packets. A call control module, CODEC stack,probe/sniffer interface 828 may be connected to a network-side port and line-side port. Thecall control module probe 828 may be in communication with both the network-side counterfunctions 814 and line-side counter functions 826 to count data packets and determinebandwidth of data packets being communicated between a network-side port and line-side port. 10 FIG. 9 is a block diagram of an exemplary pin-hole firewall device 900 showing structural and functional operations used for employing the principles of the present invention.The pin-hole firewall device 900 includes multiple physical and logical ports 902a and 902b(collectively 902) through which data packet communication sessions are passed via a sharedtrunk or Ethernet Virtual Circuits (EVCs) 904a and 904b (collectively 904). The data packets 15 (not shown) communicated over the EVCs 904 are communicated through counter andcomputation functions 906a and 906b (collectively 906) that are configured to count datapackets including real-time and non-real-time content and determine bandwidth associatedwith each. In determining bandwidth, PIP data packets, may be used as a clock. The counterand computation functions 906 may be disposed prior to a pin-hole firewall function 908 that 20 operates by allowing an application to take control of a port during a communication session.The counter and computation functions 906 have respective flow counters 910a and 910b(collectively 910) that operate to individually count real-time data packets and total datapackets. In addition, the flow counters 910 may perform the computations for bandwidth foreach port. A call control module probe/sniffer interface 912 may be in communication with 25 the flow counters 910 and pin-hole firewall function 908 to provide input into the flowcounters 910 based on call control protocols 914a and 914b, respectively. The counterfunctions store the utilization and perfonnance management information as a measuredresource for both the network element itself, and the session controllers with which the pin-hole firewall may communicate. Communication of the performance and utilization 30 information can be with but not limited to, a session controller, bandwidth reservation system,network management system, higher layer IP protocols, and other communication systems and 29 systems software. Further description of the operation of collecting and communicatingnetwork performance information from the pin-hole firewall device 900 is provided in FIG. 11. FIG. 10 is a block diagram of an exemplary access node device 1000 showingstructural and functional operations used for employing the principles of the present invention.

5 The access node device 1000 may be configured as a head-end device (e.g., video distributionsystem), cable modem termination system (CMTS), digital subscriber line access multiplexor(DSLAM), or digital line concentrator (DLC), for example. The access node device 1000 mayinclude a network port 1002 and access ports 1004a-1004n (collectively 1004) that areconnected to communication lines 1006a-1006n (collectively 1006). VoIP, video, and RTCP 10 or other real-time data packets, for example, may be communicated over the accesstransmission paths or communications lines 1006. Remote monitors 1008a-1008n arerespectively associated with each of the access ports 1004 and may operate to receive 802.1AGdata packets or otherwise and include PIP packets and performance collection-functions. Alsoassociated with the access ports 104, counters lOlOa-lOlOb (collectively 1010) may be 15 configured to count total data packets and data packets containing real-time content beingreceived via respective communication lines 1006. It is understood that some of the real-timeinformation flows at this node may be available via control protocol implementations, such asVideo IGMP, in which case the control protocol stack or other counter mechanisms can beutilized to measure and report real-time traffic to the PIP generators. It is also understood that 20 other real-time streams may exist that are not under the control of a protocols stack at this nodein which case the real-time and total bandwidth characteristics may be measured via methodsdescribed herein. A remote monitor 1012, which may include a PIP packet generator andperformance collector, may be associated with network port 1002 and be used to generate andcommunicate 802.1AG data packets having network performance information generated by a 25 statistics engine 1014. The statistics engine 1014 may be configured to generate statisticsassociated with the data being communicated from the access ports 1004 to a communicationsnetwork (not shown) via the network port 1002. The statistics may include raw data andmathematically computed data, including (i) real-time bandwidth, (ii) total bandwidth, (iii)provisioned bandwidth, (iv) historical real-time bandwidth, and (v) averages, such as average 30 real-time bandwidth and average total bandwidth. Indicia representative of the statistics maybe communicated to the remote monitor 1012 and communicated to other network devices towhich the access node 1000 is in communication by appending the indicia to PIP data packets, 30 for example, or otherwise communicate the statistics in separate data packets. The indicia may be in any form, such as XML, of a communication protocol being communicated from the RMON 1012 and/or network port 1002. Threshold crossings may trigger special messages to other systems, as well as systems polling the measured resource pool inside the statistics 5 engine 1014. Further description of the operation of collecting and communicating networkperformance information ofthe access node device 1000 is provided in FIG. 11. A management function 1016 may be in communication with the statistics engine 1014and include a number of management functions, including, but not limited to, resourcereservation protocol (RSVP), call admission control (CAC), and provisioning. Session 10 controllers 1018, which may be external from the access node 1000, may be a resource accesscontrol system or facility (RACS or RACF) that operates as a security system and providesaccess control and auditing functionality. The session controllers 1018 may communicate withthe management function 1016 to monitor operation of the access node 1000 by collecting thestatistics generated by the statistics engine 1014. 15 FIG. 11 is a block diagram of exemplary modules 1100 configured to determine network performance information associated with data packets communicated with networkcommunication devices described in FIGS. 8-10. The modules 1100 may be configured assoftware, hardware, firmware, or combination thereof and be generalized into the standards fordifferent network communications devices to ensure consistency throughout one or more 20 networks. The modules may include a probe module 1102, CODEC stack module 1104,statistics module 1106, and statistics modified operations, administration, and management(OAM) module 1108. The probe module 1102 operates using counters to count various typesof data packets, including (a) total packets and/or tolal bandwidth, (b) marked packets, (c)packets provided special bandwidth treatment, and (d) real-time protocol flows (e) or other. 25 The marked packets may be counted based on specifics within the packet such as (i) specifictype of service (ToS) level markings contained in the IP header, (ii) specific packets in anEthernet virtual channel (EVC) or Class of Service markings (COS), (iii) P-bit marking used in802.IQ tags, (iv) differentiated services (Diffserv) field (IP header), or any other suitable basis.The packets provided special bandwidth treatment may include: (i) specific packets in high 30 priority schedules (hardware specific), and (ii) being treated by a QoS engine (such asDiffserv). The investigation of real-time protocol flows may include (i) RTP & RTCP reading(observing the real use in the header), and (ii) other in-band protocols that can be read in terms 31 of use (e.g., CODEC, profile, bandwidth, call reservation, etc.). The counters may be pushed, polled or be accessed by the statistics module 1106.

The CODEC stack module 1104 may include real-time counters and be influenced byoperation of the CODEC. Real-time data packet counters may count when the CODEC stack 5 is (a) in use and/or (b) reserved or (c) other. Counters may also track IGMP use for video tocount (a) number of real-time streams and/or (b) individual stream use, where the bandwidthmay vary. Further, traffic shaping by a provisioning engine may be factored by the CODECstack module. Real-time performance engines that interwork with control stacks may alsoaffect how the count of the CODEC stack module operates to collect network performance 10 information. Client interacting control stacks that allow the user to choose functions to controlspecific applications may be tracked.

The statistics module 1106 may operate to collect statistics or performance informationof data packets including real-time content (e.g., VoIP data stream) and total data packetscommunicated to and from the network communications device. In one embodiment, the 15 statistics module 1106 is a modified Y.1731 application. The statistics module 1106 mayoperate to collect (1) transmitted statistics, (2) received statistics, and (3) individual countersfor each port, line, or entity. The statistics module may perform mathematical calculation onthe collected information. The transmitted statistics may determine, for example: (a) totalbandwidth based on all the data packets that are communicated over a time period, where the 20 time period may be determined from PIP data packets generated by the networkcommunications device, (b) bandwidth of the real-time: data packets communicated during thetime period, (c) real-time data packets counted, and (d) other collected or processedinformation. The received data packet statistics may include the same as the transmittedstatistics, including (a) total bandwidth of all the data packets counted, (b) real-time bandwidth 25 of the real-time data packets, and (c) real-time data packets counted, and/or (d) other collectedor processed information. It should be understood that other statistics, including averagevalues, maximum values, trended prediction values, or other suitable measurements orcalculations that could be beneficial to other network communications devices or soft-switches(e.g„ call control manager) may be collected. 30 The modified OAM module may include any of the following functions: (1) sequence counting to ensure that data packets are being received in the proper sequence, (2) appending asequence flag to 802.1AG packets, (3) adding a name for a circuit leg or segment, (4) including 32 indicia in data packets to identify a carrier collecting network performance information, and (5)communicating collected transmission performance statistics. The communication of thecollected network performance statistics may be performed in response to a poll from anothernetwork device (e.g., call control manager), in response to an event, or periodically. Such 5 communication may be accomplished by communicating the network performance informationor statistics by appending or otherwise including the information with other data packets. Forexample, the network performance information may be added to payloads of data packets thatare being communicated to other network nodes. FIG. 12 is an illustration of an exemplary process 1200 performed on network nodes in 10 a data packet network. The process 1200 starts at step 1202, where data packets containingreal-time content (e.g., from a telephone call) being communicated over a network nodesegment of a packet network are monitored. Monitoring the data packets may includedetermining that the data packets include real-time content in the payload by examining theheader for the P~bit or otherwise determining that the content includes real-time content. At 15 step 1204, at least one item of network performance information associated with data packetscontaining real-time content communicated over the network node segment of the packetnetwork may be determined. The network performiince may include real-time bandwidthusage, total bandwidth usage, packet loss, delay, and/or jitter, for example. While suchnetwork performance information is typically used for determining operation of a node

20 segment, other performance characteristics, such as packet rate and bandwidth, may also becollected for real-time and total data packets. At step 1206, indicia representative of the atleast one item of network performance information regarding the communicated data packetscontaining real-time content may be communicated to a network element, such as a soft switchor call control manager. The indicia may be in the form of a Call Control protocol, SNMP 25 message, XML protocol, HTML protocol, or any other protocol utilized by a data packet of acommunications standard.

DISTRIBUTED TRANSMISSION PERFORMANCE TABLES IN NETWORKNODES FIG. 13 is an illustration of an exemplary network node 1300 configured to perform 30 functionality and communications over a packet network in accordance with the principles ofthe present invention. The network node or network communications device 1300 may includea processor 1302 that executes software 1304 to perform operations for the network node 1300. 33

The network node 1300 may be a router, switch, media gateway, or other networkcommunications device, and include software that performs any function associated withtypical operations of a network node. The processor 1302 may be in communication with thememory 1306. The memory 1306 may store a table that includes network performance 5 information associated with node segments over which the network node 1300 communicates.The processor 1302 may further be in communication with an I/O unit 1310 and storage unit1312. The I/O unit 1310 may be utilized to communicate data packets 1314, including contentdata packets 1314a and PIP data packets 1314b over node segments 1316a-1316n (collectively1316) to other network nodes 1318a-1318n (collectively 1318). 10 In one embodiment, the PIP data packets 1314b may include network performance information of respective network communications devices 1318 describing transmissionperformance over respective node segments 1316. In addition, each of the networkcommunications devices 1318 may store the network performance information describing thenode segments to which each is in communication (e.g., network node 1300). The network 15 performance information may be communicated to the network node 1300 in each PIP datapacket 1314b, periodically (e.g., every 100th PIP data packet, once per second, every 5 minutes,etc.), in response to an event (e.g., in response to a network performance information valuecrossing a threshold value), or in response to a poll or request from the network node 1300, forexample. The quantity and types of performance information contained in each PIP packet 20 could vary between successive communications. For example, derived or summarizedinformation may be communicated on five minute intervals, and other information may becommunicated in other intervals.

As previously described, the software 1304 may be configured to generate networkperformance information associated with node segments between network nodes. The table 25 1308 may include network performance information associated with the network node 1300 and network nodes 1318 with which the network node 1300 communicates. Although the tableis shown to be stored in memory 1308, it should be understood that the table 1308 may bestored in storage unit 1312 in that the memory and storage unit, for the purposes of theprinciples of the present invention, are both considered to be memory. It should further be 30 understood that the term table 1308 is generally descriptive of data stored in a definedarrangement of fields and is descriptive of any organized data set, such as a database or data 34 file containing data fields. The term table is also inclusive of multiple tables that areassociated with one another. TABLE II provides an exemplary table including network performance information.The table may include segment numbers or other alpha-numeric indicia, name of associated 5 segment, and network performance information in one or both directions for each networksegment or path (e.g., east-to-west, west-to-east). The network performance information mayinclude additional and/or other information representative of transmission characteristics (e.g.,transmission rate and bandwidth) along each bearer path. Although not shown, the networkperformance information associated with data packets including real-time and non-real-time 10 content (see, for example, TABLE IV). TABLE II may also include node segments associatedwith the internal performance of a network node or network device. For example, if aparticular network node is a network switch, the operation of such network switch may impactnetwork performance and thereby may have its own network performance information. Forexample, a network node, such as a switch or router, may itself cause packet loss or introduce a 15 delay in the delivery of packets. Thus, identifying particular elements or processes within anetwork node may be useful in monitoring, reporting, compensating for, troubleshooting, orotherwise reacting to problems in network performance. More particularly as illustrated inTABLE II, specific buffers or queues may be identified such as buffer / queue A correspondingto segment number 5. For example, a particular buffer within a network device may overflow 20 resulting in lost data packets at that node and correspondingly an underflow event at adownstream device. Similarly, a particular processor 13 within a network device is illustratedin TABLE II as being associated with segment number 6. For example, a processor may notbe able to keep up with processes required to switch or route packets over a particular network.In fact, many entities are experiencing problems with processor performance as networks 25 become more utilized, Likewise, the internal switching fabric of a particular network devicesuch as a switch or router may also impact network performance as data packets are required tobe switched or routed by such device. Such a fabric is illustrated in TABLE II as beingassociated with segment 7. For example, both packet loss and delay can be introduced by theperformance of a switch. 30 Although not illustrated herein, other software, processes, processors, memory components, or any other component of a particular network node or device that may impactnetwork performance may be included in a table, such as TABLE II, based on data associated 35 with the performance of such components. Likewise, although not illustrated in TABLE II,many other types of network performance information may be included relative to anyparticular node segment. For example, a wireless switch on a wireless network may haveupwards of twenty or thirty factors that influence network performance. For example, 5 interference, signal-to-noise ratio, signal strength, and battery or power level may all affectnetwork performance and may all be represented in a table such as the one illustrated inTABLE II. Likewise, switches, hubs, bridges, or other interfaces between networks, portionsof a network, or various media of communications may instead or additionally storeinformation such as alarms, notifications, signal characteristics, or any other suitable type of 10 data capable of being used to evaluate network performance. For example, with regard to asimple Tl connection, the only information available regarding such connection may be theexistence of a red, yellow, or green indication or alarm. Likewise, a DSLAM device may havevery different information relevant to network performance than the information availablefrom a core IP router. All of the foregoing information is considered network performance 15 information for purposes of this application and may be incorporated in any table, bin,database, or PIP packet described herein.

Network Segment Status Table Segment Name East to West West to East Jitte r Delay Packet Loss Jitter Dela y Packet Loss 1 Bearer Path 1 2 Bearer Path 2 3 Bearer Path3 4 Bearer Path 4 5 Buffer/Queue A 6 Processor B 7 Fabric C TABLE II. Network Segment Status Table TABLE III is another exemplary network segment status table that includes counter and 20 timestamp. The counter and timestamp may be used to determine the network performance information. For example, the receiving node (e.g., node 1318a) may use the counter to 36 determine the total number of data packets that were communicated over the node segment ornetwork segment. If, for example, the counter indicates that 200 data packets werecommunicated since the previous PIP data packet, then the receiving node may determine howmany data packets were received to determine if any data packets were lost. For example, if 5 the counter indicates that 200 data packets were communicated and the receiving nodedetermines that 182 data packets were received, then 18 data packets were lost. Also, the delaymay be determined by recording the time that the PIP data packet is received and subtracting itfrom the PIP data packet timestamp.

Node Segment Dir. Node 1 Node 2 Delay Jitter Packet Loss Counter PIP Data Packet Timestamp (hh.mm.ss.dd.mm.yy) 1316a East 1300 1318a .04 .002 23 234723 03.18.43.12.07.07 1316a East 1300 1318a .05 .003 18 234923 03.18.44.12.07.07 1316n East 1300 1318n .03 .001 3 74832 03.22.17.12.07.07 1316n East J300 1318n .06 .002 42 75832 03.22.18.12.07.07 10 TABLE ΠΙ. Network Segment Status Table

The network performance information shown in TABLES II and III ore representativeof PIP data packets associated with unidentified content type. However, in accordance withthe principles of the present invention, the network performance information may bedetermined with respect to communications of data packets including real-time content and 15 non-real-time content. TABLE IV includes network performance information thatdistinguishes real-time content and total content (i.e., real-time and non-real-time). Althoughthe direction is shown as going east in both of TABLES III and IV, it should be understoodthat west direction network performance information may also be included in the table. Byproviding network performance information specifically related to real-time data packets, an 20 understanding of how the packet network is operating for different content types can bedetermined. In addition, these tables can provide network performance information for one-way, bi-directional unicast or multicast traffic flows. 37

Node Segment Dir. Total BW Total Delay Total Jitter Total Packet Loss RT BW RT Delay RT Jitter RT Packet Loss RT Counter PIP Data Pi Timestamp (hh.mm.ss.dd.mn 1316a East 1.54 .04 .002 23 1.25 .03 ..002 12 234723 03.18.43.12.07.0' 1316a East 1.62 .05 .003 18 1.27 .07 .001 9 234923 03.18.44.12.07.0' 1316n East 1.52 .03 .001 3 0.7 .05 .002 3 74832 03.22.17.12.07.0' 1316n East 2.25 .06 .002 42 1.85 .04 .002 40 75832 03.22.18.12.07.0' TABLE IV. Network Segment Status Table — Total and Real-Time

Continuing with FIG. 13, the software 1304 executed by the processor 1302 of thenetwork node 1300 may examine the network performance information in the table anddetermine whether any of the network performance information parameters cross a threshold 5 value. For example, one or more threshold values may be established for real-time bandwidthuse or real-time packet loss or total packet loss. Thresholds could include a derived‘watermark,* such as a condition that warrants network operator consideration, but is notcritical (e.g., a ‘yellow’ alarm condition) or ‘watermarks’ indicating a moving peak during adefined time-window. If the software determines that the packet loss or other data included in 10 the network performance information associated with a node segment crosses above thethreshold value, then a call control manager module may be notified to change a networkcomponent (e.g., slow down a CODEC) or re-route current and/or future calls from that nodesegment. Alternatively, if it is determined that the total bandwidth is high while the real-timebandwidth is also high (e.g., node segment 1316n at time 03.22.18.12.07.07), then the call 15 control manager may initiate processes to slow down or halt the set up of new data streamscontaining non-real-time or real-time content until the real-time content demand decreases orinitiate a disconnect based upon some criteria, such a priority bit markings or otherwise. A callcontrol manager or individual node may alter a device (e.g., slow down a CODEC) orcommunications (e.g., change modulation) to attempt to improve transmission performance. It 20 should be understood that other applications may be derived from monitoring the networkperformance information contained in the table(s). FIG. 14 is a flow chart of an exemplary process 1400 for managing networkcommunications. The process 1400 may start at step 1402 by communicating first data packetsvia at least two node segments on a packet network to at least two network communications 25 devices. Second data packets communicated from tlie network communications devices via respective node segments may be received at step 1404. The second data packets may include network performance information generated by the network communications devices in38 response to receiving the first data packets. The second data packets may be utilized toexchange transmitted and received performance information between network segment end-points. In one embodiment, the first data packets are PIP data packets. In addition, in oneembodiment, the second data packets are PIP data packets. At step 1406, a table containing 5 network performance information associated with the node segments over which the seconddata packets are communicated may be stored. At step 1408, the second data packets may beparsed to access the network performance information, wherein parsing includes reading thecontent contained in the second data packets. In one embodiment, field identifiers included inthe packet may define the start and end of a transmission performance parameter and may be 10 read to access the value of the transmission performance parameter. At step 1410, the networkperformance information may be stored in the table. Communications over the node segmentsmay be altered based on the stored network performance information at step 1412. Thecommunications may be current or future communications.

PEER-TO-PEER DISTRIBUTED CALL CONTROL USING DISTRIBUTED TABLES 15 The network performance information that is stored in tables at each node may be used by the nodes to make network control or routing decisions when placing or routing calls.These decisions may be based on the network performance information that is indicative ofnetwork performance at a node segment associated with a node or other node segments forwhich the node has access in tables stored therein. The decisions may include employing 20 congestion avoidance processes as understood in the art. Routing decisions, such as routing acall via a different transmission path to a called party, may be performed at the node to avoidcongestion or other transmission problem at a node segment. Still yet, the node may determinethat packet loss is high so the node may negotiate a lower CODEC bandwidth for on-going,and new sessions with another node prior to or during a telephone call in an effort to minimize 25 packet loss between the nodes. It should be understood that the node making the decisionsmay include customer premise equipment, such as a SIP telephone, or any other node within apacket network, including wireless access points, DSL modems, and/or cable modem devicesthat suffer variable bandwidth availability. The distributed call control, in essence, mayinclude the same or similar functionality as may be performed by a call control manager. 39

CENTRALIZED NETWORK PERFORMANCE INFORMATION TABLE FIG. 15 is an illustration of an exemplary packet network 1500 having a call controlmanager 1502 with a centralized table 1504 of network performance information for use inmanaging call communications over the packet network 1500. The CCM 1502 may include 5 the same or similar hardware as provided in FIG. 5, and execute software configured toperform call control operations for end-users on the packet network 1500. The centralizedtable 1504 may include network performance information generated by networkcommunications devices (e.g., end-points and intennediate points) operating on the packetnetwork 1500 that indicate operation of the node segments. For example, router or switch 10 1506 may collect network performance information for node segments 1508a - 1508f (collectively 1508). As previously described, the network performance information may begenerated through the use of PIP data packets being communicated over the node segments. Itshould be understood that node segments may traverse from end-to-end and includeintermediate points so that an overall packet network communications path can be described. 15 This path could contain one or more communication technologies and protocols, such asEthernet, SONET, IP, and ATM. For example, a node segment may extend from network-to-network interface or session border controller 1510 to end-user 1512 to describe transmissionperformance for bearer paths 1508d, 1508a, and 1508e and network communications devicesbetween the end-points, including router 1506 and network access node 1514. Generation of 20 the performance information may use standard or modified protocols, such as the IEEE802.1AG protocol, to generate the information as associated with the PIP data packets. Theperformance information may be gathered and stored using a modified standard protocol, suchas Y.1731, to include data packets containing real-time content and total data packets (i.e.,real-time and non-real-time content) so that the CCM 1502 may make call management 25 decisions based on the type of calls or sessions that are being placed or currently operating onthe packet network 1500.

Collection of the network performance information may occur at regular intervals(e.g., every second, minute, hour, four hours, day, week, month, or otherwise). Collection ofthe network performance information at regular intervals, especially shorter intervals, may add 30 overhead to the CCM 1502 and network communications devices, so other collection schemes may be utilized for communication of the network perfo rmance information to the CCM, such as event and request driven collection schemes. Event driven communications of the network 40 performance information may occur if a network communications device (e.g., media gateway1516) determines that call quality has degraded below a predetermined threshold. Forexample, if jitter of real-time data packets increases above a predetermined threshold value, thenetwork communications device may communicate cunent network performance information 5 to the CCM 1502 for storage in the centralized table 1504. Alternatively, a message or alertmay be communicated to the CCM 1502 to notify the CCM 1502 of the node segmentproblem, which may cause the CCM 1502 to store a value indicative of a problem to beincluded in the centralized table. Request driven communications may be performed by theCCM 1502 to send a poll or request to each of the network communications devices to 10 communicate current and/or historical network performance information generated and/orcollected by respective network communications devices. The centralized table 1504 mayinclude the same, similar, and/or additional information as described with respect to TABLESMV.

The table 1504 may be used by various algorithms, thresholding events, or processes to 15 determine routing changes, CODEC usage choice, or other call related functions that coincidewith obtaining suitable or the best call quality using the network available. As provided inTABLE Π, the network performance information may include transmission quality parameters(e.g., real-time bandwidth, jitter, delay, packet loss) in a duplex fashion (e.g., east to west andwest to east). The values stored in the table may include derived data and actual raw data 20 generated at the network communications devices, scaled data representative of the raw data(e.g., scale between 1 and 10 with 1 being the optimum capability and 10 being the worstcapability or vice-versa), or Indicia (e.g., grade rankings A-F) representative of the quality ofthe raw data. It should be understood that virtually any captured or derived data representativeof the network performance information may be stored in the table 1504 that provides the 25 CCM 1502 with the ability to manage calls on the packet network 1500.

In using the network performance information in the centralized table 1504, callprocessing within the CCM 1502 may use the network performance information for calls beingset up. To accomplish this, the CCM 1502 may determine the route taken for the differentcalls based on location of the end-points within the packet network 1500 of the service 30 provider. Since the CCM 1502 has end-point information recorded in conventionalprovisioning tables, segment information may be added to these provisioning tables that wouldprovide information on how the bearer path would traverse the packet network 1500. This 41 information would be added to both line and trunk provisioning tables within the CCM 1502. TABLES V and VI show possible configurations of conventional provisioning tables extended to include segment information as collected by the CCM 1502 from network communications devices. 5

Line Information Table Line Name Line Number Existing LineInformation 1st Seg 2nd Seg N01 Seg End-User 1 NPA-NXX- 1234 ... I 3 N/A End-User 2 NPA-NXX- 0987 ... 2 3 N/A * » . • - . • . · • · · • · · End-User N NPA-NXX- 0298 ... 1 7 N/A TABLE V. Line Information Table

Trunk Group Information Table Trunk GroupNumber Location ExistingTrunk GroupInformation lsl Seg 2nd Seg N01 Seg XXYYYY (Phys Location) 1 * . I 3 N/A xxzzzz (Phys Location) *». 2 3 N/A ... ... ... ... ... .. . TABLE VI. Trunk Group Information Table

As shown in TABLES V and VI, network performance information or, as shown,representative values of the actual network performance information for each segment may be 10 stored with the associated lines and/or trunk groups. The segment information may beconfigured as shown above with a given set of values that corresponds to the node segments,such as the bearer paths 1508, network communications devices, or combination thereof, usedin the transport of the call through the packet network 1500. The values could also beprovisioned in a vector, using commas to delineate the segments. That is, the segments used in 15 calls to/from the chosen end-point may be shown as X, Y, Z, AA, etc. No matter how thetables are provisioned, the network performance information or summary thereof may beutilized by call processing functions of the CCM 1502.

More specifically, to provide call control based on the lower layer status of theunderlying packet network 1500, the CCM 1502 may employ a mechanism that queries the 20 segment status table for each call. Algorithms may be used to access raw network42 performance information stored in the table and convert the raw data into a value that can be used for call processing. These algorithms may take different forms, such as determining the highest value of the network performance information columns (e.g., east-to-west delay, jitter,

packet loss or west-to-east delay, jitter, real-time packet bandwidth, packet loss (see TABLE 5 I)) and using the highest value as the status value of the segment. If using these transmission performance characteristics, three value range scales could be enacted that to signify (i)whether the segment is running normally, (ii) if there is slight impairment, or (iii) if thesegment is too congested for added traffic. The three value range scale could be set at theCCM 1502 in an overall provisioning table that is commonly used in soft-switch development. 10 For example, if the results of the algorithm are in a 1-10 scale, where 1 is the best availabilityand 10 is the worst availability, the “normal” availability could be a range of 1-3, impairedavailability could have a range of 4-7, and congested availability could have a range of 8-10.Alternatively, the status may be defined using other indicia, such as colors (e.g., green, yellow,red) or letters (e.g., A, B, C). 15 FIG. 16 is a flow chart of an exemplary process 1600 for using network performance information stored in a centralized table for controlling calls by a call control manager. Theprocess starts at step 1602. At step 1604, node segment values generated from a callprocessing module may be accessed from a table. In one embodiment, the table is centralizedand located at a call control manager located on a network and include network performance 20 information for each of the packet network communications devices located on the packetnetwork. Alternatively, the table may be located at a network communications device locatedon a packet network. In another embodiment, the table may be a distributed table, such asthose stored on network communications devices and the network performance informationmay be accessed when needed. At step 1606, value range information may be requested from 25 an office parameter, where an office parameter determines what values are considered“normal,” “impaired,” or “congested.” Value range parameters may be defined by nodesegment status value ranges: normal: a-b, impaired: c-d, and congested: e-f, where a-f areintegers ranging from 1 to 10. At step 1608, the value ranges may be received and a nodesegment status value may be compared to range values. At step 1612, a determination may be 30 made if the node segment status value is greater than 0 and less than “c” of the “impaired”segment status value range. If so, then at step 1614, a “normal” status is set back to the callprocessing module. The process ends at step 1616. If at step 1612, the node segment status 43 value is determined to have a value of “c” or greater, then a determination may be made at step 1618 as to whether the node segment status value is within the “impaired” range or “congested range.” If “impaired,” then an “impaired” status may be sent back to the call processing module. Otherwise, a “congested” status is sent back to the call processing module. The 5 process 1600 may be used for each node segment over which the CCM may route a call or overwhich a call is currently routed. If an “impaired” or “congested” determination is returned bythe process 1600, the call processing module may elect to select a different route or re-route anongoing call or otherwise. FIG. 17A is a flow chart of a high-level process 1700 for centrally controlling 10 communications on a packet network. The process 1700 starts at step 1702 by communicatingwith multiple network communications devices over a packet network. In one embodiment, acall control manager is communicating with the network communications devices. At step1704, network performance information of node segments providing communications betweenthe network communications devices on the packet network is stored in association with 15 indicia representative of the node segments. The indicia may be alpha-numeric and describe aphysical location or logical address associated with the node segments. It should beunderstood that the node segments may refer to an individual network communications devicefor describing transmission path performance through the device itself, transmission lines, orcombination of devices and lines. In another embodiment, node segments may be an aggregate 20 of two or more segments between two end-points. Network communications by the networkcommunications devices over the node segments may be controlled based on the networkperformance information at step 1706. The network communications may be controlled in avariety of ways, including re-routing calls, changing call routes during a call, and/or changingoperation of network communications devices (e.g., reducing bandwidth of a CODEC).

25 PACKET NETWORK DIAGNOSTICS

In addition to being able to re-route calls in the event of a determination being madeof a network node or segment being impaired or congested, the principles of the presentinvention provide for packet network diagnostics to be made manually, semi-automatical ly, orautomatically based on network perfonnance information collected from one or more network 30 nodes. In one embodiment, network performance information may be collected and aparameter may be monitored. One or more threshold values may be established for use in 44 determining that the network performance information parameter. For example, an upper and lower threshold may be established to ensure that transmission rate of data packets including non-real-time content so that a customer does not receive higher or lower transmission speed than contracted. In one embodiment, the network performance information may be collected 5 and monitored at a central location on a packet network. Alternatively, each individualnetwork node may monitor itself and other network nodes on the packet network and beconfigured to initiate the diagnostics.

If a determination is made that a threshold is crossed by the parameter beingmonitored, then diagnostics may be initiated. The diagnostics may include a wide range of 10 functions, including initiating a loop-back test, trace route, modified trace route, ping, orotherwise, as understood in the art. Additionally, a command may be issued to a network nodeat which a network performance information parameter that crosses a threshold is associated toinitiate a diagnostics routine and return a result from the diagnostics routine. For example, asoftware routine may be executed for the network node to execute one or more self-tests 15 associated with data packet communications at the network node. It should be understood thatthe diagnostics may be initiated to monitor network nodes, segments, gateways, or any othernetwork communications device. In addition, the network performance information from asecond packet network owned by another communications carrier may be monitored by anoperator of the packet network and diagnostics may be performed, but return limited result 20 information to avoid sharing confidential information, for example.

Still yet, if the network performance information parameter determined to cross athreshold is associated with communications of data packets including real-time content ornon-real-time content, the diagnostics may be directed to determining whether a problem existswith communications of data packets including real-time or non-real-time content individually 25 depending on the problem that exists. Other diagnostics as understood in the art may beinitiated in response to the type of network performance information parameter determined tocross a threshold.

RESTRICTING SHARED ACCESS TO TABLES FIG. 17B is one embodiment of a permission table that may be utilized to establish 30 permission, state, or access levels by various network participants to network performanceinformation that has been collected over one or more networks. Such network performance 45 information may be stored in a PIP packet, in a table, bin, or other memory structure at anetwork node or access point, or at a central network, or inter network resource, such as anoverall network performance table, or a table used by a CCM, NOC, or EMS system.

More particularly, a permission table is illustrated as Table 17b0. Table 17B0 may 5 include fields associated with an entity identifier 17b2, a segment identifier 17b4, and one ormore network performance information identifiers 17b6.

Entity identifier 17b2 may be an identifier associated with an individual networkparticipant, such as a subscriber, network operator, VPN provider, or other network participant.Alternatively, entity identifier 17b2 may be an identifier associated with a group or category of 10 network participants 17bl0. More particularly, such an identifier may identify a class ofparticipants in a network, such as a subscriber group, a network operator group, a VPNprovider group, or any other suitable category. For example, there may be an identifierassociated with the operator of the particular network or networks regarding which networkperformance information is stored in Table 17b0 such that administrative personnel, devices, or 15 processes of such network operator may have full access to all of the information inTable 17b0. Alternatively, a group of network operators who are the operators of othernetworks in communication with the network that is the subject of the table illustrated as 17b0may be given restricted, i.e., a much lower degree of, access to network performanceinformation stored in Table 17b0. 20 Segment identifier 17b4 may be utilized to identify a particular network segment, such as a connection path between two network elements, a network element itself, or a particularprocess or component of a network element. Thus, an individual network segment may beidentified as illustrated as 17bl2 while a category of network segments, such as network-to-network interfaces (NNI), may be identified as illustrated relative to 17b22. Similarly, an 25 entire network may be identified collectively to represent all network segments located withinsuch network as illustrated relative to 17b24. Additionally, a particular network path throughthe network including all network segments located along such path may be identified such asis illustrated relative to 17b26. Similarly, a category of paths may be utilized as an identifier,such as, for example, all paths between the CPEs of a particular customer or group of 30 customers, may be identified as illustrated relative to 17b28. Likewise, portions of pathsincluding perhaps only those network segments between a customer’s CPE and a network 46 access point may be identified to give the customer or a network provider access to line stateinformation for such customer as illustrated relative to 17b30. NPI identifier 17b6 may identify different categories of network performanceinformation that are available to be accessed by a particular network participant regarding a 5 particular network segment or category of either of the foregoing. For example, individualitems of network performance information, such as real-time and total bandwidth usage,packet, jitter, and latency, may be identified as illustraied relative to 17b 14. Alternatively, anidentifier representing full access to all available network performance information may beidentified as illustrated relative to 17bl6. Alternatively, NPI identifier 17b6 may be utilized to 10 differentiate between categories of packets that are communicated across a network. Forexample, identifiers may be utilized to differentiate access between overall networkperformance information and more targeted network performance information, such asnetwork performance information associated with real-time data packets. Any combination ofthe foregoing may also be utilized. For example, an identifier may be provided that allows full 15 access to real-time network performance information.

Although only one example of a portion of a permission table is illustrated inFIG. 17B, any combination of entity identifiers 17b2 associated with individual networkparticipants or groups of network participants may be used in a table or other data structurewith segment identifiers 17b4 of individual network segments, categories of network segments, 20 entire networks, particular connection paths, particular line state information, or anycategorization or grouping of the foregoing, and may be further used with NPI identifier 17b6that are associated with individual categories of network performance information, data types,and other NPI identifiers offering full access or no access at all may be utilized.

The network performance tables may include network performance information on 25 many different levels. For example, the network performance information may be collectedand status values indicative of the operational status or performance of the node segments maybe generated in addition to storing specific network performance information (e.g., packet loss,bit rate, bandwidth, etc.) in the network performance information tables. Rules may beestablished to enable certain users, partners, affiliates, or otherwise, to have access to certain 30 levels of data. The levels of data may be specified in the network performance informationtable and define parameters that each level may access. 47

Additionally, network performance information being communicated via PIP packetsmay also be protected from different entities, nodes, or otherwise, from having access tocertain information. In one embodiment, the network performance information may beencoded or otherwise identified so that the level of the information is specified and, thereby, 5 restricted to be accessed by parties or equipment that do not have permission to access nodesabove a certain level. The levels may range from 1 to 10, for example.

PIP DATA PACKET STITCHING FIG. 18 is a block diagram of exemplary multi-node packet networks 1800a and 1800b(collectively 1800) used to communicate data packets 1802 including PIP packets to convey 10 network performance information generated by each node or network element 1804a-1804n(collectively 1804) in a transmission path. As shown, there are two packet networks 1800a and1800b formed of multiple network elements or network communications devices 1804 thatmay form a network of one or more service providers. Because there are two networks 1800,network-to-network interface devices 1806a and 1806b (collectively 1806) are configured to 15 communicate with one another, thereby forming a bridge 1808 across the packet networks1800a and 1800b. This bridge could include direct connections of the same technology as in1800a and 1800b, or as another technology existing in a different type of network, such asSONET bridging two Ethernet networks, for example.

Each of Network A and B may have its own respective PIP packets associated with

20 each of their respective networks and respective modified Y.1731 protocol stacks or othermeasurement processes monitoring communications between nodes 1804 and 1806. A thirdset of PIP packets and associated Y.1731 protocol stacks may monitor the communicationsbetween the network-to-network interface devices 1806. Real-time data packet performanceand total network performance information may be generated and communicated in the PIP 25 data packets. In one embodiment, a first-end point in the network communications pathgenerates a special PIP packet that routes through devices 1804 and triggering each modifiedY.1731 measurement engine to inject its stored network performance information into the PIPpacket by appending network performance information and a segment identifier into the PIPpackets as it is passed downstream. This PIP packet with the appended network performance 30 information then continues and triggers the same performance measurements (PM) or networkperformance information pull at the next network node and associated modified Y.1731 48 protocol stack inserts its PM information concatenated behind the first nodes PM information.This process continues for each network node such lhat the network node’s node segmentidentifier and network performance information is concatenated into the next PIP packet sentin the downstream path direction. Such PIP packet travels in turn to the subsequent network 5 node and associated Y.1731 protocol stack, where additional network performance informationis appended and a node segment identifier is added and then the combined networkperformance information is transmitted again via a PIP packet until the PIP packet reaches thefar end network node and Y.1731 protocol stack. Each Y.1731 protocol stack along the pathcan read the appended segment PM information and store the network performance 10 information or optionally choose to terminate the PIP segment PM information by removingsome or all of the appended data so the next PIP packet sent downstream contains only thatsegment’s information, or simply add its own network performance information into the PIPpacket. If a network node or associated modified Y.1731 stack protocol terminates the PIPpacket, the network performance information is stored, and a new PIP packet that does not 15 include the stored network performance information is communicated to start stitching newnetwork performance information from network nodes and associated modified Y.1731protocol stacks located downstream in a network. It should be understood that data packetsother than PIP data packets and stacks or processes other than the modified Y.1731 protocolstack may be utilized to communicate the network performance information between the 20 network communications devices. The network communications devices 1804 may beconfigured to communicate the network performance information in the PIP data packets on aregular basis (e.g., every second, minute, hour, 100th PIP data packet), based on an event (e.g.,performance information parameter crossing a threshold value), in response to receiving a PIPdata packet with network performance information contained in the payload portion, or in 25 response to a request or command. The PEP data packets 1802 may be 802.1AG data packetsand be communicated over OSI Layer 2.

To correlate the performance manager measurements on a transmission path, thenetwork performance information measured may use multiple bins to store data collected overa period of time. Multiple bins can concurrently exist for different time window lengths, such 30 as 5 minutes, 10, minutes or otherwise. For example, special “bin roll” PIP packets orsequentially number packets may indicate which timeframe bin the information should bestored in. For example, PIP sequence or bin packets may be generated at time periods that 49 include 1, 5, and 15 minutes. The modified Y.1731 performance measurement function canhave multiple bins that correlate to the PIP sequencing or flags to ensure the PIP packet data isstored in the correct bin. Also, longer time periods may be added to the modified Y.1731stack, including 1 hour and 24 hours. The modified Y.1731 stack or counters may be accessed 5 to derive or compute the network performance information. Access may be performed byusing time period numbering.

To provide isolated segment performance information, each network communicationsdevice 1804 may be configured to receive a PIP data packet 1802 containing networkperformance information from other network communications devices and append network 10 performance information generated at the respective network communication device with theother network performance information within the PIP data packet 1802. This concatenationprocess may be considered “stitching” of network performance information along atransmission path. This function helps identify path verses shaping functions in thetransmission path, and also provides the ability to retrieve network performance information 15 in-band verses using multiple external EMS systems to retrieve the information for faultisolation. To ensure that errors in the bins do not occur, the PIP data packets 1802 may bemarked (e.g., <15minupdate>) so that each of the network communications devices along acommunications path appends information contained in the Y.1731 bin that is storing networkperformance information associated with the marking (i.e., <I5minupdate>). 20 FIG. 19 is an illustration of an exemplary series of network communications devices 1902a-1902n (collectively 1902) that are configured to append network performanceinformation in PIP data packets I904a-1904n (collectively 1904). Each of networkcommunications devices 1902 may be configured to append most recent and/or historicalnetwork performance information to network performance information received in the PIP data 25 packets 1904 from other network communications devices 1902, as indicated by payloadportion 1906a-1906n of the PIP data packets 1904 increasing in length after each of thenetwork communications devices 1902. Alternatively, such network performance informationmay be otherwise inserted into or represented in PIP data packets 1904. Header portion 1908a-1908n of the PIP data packets 1904 may be configured normally. In practice, the network 30 performance information may be written into the PIP data packets 1904 using XML languageor other language (e.g., <param start tag> param value </param stop tag>). The principles ofthe present invention provides support for both tag delimited and fixed width fields within the 50 performance packet. For example, the following descriptor may include network performanceinformation generated over a time period. <NSEG>A2O4</NSEG><J1TR>.OD2</JITRXDEI>.O4</DEL><PL> 125</PL>... <RTBW> 1.73</R.TBWXTBW>3.74<ZI'BW><riME>07.43.47.14.07.07<ZriMEXNSEG>A205</NSEG> ... 5 Included at the start of the network performance information is an identifier of the node segment “A204.” Additionally, an identifier of a carrier may be added to the PIP data packet,such a carrier name or code. Jitter, delay, packet loss, real-time bandwidth, total bandwidth,and time at which the network performance information was generated may also be included inthe network performance information. This network performance information may be 10 compared with historical network performance information previously sent from node segmentA204 to determine whether a problem has developed over time. As shown, networkperformance information of network segment "A205” is appended to the network performanceinformation “A204.” It should be understood that other embodiments for communicating thenetwork performance information may be utilized. 15 Continuing with FIG. 18, so that the network performance information associated with different node segments can be easily identified, an identifier that describes each node segmentmay be included in the PIP data packet 1802 by positioning the identifier in front of thenetwork performance information generated by each of the network communications device1804. Continuously concatenating the network performance information and communicating 20 the appended network performance information in PIP data packets across a transmission pathfrom end-to-end provides for a complete description of a transmission path with detailedviewing of network performance information associated with each connection, networkelement, media, or other network segment included in such transmission path. Concatenatingnetwork performance information in PIP data packets may also be performed on abbreviated 25 transmission paths to further help isolate a transmission problem. If, for example, atransmission path is having transmission quality problems, an analysis of the networkperformance information collected along each node segment of the transmission path may beperformed to identify the node segment(s) that are contributing to the transmission qualityproblem. For example, the stitching process may be performed on MEPs, MIPs, NN Is, CPEs

30 and be performed routinely or in response to a command issued by a CCM. The last node thatreceives the network performance information may be configured to perform analysis on thenetwork performance information or communicate the information to a CCM, NOC, EMS 51 system, correlation engine, or other network device. If ithe network performance information iscommunicated over multiple networks, a CCM that miinages the last node may communicatethe compiled network performance information to the originating CCM for transmissionperformance analysis. The network performance information may be stored in tables at each 5 of the nodes in the transmission path and/or one or more network devices may receive andstore the network performance information in table(s). It should be understood that someperformance information collected from different types of network nodes, such as those withinthe NNI, may contain performance information that is unlike performance informationcaptured from other nodes such as MIPs or MEPs. The embodiments of this invention allow 10 disparate types of performance information to be concatenated into a single performance flow.

The network communications devices may include performance managers (PMs) thatperform the function of managing the modified Y.1731 stack. The performance manager orother software module may perform the functions of accessing the stack to collect networkperformance information, optionally at particular time periods, concatenating the network 15 performance information into the PIP data packets, and communicating the data packets in-band or out-of-band. The performance managers of the network communications devices maybecome tools that represent correlated performance manager counter usage in-band in shorttime intervals, which results in (i) eliminating in-flight measuring accuracy issues, (ii)eliminating multi-carrier segment troubleshooting, (iii) optionally enabling in-band 20 performance managers for access versus out-of-band enhanced messaging services or graphicaluser interfaces, (iv) and correcting the issue of stacking access technologies that introducemultiple in-line PIP packet flows that have to be polled. FIG. 20 is an exemplary process 2000 for communicating network performanceinformation of a node segment of a packet network. At step 2002, network performance 25 information indicative of transmission characteristics of a node segment on a packet networkmay be generated. The network performance information may be included in a data packet atstep 2004 and communicated using in-band signaling to a network communications device atstep 2006. The network performance information may be appended to other networkperformance information received in a data packet from another network communications 30 device and communicated in turn via a third data packet to another network communicationsdevice. The data packets may be PIP data packets. 52 FIG. 21 is an exemplary process 2100 for identifying communication problems withinone or more packet networks. At step 2102, network performance information may begenerated at network communications devices in communication with one or more packetnetworks. At a first network communications device, a first data packet including first network 5 performance information generated at the first network communications device may begenerated at step 2104. At step 2106, the First data packet including the first networkperformance information from the first network communications device may be communicatedto a second network communications device. At the second network communications device, asecond data packet including the first network performiince information received from the first 10 network communications device and second network performance information generated at thesecond network communications device may be generated at step 2108. The second datapacket including the first and second network performance information may be communicatedfrom the second network communications device to a third network communications device atstep 2110. This process of generating, concatenating, and communicating network 15 performance information may start at a first end of a transmission path and finish at the secondend of the transmission path so that each network communications device has providednetwork performance information that may be used to determine where a transmissionperformance problem exists along the transmission path. For example, if bandwidth for real-time applications is being lost at a node segment, a service provider may identify which node 20 segment along the communications path is losing bandwidth for real-time applications. FIG. 22 is an illustration of an exemplary packet network 2200 with one service provider and two operators. The packet network 2200 includes operator network equipment2202a-2202f (collectively 2202) and subscriber equipment 2204a and 2204b (collectively2204). The Metro Ethernet forum has defined Operations, Administration and Maintenance 25 (OAM) Maintenance Entities (MEs) as shown. More specifically, the Metro Ethernet hasdefined multiple administrative domains, such as Subscriber Maintenance Entity (ME) 2206,Test Maintenance Entity, User Network Interface (UNI) 2208a and 2208b (collectively 2208),Operator Maintenance Entity 2210a and 2210b (collectively 2210), and Network-to-NetworkInterface Maintenance Entity (E-NNI ME) 2212. Each operator is provided with visibility 30 across its respective network via the operator MEs 2210, but cannot view information in the

Operator ME of other operators unless the other operators provide the proper permissions to allow this view. 53

In accordance with the principles of the present invention, an OAM domain, shown as aStitched ME 2214, extends between the subscriber equipment 2204 through the transmissionpath of the operator equipment 2202 of both operators. The Stitched ME 2214 provides forcommunication of network performance information generated at each element of subscriber 5 and network equipment in the packet network 2200, and PIP data packets (not shown) may begenerated and communicated from the subscriber equipment 2204a and successively throughthe operator equipment 2202 on the packet network 2200 as a single flow that is stitched orconcatenated together via higher order packets to the subscriber equipment 2204b (upstream todownstream). It should be understood that two flows may be operating in opposite directions 10 since the full duplex nature of some communication technologies allow divergent receive andtransmit paths. Each MEG Intermediate Point (MIP) (set of stitched MEG End Points (MEPs))may transparently append or block and re-start the PIP data packet flow, or selectivelyincluding network performance information. Bins having predetermined time periods (e.g., 5minutes, 15 minutes, 1 hour, 24 hours) may be created to create a “stitched” PIP data packet 15 that pulls network performance information from each of the communications devices(subscriber equipment 2204 and operator equipment 2202) on the packet network 2200 in acorrelated manner. Sequences and counter resets may additionally be adopted for providingthe PIP data packet stitching. To perform the stitching operations, the MIPs may pull themodified Y.1731 information from the upstream node segment and append it to the stitched 20 packet traveling downstream. As previously described, node identifiers and/or carrier namesor codes may be included in the PIP data packets to identify the carrier and node segment thatthe modified Y.1731 performance manager data was inserted. In one embodiment, MIPs, orcertain MIPs, may remain “unstitched** and operate as a pass-through.

Although the Stitched ME domain extends from end-user to end-user, the principles of 25 the present invention may provide the ability for operators to be limited to accessinginformation from their own network or limited information from other service providernetworks. Subscribers, similarly, may be limited to having access to their own equipment or asum of operator information. There may be a number of different techniques used to providesuch limited visibility for operators and subscribers, including safeguards built into 30 performance managers at each network communications device. FIGS. 23A and 23B are illustrations of a multi-carrier network 2300 having multiple

Ethernet service providers (ESPs) 2302a, 2302b, and 2302c (collectively 2302) and a multi- 54 point network 2304 having a multi-point device 2306 in communication with network interfacedevices 2308a-2308d (collectively 2308). Using a stitched PIP packet stream enables an end-user to determine performance of each node segment to determine if equipment operated byone of the service providers 2302 is having a communications problem. In the case of multi- 5 point communications, a transport performance manager may be isolated from the switchingperformance manager. It should be noted that non-Ethernet performance information mayexist and be included in the PIP packet stream. CALL CONTROL MANAGER FUNCTIONALITY ENHANCEMENTTo provide a better experience for end-users, a mechanism is being introduced to 10 provide near real-time monitoring capabilities of the path and link status of the underlyingpacket network upon which a voice-only or multi-media call is carried. This information canbe sent to the CCM and acted upon to choose or alter the call characteristics and routing ofcalls, such as change codec use, provide call treatment and routing, and alter overall use of thecall path, etc., thereby providing a better quality of sendee for the end users. 15 One technique for providing this network monitoring capability is the use of a link state reporting structure in the form of PIP packets. Both line state (i.e., transmission path to a user),and trunk state (i.e., shared transmission path state between network nodes) can be provided tothe CCM to convey the transmission path state of Ihe packet network. The PIP packetsthemselves provide a line or trunk state, respectively, at each end of a line or trunk 20 transmission path. To enable CCM management capabilities, line and trunk states may becommunicated to the CCM via call control protocols or some other type of packet networksignaling. The PIP and PM measurements protocols provide the means to monitor the qualityof the link states and report findings to a separate network element. As previously described,included in this near real-time report may be real-time bandwidth usage, packet loss, latency, 25 and jitter or any other network performance information. The monitoring of this informationcan happen within any area of a network and can provide a means to report the lower layerstatus of the network. As shown in FIG. 15, there are many places that these measurementscan be taken. The PIP packets provide the information used to determine path capabilitiesfrom the network end points. Bearer path monitoring may be accomplished between the 30 following elements of FIG. 15:

End User 1512 and Network Access Node 1514., or optionally to router 1506End User 1518 and Network Access Node 1514., or optionally to router 1506 55

Network Access Node 1514 and the Network Router 1506

Network Router 1506 and Network-Network Interface 1510, or optionally to a Media

Gateway deployed in another carrier’s network

Network Router 1506 and Media Gateway 1520 5 Network Router 1506 and Media Gateway 1516 PIP packets may also provide information belween two end devices even though a network element is located between the two end devices. That is, if the provider would want tosee the overall “health” of the path between media gateway 1520 and media gateway 1516, thePIP packets can be configured to monitor this route even though the router 1506 is part of the 10 routing of this path. Once collected, the raw information from these paths can be configuredto show the overall health of the route. Information contained within the PIP packets may beused to determine metrics, such as real-time bandwidlh usage, jitter, packet loss and overalldelay, of the path being measured. These calculations may be performed at the individualelement, or information may be transported to another collection device to be used by other 15 call processing functions as shown in the CCM 1502. These real-time events may be used toprovide input into the decision functions used in call routing within the CCM. The measure ofreal-time bandwidth being provided by the PIP packets also enables a summation of the real-time bandwidth on that path. Historically, this metric is part of a TDM CCM function, but wasnot replicable or available without both the number and amount of real-time bandwidth usage 20 on a node segment. These combined functions provide for such a measure, thereby the CCMmay contain a table of the amount of “Erlangs” being used on trunking facilities. Other timeintervals may be used to accommodate other non-Erlang-like measures.

In accordance with the principles of the present invention, new steps are added to thecall processing 1504 of the CCM 1502, over and above the normal call processing currently 25 done. Since the CCM now has the capability to track the lower layer performance andbandwidth availability of the underlying network, a ne w type of status table may be added tocall processing that systematically updates during specific time intervals throughout the day.These updates, which may occur periodically (e.g., once per second) may be placed in one ormore tables (see TABLES II-IV) in a form that may show utilization, latency, jitter, and packet 30 loss. While there are other types of information that could be shown, such as Mean OpinionScore (MOS) voice values, for simplicity, these three basic parameters are illustrated anddiscussed herein. 56

In normal operation, when an end-point, either a trunk or line, initiates a call, callprocessing operating in the CCM 1502 determines the terminating end-point to complete thecall. In conventional calling scenarios, the call would then be set-up and the call pathestablished for the end-users to converse. This is conventional call processing based on 5 provisioned information that would give call processing the ability to route the call. Theprinciples of the present invention take advantage of collecting the network performance andutilization information from the node segments to aid in performing call processing. Callprocessing may perform normal information lookup to determine the originating andterminating end-points of the call, but before routing the call, the end-point or node segment 10 information from these end-points may be retrieved and call processing may query a networksegment status table (e.g. TABLE IV) to determine the line or trunk state availability of thenode segments that could be used to connect the originating and terminating end-points for thecall. Depending on the availability of the paths used on the call, special call handling, loadbalancing, call spacing, or other special call handling cjin be invoked to sustain call processing,

15 and provide relief for the call path or in extreme congestion, alternate routing could take placeto provide for a satisfactory voice path on the packet network. This management can be doneat call set-up and/or anytime during the call. As transmission state is available at both theCCM and call protocol stack at the user location, multiple enhanced call functions may bepossible. For example, outgoing user calls could automatically query the line state on the CPE 20 to provide the user with graphical or text based feedback as to call options for multi-mediasetup given they have a specific transmission quality to utilize. For example, a multi-mediacall could revert to a frozen image and voice-only call until congestion clears. The samecondition may enable the CCM to know the state before setting up a call and either make adecision by itself or query the user or user’s equipment about how to alter ongoing sessions to 25 allow more communications. The line state information availability to the switch and user maybe used to provide session control feedback. The same information of threshold crossings maybe used to convey that a call may be dropped prior to (he incident occurring. These functionsmay have significant value to the customer experience. The trunk state of shared resources isparamount for inter and intra-switch path state knowledge. Packet networks may be 30 considered to operate autonomously given that the bandwidth being used by the CCM is alsobeing used by other services without knowledge by the CCM. To operate appropriately, theCCM may use transmission state feedback so it can be pre-cognitive of the communications 57 path state during call handling. Without the trunk-state information between two switches,each switch operates under the assumption that enough bandwidth exists to sustain all calls.Often, neither switch will “own” the bandwidth flow control mechanisms for flows betweenthe switches, so this assumption is dangerous in terms of providing carrier grade call handling. 5 Conditions can arise in which inadequate bandwidth or device resources are available tosupport all calls and packets are dropped. If the switch lenows the path state (line or trunk), callhandling alternatives may provide customers with feedback that was previously unavailableand provide better call quality and call handling. It should be understood that the CCM mayuse both line and trunk state tables and make call handling and customer call feedback 10 decisions based on the severity of node segment congestion, including, for example: (i)CODEC modification, (ii) rerouting the call, and (iii) congestion control.

CODEC MODIFICATION

In a line-to-line call between two end-users 1512 and 1518 (FIG. 15), alternate routing 15 to the line end-points is not a viable alternative since each communicates via the networkaccess node 1514. Since most end-lines have one path for transport to the packet network,other modifications are performed to provide better call quality. One modification thatprovides a better call capability would be a CODEC change to raise or lower bandwidth of theCODEC (i.e., a CODEC that operates at a different speed). In one embodiment, the bandwidth 20 is raised or lowered by sending a command to the CODEC to raise or lower its bandwidth.Alternatively, a different CODEC may be employed for performing the call. This replacementcould occur in mid-call. FIGS. 24A-24C are flow charts of an exemplary process for performing line-to-line callflow. The process 2400a starts at step 2402, where (be call processor is idle. It should be 25 understood that the call processor may be hardware, software, or a combination thereof. Atstep 2404, an originating line or calling party goes off-hook and dials a number of a calledparty or destination line. This call, in turn, is received by the CCM as an incoming call.Information for this call is passed from the end-unit to the CCM. In IP telephony, the signalingprotocol may be Session Initiation Protocol (SIP), but other signaling protocols, such as Media 30 Gateway Control Protocol (MGCP) or Megaco (H. 248) may be used.

At step 2406, a decision is made if the call is allowed by determining whether the calling party is registered, authorized or otherwise. The CCM, more specifically, retrieves 58 terminating end-point addressing and location based on conventional table lookups within the CCM. At step 2408, a decision is made as to whether the call is allowed. If it is determined that a call is not allowed, then at step 2410, a “reject” message may be sent to the originating line and the process ends at step 2412. If at step 2408, a determination is made that the call is 5 allowed, then at step 2414, routing translations from call control is checked. At step 2416,routing information is found and termination line information is received. In addition, nodesegment assignment information for the originating and terminating lines is retrieved.

At step 2418, a determination as to whether the terminating line is available is made. Ifthe terminating line is not available, then at step 2420, a “reject” message may be sent to the 10 origination line and the process ends at step 2422. If, however, at step 2418 the terminationline is determined to be available, then at step 2424, the node segment status table is accessedto locate usage status of node segments to be used for connecting a call between the originationand termination lines.

At step 2426, node segment state information stored in the node segment status table is 15 received. The information from the node segment status table includes origination lineinformation and termination line state information. Tbe node segment status information isused to determine if a transmission path state to be used for the call has any congestion.Depending on numerical or other indicia status retrieved by the call processing, adetermination of the congestion of the transmission path will be (i) normal or (ii) impaired, or 20 (iii) congested, for example. The determination of the transmission path being normal,impaired, or congested is made based on network performance information having valuesdetermined to be within ranges, where the range may be a single value (e.g., 1, “A,” “normal”).It should be understood that the range may be defined by a single value, such as “Congestion”representing status values between 7 and 10, for example. As previously described, the values 25 may be processed to be within a scale, such as 1-10, iindicative of the status of transmissionperformance.

At step 2428, results from the node segment status table for the originating line isreceived. At step 2430, a determination of a largest node segment status is made.Determination of the largest node segment status is made by determining a highest value of 30 status indicators in the node status segment table associated with the originating lineinformation. Determining the largest node segment status is performed to identify a limitingtransmission parameter (e.g., bandwidth usage, packet loss). As previously described with 59 regard to TABLES V and VI, the larger the value, the worse the network performanceinformation associated with a node segment, thereby resulting in poor voice quality during acall, this information may be provided back to the caller as system feedback. Note, that thelargest value may be a high quality value and is indicative of a well performing network; i.e., 5 all paths are equal and capable of supporting high quality calls. At step 2432, a determinationas to the status of the originating line node segment is made, which, in one embodimentproduces one of three results, normal, impaired, or congested. If normal, the process continuesat step 2434 in FIG. 24B. Alternatively, if the status of the originating line node segment isimpaired, the process continues at step 2436 in FIG. 24C. Still yet, if the status of the original 10 line node segment is congested, then the process continues at step 2438 in FIG. 24C.Continuing with a normal status of the originating line node segment, at step 2440 in FIG. 24B,a call invite set-up message with normal request is sent to the terminating line. Normal callcontrol is continued at step 2442. The process ends at step 2444.

If it is determined at step 2432 that there is an impaired condition, then call processing 15 being performed by the CCM may change the message that would be sent out to theterminating end-point to request a CODEC to use a lower bandwidth for the call path. Thislower bandwidth request may be performed in concert with a user interface or performed viathe user CODECs without user participation. This is shown at step 2452, where CODECcapabilities of the originating and terminating lines are checked and a determination is made at 20 step 2454 as to whether a lower speed CODEC is available. If it is determined that no lowerspeed CODEC is available, then at step 2456, a “reject” message may be sent to the originatingline. The process ends at step 2458. If, however, a determination is made at step 2454 that alower speed CODEC is available, then at step 2460, an invite with a lower speed CODEC maybe sent to the terminating line. At step 2462, a wait may be performed for a subsequent 25 message from the terminating line. At step 2464, a positive response message may be receivedfrom the terminating line. At step 2466, a message is sent with new CODEC information tothe originating line and, at step 2468, normal call control is performed. The process ends atstep 2470. Not shown in this embodiment is that other call set measures could be considered ina serial or parallel fashion by the CCM in addition to CODEC negotiation to establish a quality 30 call.

In summary, FIG. 24C operates to change a message that is sent out to the terminatingend-point to request a lower bandwidth for the call path. For example, if the originating calling 60 party requested use of a G.711 voice CODEC that uses 64 Kb/s for the bandwidth, the callprocessing may change the request to the terminating called party to a G.729 CODEC that usesonly 8 Kb/s. While the voice quality may not be as good as the higher bandwidth CODECoriginally requested, the bandwidth selected may be reduced enough to allow the call to be 5 completed with better voice quality than if it was impaired while using the originally requestedhigher bandwidth CODEC. It should be understood that the line state information may be usedto facilitate customer call setup feedback, and possibly call setup control with CODECselection choice. It is understood that line state can apply to wireless network devicesconnected behind multiple access technologies, where a line state PIP packet may originate at 10 the end-user device and terminate at a specific access node or at some point between the CCMdedicated switch or router. The access node may enable line state transmission path utilizationand performance management tracking. Also, it should be understood that calls could be ofany type, including voice, multi-media, or otherwise, where timely and quality delivery may becall path considerations. 15 To provide the originating caller with a CODEC change, call processing may wait for the return information from the called party to be received. Once the information is receivedfrom the called party, then the call processing may alter the message to include the change to alower bandwidth CODEC and pass it on to the originating party. From this point on, normalcall processing would continue and the call would he set-up with the lower bandwidth 20 CODEC.

If at step 2432, a determination is made that the originating line node segment iscongested, then the process continues at step 2438 in FIG. 24C where a “reject” message maybe sent to the originating line at step 2456 and the process ends at step 2458. The “reject”message is sent because call processing determined 1hat the call could not continue even 25 though a lower bandwidth CODEC could be used. A user notification, such as an audible orvisual ‘Network Busy’ message, may be sent to the calling party. Depending on the severityof the deterioration of the transmission path, the CCM may send out a response to the callingparty request not to allow the call to continue. This “throttling” of calls coming into the packetnetwork provides established calls more bandwidth to use, and the calling party of the rejected 30 call may receive a busy signal. The calling party may place the call at a later time and adetermination may be made at that time as to whether the status of node segments associatedwith the calling party is normal (i.e., status value within a range). In an alternative 61 embodiment, the CCM may automatically continue to regularly attempt to set-up the call.When the congestion clears, the CCM notifies the calling party that a call can now be set-upand completes the call per the calling party instructions.

Continuing with FIG. 24A, steps 2446, 2448 arid 2450 are steps performed in response 5 to receiving terminating line information and minor the steps 2428, 2430, and 2432,respectively. In other words, the process 2400a makes the same or similar determinations onboth the originating line and terminating line to ensure that status of node segments associatedwith each of the calling and called parties is operating properly.

BEST PATH METRICS 10 In determining transmission paths through a packet network, a CCM or other node may make a determination of the transmission path for a call or other communication to be madeover the packet network based on current, historical usage, or network performance of nodesegments on the packet network. In one embodiment, a transmission path to route the call orcommunication may be determined by using network performance information or information 15 derived therefrom (e.g., network segment status information) available in a table or at eachnode along a potential transmission path. In one embodiment, a calculation may be made todetermine metrics along one or more transmission paths through the packet network todetermine that the metrics result in a cumulative value below a threshold or the best metric ofthe potential transmission paths. Currently, most best-path algorithms use total utilization and 20 bandwidth size for determining the quality of the path. In accordance with the principles ofthe present invention, characterization of real-time jitter and delay performance characteristicsmay be used to determine best path metrics. Modification of the best path metrics to includethe real-time usage, and performance enables enhanced load balancing and path choicedecisions for real-time flows. In one embodiment, these real-time network performance 25 information characteristics may have a higher priority on the network. This modified metricenables the network to make enhanced routing decisions for traffic routing that was notpossible without the transmission state or network performance information. One example ofbest route calculations improvement may include averaging, and, optionally, burstinesscharacterization. Best path calculation methods may include calculations, such as root-sum- 30 square (RSS) and weighted vector calculations, that may be utilized to determine the path orpaths with the optimum best path metrics. Further, a weighted average of the networkperformance information or status levels may be determined. In one embodiment, the best path 62 metrics may create a real-time utilization state by which engines, EMS systems, and othernetwork protocols may retrieve and utilize to gain system feedback as to the nature of the real-time network state. Also, a search for a transmission path having lowest sum of status levelsmay be used to determine best path metrics. In response to determining the transmission path 5 with the best metrics, that transmission path may be used for establishing the transmission pathfor a call or communication. REROUTING CALLS USING NETWORK STATUS SEGMENT TABLEWhile calls between two end-users, such as end-users 1512 and 1518 (FIG. 15), on the same network access node 1514 does not allow rerouting of calls between the two end-users, 10 calls from an end-user 1512 through a media gateway 1516 or 1520 or other network interfacedevice may provide additional options for alternate call routing destinations verses altering aCODEC selection. For a call between an end-user and a media gateway or other trunkingdevice, there may be more than one route or termination point to successfully complete thecall. These routing options are often the case with PSTN switching, where an end-office 15 switch (class 5) may have an alternate tandem call termination point to reach that same end-office. That is, if the call is routed to a specific gateway and that route is congested, it may bepossible to locate another media gateway with a route to the destination. By determining atransmission path and using the node segment status table (e.g., TABLES V and VI) on theroute to the terminating media gateway or trunk, call processing could be instructed to 20 determine whether a secondary trunk capability is available for the call and determine if thesecondary trunk has an uncongested path to the destination of the call. This same functionenables geographical fail-over or call routing when network congestion or network failuresignificantly impairs the packet transmission path to a, remotely deployed media gateway. Inaddition, predictive algorithms that trend performance information may recognize that a link is 25 failing and systematically re-route traffic to an optimum link while managing the quantity andquality of the calls.

In a typical line-to-trunk call, the combination of line segment congestion and trunksegment congestion may be taken into account. It should be understood that a network switchmay track all transmission paths to a central point, trunking point to trunking point, hybrid of 30 line to central point, or line to trunk in a transmission state table. Since the end-user initiatesthe call, the first half of the call would use node segment analysis described previously todetermine if the transmission path at the calling node segment is operating properly or has 63 impairment If the calling node segment is found to be impaired, then call processing may determine that a lower bandwidth CODEC may be utilized to improve the call quality or take other steps, such as allow the call to be made as a voice-only call rather than a multi-media call. If the originating node segment is congested, then the call processing may reject the call 5 since there is no other path for the end-user to use. However, if there is a transmission qualityor utilization problem at the terminating trunk node segment, then a rerouting option for thecall may be available. In one embodiment, utilization means real-time utilization as comparedto total bandwidth utilization with packet loss or the statically provisioned bandwidth allottedin that physical or virtual channel. Any indicators can serve to calculate the state of the user’s 10 transmission “line*’ or shared resource “trunk” transmission path. As stated, the CCM cannow have a secondary “state” for that segment, line, or trunk by which it predetermines howcall processing for that end-point should be handled. This secondary state is indicated in tableVII below. TABLE VII includes an exemplary list of scenarios for the call processing to followbased on the combined status of the originating line and the terminating trunk. 15

Scenario Originating Line Terminating Trunk Call Processing 1 Normal Normal Normal 2 Impaired Normal Adjust CODEC or Reroute 3 Congested Normal Reject Call 4 Normal or Impaired Impaired Adjust CODEC or Reroute 5 Normal or Impaired Congestion Reroute

TABLE VII. NETWORK STATUS AND RE-ROUTING CALL OPTIONS

Scenario 1

In this scenario, normal call processing may be used since none of the transmission20 paths are constrained or impaired. The call may be routed without any changes to the voice coding of the call through the transmission path.

Scenario 2

Since the originating line is impaired, call processing may adjust rate of a CODEC for the call The rate adjustment may be performed by lowering the rate of a CODEC or routing 25 the call to another CODEC having a lower rate. Call processing may check the segments of the outgoing trunk to determine if the media gateway on the transmission path has capability to 64

alter the CODEC used to convert the packet information (e.g., IP Packet Information) to aTDM format. If CODEC alteration is possible, then the CCM may negotiate the CODECspeed between the originating call device and the terminating trunk and establish the call viathe CODEC having the lower speed. If the media gateway does not have multiple CODEC 5 speed capability, then the call controller may have the option of routing the call via anothertrunk group if an alternate route to the terminating call device is available. If another routeexists, then the call processor may reroute the call to the next trunk group and the nodesegment status check may be performed prior to establishing the call via the trunk group. If thetrunk group has CODEC modification capabilities, then the call may be established via a 10 CODEC with a lower speed and the call may be established. If another trunk cannot be foundwith CODEC speed alternatives, then the call may be dropped.

Scenario 3

If the originating line is determined to be congested, then since there are no alternativeroutes for the originating part of the call, then a call “reject” may be sent to the user and the 15 call dropped.

Scenario 4

If the terminating side of the call is determined to be impaired, then a determination asto a lower bandwidth CODEC may be used. If the terminating trunk group has the capabilityto use a different CODEC, then a determination as to the CODEC capabilities of the 20 originating line may be performed. If a lower bandwidth CODEC is available, the call may beestablished with these CODECs and the call may proceed normally. If there are no CODECsavailable at the originating side with a lower bandwidth, then the call processing may performa reroute as described in Scenario 2.

Scenario 5 25 If the terminating trunk is determined to be congested, then call processing may search for a reroute for the call over a terminating trunk that is not congested. The call processingmay include locating a trunk group having a normal or lower speed CODEC for establishingthe call. FIGS. 25A-25C (collectively FIG. 25) are flow diagrams of an exemplary process for 30 providing call processing for rerouting a call between an originating line and terminating trunk.The process in FIG. 25 may be performed by a call processor at the CCM 1502 (FIG. 15) or,optionally, other call managers if distributed on the packet network. The process 2500a starts 65 at step 2502. At step 2504, an incoming call is received. In one embodiment, the incomingcall is an SIP invite. At step 2506, a determination may be made if the call is allowed bydetermining if the caller is registered with the service provider. A determination may be madeat step 2508 to determine if the call is allowed. If not, the process continues at step 2510, 5 where a “reject” message is sent to the originating cidl device and the process ends at step2512. If the call is determined to be allowed at step 2508, then the process continues at step2514, where routing translations from the call controller are checked. Terminating trunkinformation is received at step 2516. Additionally, node segment assignment information forthe originating line and terminating trunk group may be received. 10 At step 2518, a determination may be made its to whether a terminating trunk in a transmission path between the originating call device and terminating call device is availablewithin the trunk group. If so, then a node segment status table may be used to determine usagestatus of the node segments along the transmission path. At step 2522, results from the nodesegment status for the originating line may be received. At step 2524, determination of the 15 originating line segment status may be determined. Irx one embodiment, three node segmentstatuses may be determined, including “normal,” “impaired,” and “congested.” At step 2526, adetermination as to the status of the originating node segments may be determined. If it isdetermined that status ofthe originating node segment is normal (i.e., status is within a rangethat provides for normal, full transmission rate operation), then the process continues at step 20 2528 in FIG. 25B. Otherwise, if it is determined that the originating node segment is impaired, the process continues at step 2530. If, however, it is determined that the originating nodesegment is congested, since there are no alternatives, the process continues at step 2532, wherea “reject” message is sent to the originating call device. The process ends at step 2534.

In one embodiment, the line and trunk state checking may become part of the call 25 processing procedure. Typically, Call Admission Control functions are blindly appliedwithout regard to path state to reserve bandwidth. In one embodiment, the line and trunk stateutilization and performance management may be provided as a state to the reservation enginein a switch to validate or accelerate the “CAC approval” verses statically assigning the numberof calls allowed. This modification provides enhanced value given that CAC function assumes 30 a static CODEC utilization and cannot predict the use of silent suppression or unknown real-time use in the transmission path. It should be understood that the CAC function may be part 66 of the CCM or reside outside the CCM on a centralized CAC resource, such as a RSVP or RAC server.

From step 2528 in FIG. 25B, the process continues at step 2536 where the node segment status table is accessed to determine usage status of the terminating trunk. The results 5 from the node segment status table for the terminating trunk are returned at step 2538. Adetermination at step 2540 is made of the node segment status of the terminating trunk. If thedetermination at step 2542 of the status of the terminating segment is normal, then the processcontinues at step 2544, where an invite to establish the call via the terminating trunk isperformed. At step 2546, normal call control is performed and the process ends at step 2548. 10 If at step 2542, a determination is made that the status of the terminating segment is impaired, then at step 2550, a check of CODEC capabilities of the terminating trunk may beperformed. If at step 2552 it is determined that a CODEC having a lower rate is available, thenat step 2554, a check as to the CODEC capabilities of the originating line is performed. Atstep 2556, if the determination is made that a CODEC is available with a lower rate at the 15 originating line, then the process continues at step 2558 to send a set-up request for a lowerspeed CODEC to the terminating trunk. At step 2560, the call processing waits for asubsequent message from the terminating trunk indicating that the terminating trunk has beenable to set-up a CODEC with a lower speed at step 2562. At step 2564, a message is sent tothe originating line with new CODEC information. At step 2566, the call control continues 20 normally and the process ends at step 2568. If at step 2556 a determination is made that noCODECs are available at a lower rate for the originating line, then a “reject” message is sent tothe originating call device at step 2570 and the process ends at step 2572.

If (i) at step 2542 a determination is made that the terminating segment is congested or(ii) at step 2552 that no CODEC is available at the terminating trunk, then the process 25 continues at step 2574, where call processing is checked to determine if there is another routeavailable from the originating call device to the terminating call device via a different trunk.At step 2576, results for a trunk group selection is returned and a determination as to whetheran alternative trunk group is available at step 2578. As understood in the art, a trunk group istwo or more trunks of the same type between two different nodes. If an alternative trunk group 30 is available at step 2578, then at step 2580, a message may be sent to restart terminating trunkprocessing with a new trunk group. The new terminating trunk information is received at step2582. Additionally, node segment assignment information for the alternative terminating trunk 67 group may also be received. The process continues at step 2584, which repeats the processfrom step 2536 using the new terminating trunk for determining whether a call may beestablished via that trunk.

If at step 2578 it is determined that an alternative trunk group is not available, then at 5 step 2586, a “reject” message may be sent to the originating call device. The process ends atstep 2588.

Continuing from step 2526, if a determination that the status of the originating segmentis impaired, then the process continues at step 2530 (FIG. 25C). At step 2590, CODECcapabilities of the originating line are checked. At step 2592, a determination is made as to 10 whether a lower speed CODEC is available. The lower speed CODEC may be programmed tobe lower or be another CODEC that operates at a slower speed or change from multi-media tovoice-only or reduce to voice-only speed. If a lower speed CODEC is not available, then atstep 2594, a “reject” message is sent to the originating call device. The process ends at step2596. 15 If it is determined that a lower speed CODEC is available at step 2592, then at step 2598, the network segment status table is accessed to find usage status of the terminating trunk.At step 25100, results from the network segment status table for the terminating trunk arereturned, and a determination as to the terminating trunk segment status is made at step 25102.At step 25104, a determination is made as to whether the terminating trunk segment status is 20 normal, impaired, or congested. If it is determined that the terminating trunk segment status isno more impaired, then at step 25106, CODEC capabilities of the terminating trunk arechecked. At step 25108, a determination may be performed to determine whether the CODECis available. If a CODEC is available, then at step 25110, a set-up request for a lower speedCODEC may be sent to the terminating trunk. At step 25112, the call processing waits for a 25 subsequent message from terminating trunk until the terminating trunk notifies the callprocessing that the lower speed CODEC is available and ready at step 25114. At step 25116,the new CODEC information may be sent to the originating line. The call control processingmay continue normally at step 25118 and the process ends at step 25120.

If at step 25104 a determination is made that the terminating segment is congested or at 30 step 25108 no CODEC is available at the terminating trunk, then the process continues at step25122 to request from the call processing as to whether there is another route available viaanother trunk. At step 25124, results for another trunk group selection are returned. At step 68 25126, a determination is made as to whether an alternative trunk group is available. If not,then at step 25128, a “reject” message may be sent to the originating call device and theprocess ends at step 25130. If at step 25126 a determination is made that an alternative trunkgroup is available, then at step 25132, a message to restart the terminating trunk processing 5 with a new trunk group is initiated. At step 25134, new terminating trunk group information isreceived along with segment assignment information for the alternative terminating trunkgroup. The process continues at step 25136, which causes the process to use the newterminating trunk group to determine whether a call may be established via that trunk group forthe call by the originating call device to the terminating call device. 10 Continuing at step 2518 of FIG. 25A, if it is determined that no terminating trunk is available within a trunk group, then at step 25138, the call processing checks to determine ifanother trunk route is available. At step 25140, tlie call processing returns informationindicative of the trunk availability. A determination at step 25142 is made as to whether analternative trunk group is available. If an alternative trunk group is available, then at step 15 25144, the call processing may be restarted with the alternative trunk group and the process returns at step 25146 to step 2514 (FIG. 25A).

In summary, the process of FIG. 25 is used to determine status of a transmission pathbetween an originating call device and a terminating call device via a trunk group. Indetermining the status, if the trunk group is having a communication problem as determined by 20 a network segment status table that derives its information from network performanceinformation received from node segments on the packet network, then the call processingdetermines whether it can lower the bandwidth of a CODEC or find an alternative route viaanother trunk group that has better communication performance for routing the call to arequested end-point.

25 ADDITIONAL CALL REROUTING

In one embodiment, the principles of the present invention provide for network performance information to be utilized in rerouting calls to subscribers in the event of a nodesegment being determined to be impaired or congested, or otherwise unavailable for example.In such an event, when a call comes into the CCM, the CCM may use a directory to look up

30 other potential contact’s telephone numbers or addresses to which the incoming call may berouted in an attempt to connect the calling party with the called party. For example, if a callingparty has attempted to reach a called party on his or her mobile handset and the CCM 69 determines that the transmission path to the subscriber’s mobile handset is not workingproperly, then the CCM may locate an alternative number of the called party, such as a homeor work telephone number or other identifier, such as an SIP Universal Resource Identifier, androute the incoming call to the called party’s alternative number or identifier. In one 5 embodiment, the CCM makes the decision as to which number to call based on time of day orother factors (e.g., a subscriber preference parameter).

In another embodiment, the CCM may receive a call to a subscriber that the CCMknows to be on a heavily congested or otherwise degraded node segment. The CCM maymake a decision to place the call directly into a called party’s voicemail rather than tie up the 10 heavily congested or otherwise impaired node segment with additional real-time contentcommunication. Alternatively, the CCM may notify the heavily congested or otherwisedegraded node segment to slow down, halt or otherwise offload non-real-time contentcommunications being communicated through the node segments so that the telephone call,which is a real-time content communication, may be properly and timely placed to the called 15 party.

CONGESTION CONTROL

Calls from trunks, such as the network-to-network interface or session border controller1510 of FIG. 15, to lines over a packet network present different challenges than line to line orline to trunk calls. Since the call control manager does; not have complete control of a packet 20 trunk path being selected for in-coming calls into the soft-switch, congestion control issomewhat limited. If a call enters the soft-switch from another network, trunk selection isactually controlled by the other or far-end network. Tire call control manager, however, mayhave some level of call control utilizing the principles of the present invention.

Generally, when a call comes into the network, call processing operating in the CCM 25 receives an incoming call message with data identifying the port and address of the incomingcall. Based on the port, address, and called number information, the CCM determines thetransmission path, including the node segments, over which the call is assigned. In accordancewith the principles of the present invention, the CCM may examine the status of the nodesegments associated with the transmission path. If the status of the node segment is classified 30 as impaired, call processing may determine if the terminating line has the capability of using alower bandwidth CODEC. If so, then the CCM may send a set-up message to the end-pointrequesting use of the lower bandwidth CODEC. In a return response to the other network, call 70 processing may pass the new request for the lower bandwidth CODEC. If accepted by theother network, the call may proceed. Otherwise, the call is terminated,

For other calls coming into the network via the trunk, the CCM may follow the sameprocess of determining whether a CODEC having a lower bandwidth is available. If a network 5 segment status is congested, call processing may not try to process the incoming call and senda release to the other network via the originating trunk.

In one embodiment, the CCM may manually or automatically enact call throttlingprocedures based on congestion of the originating trunk segment interconnecting to the othernetwork. These throttling procedures may be in the form of automatic congestion control 10 (ACC), selective incoming line control (S1LC), call gapping, number or IP address blocking, orany other well-known throttling call control mechanisms. Based on timers or incoming callcounts, the CCM may allow calls to be attempted at certain times to test the congestion of thepath. If the node segment becomes uncongested, call processing may allow calls to enter thenetwork and throttling mechanisms may be taken off of that path. 15 FIG. 26 is a flow chart of an exemplary process for performing congestion control for calls coming through an IP trunk to a line. The process 2600 starts at step 2602. At step 2604,an incoming call is received on an IP trunk. At step 2606, a determination is made if the call isallowed, where a call may not be allowed if an improper message is received, for example. Atstep 2608, determination of the call being allowed is performed. If the call is not allowed, then 20 a “reject” message is sent to the originating call device at step 2610 and the process ends atstep 2612. If the call is allowed, then the process continues at step 2613, where a request forrouting translations from a call controller is made. At step 2614, the determined routing andterminating line information is received. Additionally, node segment assignment informationfor the originating trunk and terminating line information including all node segment 25 assignment information may also be received.

At step 2616, a determination is made as to whether the terminating line is available. If not available, then at step 2618, a “reject” message is sent to the originator and the processends at step 2620. If at step 2616 it is determined that the terminating line is available, then atstep 2622, the network segment status table (e.g., TABLES V-VI) may be accessed to find 30 usage status for the originating trunk. Results from the network segment status table for theoriginating trunk are received at the call controller at step 2624, and a determination of thelargest segment status is made at step 2626 to determine a worst parameter of the trunk. 71

At step 2628, a determination is made as to the status of the originating segment at the trunk. If the status is determined to be normal, then the process continues at step 2630. If the status of the originating segment is determined to be impaired, then the process continues at step 2632. If the status of the originating segment is determined to have congestion, then the 5 process continues at step 2634. It should be understood that the status may have more or fewerlevels than those presented herein. The levels (i.e., normal, impaired, and congested) representa range of values determined from network performance information reported to the CCM andstored in a table as collected by network communications devices or nodes on the packetnetwork. 10 If a determination is made at step 2628 that the status of the originating segment at the trunk is normal, then the process continues at step 2636 (FIG. 26B), where a request to accessthe network segment status table to find usage status of the terminating line is made. At step2638, results from the network segment status table for the terminating line is received. Thetermination of the network segment status of the terminating line is performed at step 2640. At 15 step 2642, a determination is made as to the status of the terminating segment. If the status ofthe terminating segment is determined to be normal, then the process continues at step 2644,where an invite is sent for set-up with a normal or conventional request to the terminatingtrunk. At step 2646, the process continues normal call control and the process ends at step2648. 20 If, at step 2642, a determination is made that the status of the terminating segment is impaired, then the process continues at step 2650, where CODEC capabilities of theterminating line are checked. If a lower rate CODEC is available, as determined at step 2652,then the process continues at step 2654, where a set-up with the lower speed CODEC is sent tothe terminating line. At step 2656, the call controller waits for a subsequent message from the 25 terminating line, and, upon receiving a response from the terminating line indicating that thelower rate CODEC is available at step 2658, a message is sent to the originating IP trunk withthe new lower rate CODEC information at step 2660. At step 2662, the call control processcontinues to complete call set-up and ends at step 2664.

If at step 2642 a determination is made that the status of the terminating segment is 30 congested or no lower rate CODEC is available at step 2652, then the process continues at step2666, where a “reject” message is sent to the originating call device. The process ends at step2668. 72

Returning back to step 2628, if the status of the originating segment is determined to beimpaired, then the process continues at step 2632, which enters the process at step 2650 in FIG.26B. If at step 2628 the status is determined to be congestion, then the process continues atstep 2634 in FIG. 26C. At step 2670, an instruction to the call processing to enact normal 5 traffic management tools is made and a “reject” message is sent to the originating call device.The process ends at step 2674.

In summary, the process provided in FIG. 26 attempts to improve call quality in thepacket network when calls enter the network via a tnink to a line in the packet network byexamining node segment performance for the node segments over which the call is to be 10 routed. The CCM may examine tables that include node segment status and, if a node segmentis found to be impaired, attempt to lower the rate of a CODEC through which the call is routed.Otherwise, the call may be dropped.

DATA ROUTING 15 The network performance information may include information indicative of a node segment being impaired or congestion to the point that non-real-time information is buffered,blocked or otherwise impeding real-time content from being timely communicatedtherethrough. §The node, layer 2, or above protocol stack, such as the Multi-Protocol LabelSwitch (MPLS) Label Description Protocol (LDP) stack, may determine that the node segment, 20 such as a router, is being overloaded with non-real-time content and cause the node to slowdown, delay, stop, or drop the non-real-time content from being communicated through thenode segment. The higher protocol stacks may use thts transmission state information to makedecisions for Label Switched Paths (LSP) to modified, rerouted, or shaped based upon the linkstate measured for both real, and non-real-time content. Once the higher protocol stacks have 25 the real-time information, functions, such as choosing LSPs or load balancing are possible.Oversubscription rules may also be dynamically calculated based, upon an amount of real-timetraffic traversing over a path or segment, utilization and performance informationcommunicated to the higher protocol stacks, and provisioning engines associated with thehigher network protocols. Given higher stack protocols, such as MPLS, or Provider Backbone 30 Transport (PBT), traffic engineering may be used to setup and reroute virtual circuits knowingthe amount of real-time bandwidth usage and a path state that enables a higher reliability sothat failovers will not exceed oversubscription parameters. This state knowledge may be used 73 by packet mesh networks where multiple paths exist and each path has multiple backup paths.

In general, data networks use a l:n path protection schema. When three or more links exist, protection is typically non-linear as potential bandwidth usage is a function of the destinations identified in the routing tables. To enable packet failover in a 1 :n configuration where the 5 amount of real-time traffic is known provides a network carrier with greater service assurancereliability and metrics to manage the network. In summary, the network performanceinformation for segments is stored at network nodes tracking the real-time bandwidth usageand other performance data. The stored network performance information is made available tothe higher protocols, such as MPLS, LDP, and EMS systems to track the amount of real-time 10 or near real-time bandwidth being used. Tracking the real-time bandwidth usage enhancesnetwork management for provisioning systems, failover protocols, traffic managementanalysis, and billing system utilization.

In one embodiment, a decision as to which real-time content or non-real-time content toprioritize, slow, throttle, block, rate, re-route, or otherwise control may be made based on both 15 network performance information and service level commitments or guarantees of the qualityof service that have been made to a particular customer. For example, such decision may bemade to minimize the amount of service level credits that have to be made to a particularservice provider’s customers based on how such decisions would impact the ability of theservice provider to satisfy one or more such service levels or quality of service guarantees. If 20 customer quality of service levels and guarantees are to be used for managing networkperformance, then a database including customer quali ty of service and other service contractparameters may be stored and accessed to verify that the network performance informationmeets the contractual requirements for customers of the communications carrier. In oneembodiment, a determination may be made that a particular application is utilizing too much 25 bandwidth through a node segment. For example, iin application for streaming a movie,television show, or other entertainment content may be utilizing bandwidth at a network nodethat is being strained to deliver real-time content during a particular time period. The non-real-time content associated with that application may be slowed down, dropped, or rerouted toanother node segment so that the real-time content being communicated over the node segment 30 may be properly serviced. The CCM may additionally track applications over time todetermine that other provisioning may be utilized for that application during certain time 74 periods or permanently due to increased traffic, either real-time or non-real-time content, viaone or more network nodes.

ENHANCED MESSAGING SERVICES

An Element Management System (EMS) may be used by communications carriers to 5 monitor and manage performance of their respective networks. Network performanceinformation may be collected and sorted in a manner to provide for reporting, provisioning,billing, and troubleshooting purposes. The functions may use the network performanceinformation and distinguish between real-time and non-real-time content communications. FIG. 27A is an illustration of an exemplary network system 2700 that includes two 1.0 networks 2702 and 2704 operated by different communications carriers. Each of the networks2702 and 2704 may be used for providing communications services for customers of therespective carriers. In one embodiment, the carriers are telecommunications carriers.Alternatively, the carriers may provide Internet services or other networking services and useequipment that collects network performance information indicative of performance of the 15 network in communicating real-time and non-real-time content over the respective networks2702 and 2704. The network equipment may be configured to use PIP packets for generatingand collecting the network performance information.

One or more performance data collectors (collectively performance data collector) 2713may be configured to be in communication with network equipment that operates on the 20 network of a carrier, such as network 2702. As shown, the performance data collector 2713 isin communication with end-point devices, such as IP service point 2706, network-to-networkinterface 2708, and customer access device 2710, for example. However, other networkcommunications devices may also be in communication with the performance data collector2713, either directly or indirectly. In one embodiment, the performance data collector may 25 communicate with the network communications devices via out-of-band communications paths2712a-2712n (collectively 2712). Alternatively, the performance data collector 2713 maycommunicate with the network communications devices via in-band signaling paths (notshown). A performance data manager 2714 may be configured as one or more computing 30 devices and be in communication with the performance data collector 2713. Although shownas two or more separate devices, the performance data manager 2714 and performance datacollector 2713 may be configured as a single computing device. The performance data 75 manager 2714 may further be in communication with a database server 2716, optionallyconfigured in multiple devices, that is operable to store one or more databases 2717, includingthe network performance information collected from network communications devices on thepacket network 2702 by the performance data collector 2713. The databases 2717 stored in a 5 database server 2716 may be managed by an off-the-shelf database system, such as an Oracle®database or any other commercially available database. Alternatively, the database may becreated and managed by a communications carrier or other entity.

In operation, the performance data manager 2714 is configured to instruct theperformance data collector 2713 to request and access network performance information from 10 network communications devices on the packet network 2702. The performance data collector2713 may, in turn, issue requests or polls to the desired network communications devices,either directly or indirectly, to obtain network performance information desired by theperformance data manager 2417. In one embodiment, the performance data manager 2714may issue commands to the performance data collector 2713 on a periodic basis (e.g., every 15 15 minutes). More particularly, the performance data manager 2714 may be configured to requestcertain network performance information more often than other network performanceinformation. For example, transmission quality and connectivity may be collected everysecond or minute while transmission rate and bandwidth is collected every 15 minutes.Alternatively, the performance data manager 2714 may be synchronized with the modified 20 Y.1731 stack bins in requesting counter values in each bin at the appropriate time intervals.

Still yet, the performance data manager 2714 may be configured to request networkperformance information in response to an event after parsing and examining networkperformance information previously collected. In one embodiment, the performance datamanager 2714 operates to collect data from a data packet of a single carrier. Alternatively, the 25 performance data manager 2714 may be configured to collect network performanceinformation from multiple networks of multiple carriers, if such permission is provided by thedifferent carriers. The performance data manager 2714 may be managed by a carrier or a thirdparty, where the third party is independent from the carriers and has permission to access andmanage certain or all network performance information post-processing operations for the 30 carriers. In these later two cases, where a third party is involved, quantity of access requestsand/or information may become a basic billing element used in providing access to thisinformation. 76

In accordance with the principles of the present invention, the performance data manager 2714 and performance data collector 2713 may be configured to request and receive network performance information, including performance and utilization, associated with communications of data packets including real-time and non-real-time content. The 5 performance data manager 2714 or performance data collector 2713 may store the networkperformance information in the databases on the database server 2716, as distinguished by thedifferent types of content being communicated on the data packet network 2702. It should beunderstood that if other types of content were communicated over the packet network 2702 andidentified as a particular data type (e.g., video, music), network performance information 10 indicative of the particular data type may be similarly collected and stored, accordingly.Because the network performance information is stored in a manner that distinguishes networkperformance and utilization for communication of real-time content and total content, theservice provider, its partners, and customers may use tlie network performance information tomanage network communications equipment, monitor network usage, generate reports, and 15 provide billing based on real-time and non-real-time content communications over thenetwork.

Collection of the network performance information may be directly or indirectlycommunicated from each individual network communications device on the network 2702 orfrom a table or other repository of a call control manager (e.g. CCM 1502 of FIG. 15) or other 20 device that has collected some or all of the network performance information desired by theperformance data manager 2714. In one embodiment, when the performance data manager2714 instructs the performance data collector 2713 to collect the network performanceinformation from the network communications devices, counters are read to collect theircurrent values. For example, the modified Y.1731 counters configured as bins for different 25 time periods over which the counters are used to count the real-time and total data packetsbeing communicated to and from the network communications devices. In response to thecounters performance information being collected by tlie performance data collector 2713, thecounters within each of the network communications; devices may be reset so as to avoidrollover of the counters, a mathematical situation that is inherently more difficult to manage. 30 Furthermore, tables of the network performance information that ore stored at the networkcommunication devices may be cleared or otherwise archived at the network communicationsdevices in response to the performance data collector 2713 retrieving network performance 77 information from the tables. Real-time archiving of all collected information from the deviceto the databases 2717 may be required to facilitate security or other business purposes.

The databases 2717 stored in the database server 2716 may be organized in a variety ofways to enable the network performance information to be processed and used for a variety of 5 functions, including billing, reporting, provisioning, generating alerts, managing networkcommunications devices, or otherwise. TABLE IV, presented hereinabove, is an exemplaiytable of network performance information that may be stored in the databases 2717 in thedatabase server 2716. It should be understood that cither network performance informationmay be stored in the databases to provide additional visibility into the network performance at 10 each node segment. Still yet, it should be understood that virtually any network performanceinformation that can be collected by network communications devices may be collected andstored in the databases 2717 on the database server 2716.

The database may be further expanded to include statistical or other informationderived from the network performance information or other database systems and/or database 15 information. For example, trends, such as usage over a time period of an hour, a day, a week,a month, or a year may be stored in the database in association with each node segment orotherwise. For example, customer information, circuit IDs, or other may be stored. Thenetwork transmission information and statistics may be configured to accommodate any billingor post-processing operations. For example, if the principles of the present invention provide 20 for charging customers differently for real-time bandwidth and non-real-time bandwidth usage,that information may be separately determined and stored in the databases 2717. Thedatabases 2717 may include virtually any data structure to accommodate current cost, andpricing structures associated with real-time and non-real-time content usage. The current costmay be defined for consumer, commercial and/or wholesale subscribers or on a customer-by- 25 customer basis, for example.

The database 2716 server may be configured to enable access to the networkperformance information stored in the databases to various entities, including, but not limitedto, web entity 2718, user entity 2720, billing entity 2722, and operations entity 2724. Each ofthese, entities may access the network performance information stored on the database server 30 2716 via a communications device, such as a personal computer, mainframe computer, wireless device, or otherwise. Another embodiment may include pushing portions of this data 78 from the database to similar entities, paging/text terminals, and other alarming and alertingentities.

WEB ENTITY

The web entity 2718 may utilize an Internet interface for displaying the network 5 performance information, as well as customer billing plan information that distinguishesbetween real-time network performance information and non-real-time network performanceinformation stored in the databases 2717 web interface. FIGs. 28A and 28B (collectivelyFIG. 28) are screenshots of exemplary web browser interfaces 2800a and 2800b, respectively.In web browser 2800a, an exemplary customer billing plan table 2802 may be used to display a 10 customer billing plan that includes usage allocation 2804 and billing rates 2806 associated withthat usage allocation. The billing rates and other billing related information may be stored onthe billing party computer, server hosting the website, databases 2717, or other server. In oneembodiment, the usage allocation may include bandwidth, peak (megabits per second), accesstime for “anytime” minutes, and access time for daytime minutes, for example. In another 15 exemplary embodiment, the billing rates 2806 may include parameters, such as bandwidth,peak rate, access time (anytime on a per minute basis), access time (daytime on a per minutebasis), and total data (on a per one hundred megabit basis). As shown, real-time and non-real-time settings may be different as network performance information is available for both real-time and non-real-time usages. It should be understood that total usage could also be shown or 20 shown in place of the non-real-time column and the non-real-time information could bederived by subtracting real-time content network performance information from the totalinformation. In addition, the information may be itemized into directional informationshowing the same types of information identified as information into or out of the customerlocation. 25 FIG. 28B shows a web browser interface 2800b includes exemplary table 2812 that shows customer actual usage parameters 2814. These parameters reflect the real-time and non-real-time content network performance information collected by the performance data collector2704 (FIG. 27A) and stored by the performance data manager 2714 in the database server2716. It should be understood that other parameters that distinguish between real-time and 30 non-real-time content usage may be utilized for billing customers. It should further beunderstood that parameters that do not distinguish between real-time and non-real-time usageof the packet network may be used for billing purposes as well. 79

Although the tables shown in the web browsers 2800a and 2800b show informationassociated with billing, it should be understood that other non-billing information may bedisplayed in a web interface. More specifically, in addition tp the usage information, otherinformation, such as service agreement terms, including quality of service, guaranteed 5 bandwidth, base subscription fees, or any other terms or conditions between a carrier andcustomers, partners, other carriers, or other commercial or governmental entity may be storedand presented on the web interface. It should further be understood that the web interface mayenable other, non-subscriber partners to access various information stored in the database. Forexample, a partner, such as a local service provider, or other communications carrier may have 10 access to certain network performance information that the carrier who owns the networkperformance information may wish to share. For example, transmission connectivity of anetwork-to-network interface that communicates directly with the other carriers’ network-to-network interface may be shared. A permissions database or table and associated securityconstructs, such as authentication, may be managed by the database server 2716 or other 15 device that define the data that the communications carrier is willing to share with othercarriers, customers, equipment manufacturers, or otherwise. The permissions table mayprovide different levels of information to different entities.

USER ENTITY

The user entity 2720 may be a user of the communications carrier who manages the 20 database. The user 2720 may access the network performance information stored in thedatabase 2717 and also perform various other management operations on the databases 2717.For example, the user 2720 may generate additional tables, reconfigure the tables, design newdatabase architectures, and so forth, so that network performance information may beexpanded and provide customers, partners, vendors, etc., with different or more detailed 25 information, for example. In addition, the user may generate different ways of managing thenetwork performance information, such as generating statistics based on the modified Y.1731counter bins, setting up thresholds to cause event messages for alerts to be created, setting upand initiating polls to network communications devices for various event-driven or non-event-driven reasons, and adding statistics processing for the network performance information to 30 provide additional information to management of the communications carrier, customers,vendors, etc. It should be understood that the user 2720 may perform any other database 80 management operation for which the user has proper administrative permissions to manage the real-time and non-real-time network performance information as understood in the art.

BILLING ENTITY FIG. 27B is an illustration of an exemplary billing entity system 2722 for use in5 determining billing for customers and partners of a communications carrier. The billing entity2722 includes a processing unit 2726 that may include one or more processors. A memory2728 may be in communication and used for storing data and program instructions duringprocessing operations. A storage unit 2730 may be in communication with the microprocessor2726 and be used to store one or more databases or other storage repository that include 10 network performance information, information derived from the network performanceinformation, and billing information. Input/output (I/O) ports 2732 may be in communicationwith the processing unit 2726 and be configured to communicate over a packet network usingone or more communication protocols as, too, may be the processing unit 2726. The I/O ports2732 may be virtual in nature. For example, the I/O ports 2732 may operate as an Internet 15 protocol socket or otherwise.

The billing entity system 2722 may use programs for managing and preparing bills.The programs may include data collection programs 2734, billing programs 2736, and databaseprograms 2738. These programs 2734-2738 may be executed by the processing unit 2726.The data collection programs 2734 may be configured to communicate with one or more 20 network communications devices in a virtual call path. The communication with the networkcommunications device(s) may transfer raw (e.g., uncompressed) data records between thenetwork and/or end-devices and the billing entity system 2722. The communication transfermay be initiated by either the billing entity system 2722, considered as an “information pull/*or by network communications devices, considered an “information push.** The remote 25 network device may contain storage to aggregate multiple records and programming logic toclear information from the device once the raw information transfer has occurred.

Billing programs 2736 may use the raw data records contained within a data packet(e.g., PIP data packet) and parse data fields, such as concatenated data fields, contained withinthe received data packets into individual raw data fields. Each individual raw data field may 30 be utilized by the database programs 2738 for storage in a database.

The billing programs 2736 may further routinely process the database records. Thisprocessing may include consolidation of multiple raw data records into one or more processed 81 records, summation of real-time and/or non-real-time raw data Field information into totalsand/or sub-totals over a time-window or session duration. These totals or sub-totals mayinclude start and stop time of usage, summation of time of usage, total packets sent/receivedwith and/or without error, statistical performance calculated values, and/or any other types of 5 information that can be derived via processing raw data records of network performanceinformation. Additionally, the billing programs 2736 may perform ratings, which are themonetization of billing records. Totaled or derived fields may be assessed against a set ofbusiness charging rules and a monetary charge amount may be established for each data recordstored in the storage unit 2730 by the database programs 2738. The billing entity system 2722 10 may consolidate multiple rated billing records on a per customer basis. By consolidating themultiple rated billing records, additional calculation or rating function may provide specificbusiness functions, such as discounting or otherwise.

Continuing, the billing entity 2722 may use the network performance informationstored in the database to provide for billing plans for customers and other carriers to include 15 billing for both real-time and non-real-time network usage. This additional resolution ofbilling (i.e., real-time usage billing) is a result of being able to determine packetcommunication of real-time content over the packet network by using performance informationpackets, for example. Consumers may be billed for real-time content usage, non-real-timecontent usage, and total usage of network communication capacity. The capacity may be a 20 function of the bandwidth usage for real-time content and total packet communications overthe network. In one embodiment, Erlangs, which is generally understood to be mean totaltraffic volume over a period of one hour or 3600 seconds (centum call seconds), may be usedas a measure for the carrier to provide accurate billing for customers. The specific calculationof Erlang may vary to account for different network performance information being used to 25 determine the number of Erlangs used during a billing cycle. In accordance with the principlesof the present invention, the Erlang measure may be used to determine real-time, non-real-time, and total usage by a subscriber or other carrier by calculating total traffic volume ofsubscribers of other communications carriers communi cating on the communication carrier’snetwork in a roaming situation. In addition, because a communications carrier may monitor 30 bandwidth and other network performance information for both real-time and non-real-timecontent communication, the communications carrier may add or offset a subscriber’s bill basedon factors, such as transmission quality, connectivity, or rate or other network performance 82 information and/or business purpose, such as a Service Level Agreement, that may be collectedduring a billing cycle. Such offset may also be utilized for other carriers’ bills as well.

As an alternative or complement to using Erlangs as a standard of measure, the carriermay assign points or other units of measure to a subscriber for real-time usage and non-real- 5 time usage. For example, a real-time usage minute may be worth three points and a non-real-time usage minute may be worth one point. The billing may indicate the number of points thatthe subscriber has used and charge the subscriber accordingly. For example, if the subscriberuses thirty minutes of real-time usage, which translates to ninety points, then that subscribermay be charged differently from a subscriber using tliirty minutes of non-real-time minutes, 10 which is only thirty points. Other creative ways of bil ling based on real-time usage resolutionmay also be utilized in accordance with the principles of the present invention. Furthermore,because the database of network performance information may include timestamps withcollected usage information of a subscriber and other carriers, the billing entity 2722 (FIG.27A) may use that information based on a time of day to set rates during peak and non-peak 15 network congestion time periods. This time of day or network congestion time period may beutilized by the carrier to bill the customer for usage during peak and non-peak times.

The billing process may further use terms of a customer’s plan to limit network usagefor real-time and non-real-time communications. If, for example, a customer has a serviceagreement for two thousand minutes of real-time minutes, the performance data manager 2714 20 may monitor a customer’s real-time content minute usage, optionally as measured in terms ofErlangs or bytes of real-time traffic, and determine that a customer has exceeded the limitbased on the customer’s usage plan. In response to the customer exceeding the usage minutesin his or her usage plan, the carrier may perform a number of different options, including(i) shutting off the user’s real-time content communications, (ii) allow the customer to continue 25 using the network for real-time content communications, for example, but use a “best efforts”process or lowest available CODEC for allowing access to the network, where “best efforts”means that the user will receive a lower priority status, such as non-real-time datacommunications access priority, (iii) premium bill the client so that the client pays extra tocontinue having priority for real-time content communications, (iv) trade units, such as 30 allowing the customer to use additional non-real-time units for real-time usage at a higher exchange rate (e.g., five non-real-time usage points for every minute of real-time content communications usage), (v) take an advancement towards next month’s usage minutes, or (vi) 83 any other plan that enables the user to continue with real-time usage or non-real-time usageover the usage plan limits. In one embodiment, a message may be sent to the customer toselect an option for continued service above his or her service agreement limits. In anotherembodiment, a customer may “pre-pay” for real-time units, and be denied service once the 5 units are used. In yet another embodiment, two carriers may make business and connectivityarrangements to inter-exchange database information to allow a subscriber to “roam” ontoanother provider’s network and still have access.

The same, similar, or different billing arrangements may be utilized for determiningbillings for commercial entities, such as reciprocal billing between carriers based on the real- 10 time bandwidth transmission from one carrier to another or on an aggregate basis, for example.When managing accounts with other carriers, trade units of usage, including real-time and non-reai-time content usage, may be resolved at the end of a billing cycle. By having real-time andnon-real-time content usage information, trading units can become “creative” such that thecarriers may either better balance the usage of each others’ networks or gain a business 15 advantage by being able to (i) restrict another carrier’s usage of the packet network or (ii)collect additional fees for providing additional real-time content usage or non-real-time contentusage of the carrier’s network. It should be understood that many real-time and non-real-timecontent usage network performance information parameters may be utilized in determiningbilling arrangements with subscribers and billing and sharing level arrangements with other 20 carriers having reciprocal billing arrangements.

RECIPROCAL BILLING

Carriers typically have inter-carrier service agreements that enable communicationsfrom one carrier to be routed over a network of another carrier. These service agreementsoften have reciprocal billing arrangements whereby the amount of usage of a carrier’s network 25 is balanced or paid for at a certain time period against the usage of that carrier’s network by theother carrier. This enables the carriers to balance the service payments other carriers based onusage differentials. In accordance with the principles of the present invention, the carriers mayinclude metrics or parameters that track both real-time and non-real-time contentcommunications over each other’s respective networks. Adding resolution to identify real-time 30 content usage may identify imbalances occurring between carriers (i.e., one carrier is communicating significantly higher real-time content over another carrier’s network). The CCM or other monitoring device may recognize this imbalance and determine that 84 communications to subscribers may be routed to the network of the carrier that has a highbalance as a credit for communications routed over that carrier’s network may exist. Adecision may be made to route the communications, real-time or non-real-tirae contentcommunications or both, over that carrier’s network. 5 As another example of carrier level service being imbalanced, carrier service level agreements may specify a certain quality of service or transmission rate, possibly with real-time content and non-real-time content being separately specified. A carrier may monitor forthese service level agreement parameters to determine if another carrier is meeting itsobligations under the agreement. If the obligations under the agreement are not being met, 10 then the service provider may receive credits toward additional free communications services.Routing decisions may be made in response to determining that these or other service levelagreement parameters are not being met and credit is available.

Another example of routing decisions being made in response to tracking networkperformance information or of service level agreement information may include monitoring 15 pricing by other carriers throughout times of the day that are scheduled or in response to a highdemand occurring within that carrier’s network. The other carrier may “advertise” pricing orother parameters, such as bandwidth availability at an NNI node, for example, to notify othersubscribers of pricing changes, availability, transmission problems, etc. The CCM orperformance manager 2714, in learning of price changes either upwards or downwards, of 20 other carriers may make routing decisions based on those pricing changes. The decision mayalso include factoring current credit, cost to carrier, customer bandwidth requirements, or anyother parameter associated with performing communication services at a certain transmissionquality, and cost to the carrier. Routing “shopping” may be performed by collecting such“advertised” information during regular PIP packet communications or special rate collection 25 requests to each carrier network-to-network interface or session border controller with whichthe carrier has a service level agreement.

OPERATIONS ENTITY FIG. 29 is an illustration of an exemplary graphical user interface (GUI) 2900 thatdisplays a schematic 2902 of a packet network 2904 and performance monitoring devices 30 2906. The schematic 2902 is a graphical representation of network communications devices located on the network 2904 including node segments over which real-time and non-real-timecontent communications are communicated. It should be understood that more or less detailed 85 schematics of the packet network 2904 or other external devices or networks may be displayed.The network performance information stored in the database 2717 on the database server 2716(FIG. 27A) may be utilized to graphically represent problems and alerts on the schematic 2902.For example, if communications on a node segment is determined to be normal, then a solid 5 line, such as line 2908 may be displayed on the schematic. If the network performanceinformation indicates that a node segment bandwidth is being utilized to either full or over-capacity, then the node segment may be highlighted, such as node segment 2910 using athicker line than other node segments that are operating normally. Alternatively, color coding,flashing, or other graphical representations may be utilized to indicate high traffic volume. If a 10 node segment is determined to be impaired, then the schematic may show a dashed line, suchas line 2912. If the network performance information indicates that a node segment hascongestion, then the node segment line may be dashed, such as line 2914 being visuallydifferent from a line indicating impairment. It should be understood that other graphicalrepresentations indicating high usage of real-time bandwidth, non-real-time bandwidth, or any 15 other network performance information that is within a range or outside of a threshold may beused for graphical notification or alerting a user of an abnormal condition occurring on thenetwork 2904. Other colors, text, pop-up windows, or any other graphical features may bedisplayed on the schematic for normal or abnormal operation of the network. Sounds may alsobe used for notification, alerts, or alarms. In one embodiment, the graphical user interface may 20 enable the subscriber to position a cursor using a pointing device, such as a mouse, over a nodesegment to cause network performance information to appear in a pop-up window or otherwisedisplayed in relation to the node segment. The node segment information displayed mayinclude current and, optionally, historical network performance information, and be displayedeither as a value or graphically. Notifications may also be displayed in response to cursor 25 positioning. FIG. 30 is a screenshot of another exemplar)' graphical user interface 3000 that isdisplaying a chart 3002 of node segments status usage for a particular node on a network.Three network performance information parameters are displayed on the chart 3002, includingtotal usage, real-time usage, and non-real-time usage; as shown in the legend 3004. Total 30 network usage is shown by line 3006, which changes over the course of a day as customers areincreasing and decreasing usage of the node segment from which the usage data has beencollected and stored in the database server 2716 (FIG. 27A). The total usage line 3006 is a 86 sum of the real-time and non-real-time content usage. A. total usage line 3006 is shown to have a morning peak at about 9:00 a.m. at point 3008, mid-day peak at about noon at point 3010, and evening peak at approximately 8:00 p.m. at point 3012. The real-time and non-real-time usage lines show that real-time usage increases at various times of the day and non-real-time 5 usage increases at other times of the day. For example, the non-real-time usage spikes at about9:00 p.m. at point 3014 when, presumably, customers are downloading movies, music, orotherwise web surfing. It should be understood that other graphical representations may bemade of one or more node segments or transmission paths through a network. It should also beunderstood that other types of network performance information, including derived 10 information (e.g., trend lines), may be displayed to show transmission quality or othertransmission characteristics or node characteristics at any point within a network beingmonitored in accordance with the principles of the present invention as provided by theperformance data manager 2714 (FIG. 27A) and use of PIP packets.

In addition to showing the usage information on a chart, alerts, trends, or other statistics 15 may be presented on the GUI or via any other repoiting method. Still yet, reports of thenetwork performance information may be generated through the use of the stored networkperformance information stored in the databases 2717 and provide a user interface forselecting, sorting, tabulating, and any other function that can help a user generate current andhistorical reports, alerts, alarms, or any other information associated with or resulting from 20 network performance information collected from a network.

PROVISIONING ENTITY

The performance data manager may additionally use the network performanceinformation that is collected from the packet network to provide provisioning functions.Provisioning may include a variety of functions, including (i) tracking path or element 25 oversubscription rates and utilization prior to allowing network provisioning to occur, (ii)managing network performance tracking by creating reports for newly created entities, (iii)dedicating or calculating failover, (iv) load balancing for re-routing of real-time or non-real-time content communications, (v) retrieving and presenting state information on networkutilization and available resources to network managers in the form of reports and trend lines 30 to determine where congestion is occurring, (vi) displaying locations where additional routers,gateways, or other network communications devices may be desired to alleviate congestion orprovide safety valves for network communications devices that require higher bandwidth 87 capacity during certain times of the day, or (vii) providing any other network managementfunctionality based on the network performance information as described herein. In addition,the provisioning may enable automatic response to alerts or warnings that are detected by theperformance data manager on a real-time or near real-time basis. For example, if an alert is 5 created by a threshold for bandwidth capacity, the performance data manager may seek to re-route real-time or non-real-time content communications. Alternatively, if a spike in real-timecontent communications is occurring at a node that has non-real-time data being communicatedat the same time, the performance data manager may notify the node to halt new provisioningor new communications sessions of the data packets including non-real-time content until the 10 real-time content communications rate has decreased. The performance data manager 2714may further be configured to direct one or more network nodes to change the bandwidth of aCODEC, close ports, send messages to other carriers to notify the other carrier of an overloador over-usage condition coming from their network, or perform any other provisioningfunction through the use of monitoring the network performance information, as provided. 15 The performance data manager 2714 may be configured to automatically detect a problem within the network and issue one or more tests, such as a trace route, to be performedon an end-to-end basis. For example, in FIGS. 27 and 29, a message may be sent from theperformance data manager 2714 via the performance data collector 2704 to cause a test to bemade between customer equipment 2916 and 2918. The test may include sending PIP packets 20 for a one minute time period, for example, between the customer equipment 2916 and 2918.During that time period, the customer equipment 2918 may collect network performanceinformation, such as transmission quality, transmission rate, and transmission connectivity,optionally as associated with real-time and non-real-time content communications. Eventhough the customer equipment 2918 resides with a different carrier, the network performance 25 information that was collected from running a test between the customer equipment 2916 and2918 may be collected by the performance data collector 2704 without sharing any companyspecific, sensitive information of the network carrier or carriers managing network 2920. Inone embodiment, the network performance information collected from the customer 2918 is aresult of “stitching” (concatenation) of network performance information by appending the 30 information to PIP packet payload as communicated through each of the nodes in thetransmission path between customer equipment 2916 and 2918. Alternatively, the 88 performance data collector 2704 may request data directly or indirectly from each of the nodesalong the transmission path between the customer equipment 2916 and 2918.

MODIFIED TRACE ROUTE

Network performance information collected for real-time content and non-real-time 5 content communications may provide an indication that there is a performance problemexisting at a node segment within a packet network. A call end-point, CCM, or node withinthe packet network may determine that a problem exists based on the PIP performanceinformation and automatically trigger a path trace route in the PIP packet flow. Given thathigher protocol stacks can move or otherwise alter a packet transmission path without 10 consulting the CCM, this function facilitates identifying nodes and segments being traversed atthe time the trouble is encountered. It should be understood that aside from a staticallyconfigured PIP data stream for embedded network equipment, that PIP sessions may beconstructed in an ad hoc basis for use on packet devices not normally associated with thatnetwork. In the ad hoc PIP case, the user end-point creates a PIP session from and to a point

15 inside the network provider’s network or to the far end-point to provide the real-timebandwidth and PM data function. An ad hoc PIP packet flow may be set up with each callfrom end-point-to-end-point or, alternatively, to an anchor point in the serving networkprovider’s network. Once the PIP and PM stack detect a performance threshold crossing, atrace route may be initiated to identify the location within the path that is having a problem. A 20 network node element may store the trace information and make the trace informationavailable directly to the CCM and/or user. Additionally, the trace information may becommunicated to the call control protocol stacks to be passed back to the CCM or EMS fortroubleshooting. Other information may be stored with the trace information, such as time,date, session information and so on. The troubleshooting procedure may be performed to 25 isolate the node segment having a problem with either or both real-time and non-real-timecontent being communicated through the node segment. The CCM or node may initiate amodified trace route to communicate one or more data packets, such as PIP packets to orthrough the node segment of concern to collect network performance information through thatnode segment that may be having a transmission problem. The network performance 30 information generated from the trace route, which may last for one or more PIP packets beingcommunicated over a long enough duration to determine the network performance informationat the node segment of concern. If the node segment is a node segment located at the edge of 89 another network or type of network (e.g., network-to-network interface) then networkperformance information collected at the other node in the other network may becommunicated back to the CCM or originating node with network performance informationspecifically related to the modified trace route. Other “carrier specific sensitive” network 5 performance information of the other network may otherwise be prevented from beingaccessed by the other network. It should be understood that ad hock PIP packet flows may alsobe associated with encrypted path protocols and presence protocols that establish remotenetwork connectivity, such as PPP, SLIP, and/or other remote agents.

The collected network performance information may enable not only messages and 10 alerts to be sent to operations management of the network, but also notify customers, partners,affiliates, and other network carriers of problems, congestion, or other situations or events ofthe network. For example, if a determination that real-time content communications usage ishigh, a notice may be sent to subscribers and other carriers of the situation and notification ofincreased billing rates. Similarly, if a determination is made of high amounts of 15 communications to other carriers, the carrier may elect to “shop” communications todestinations via other, lower priced carriers. The carrier may have thresholds for many termsand conditions of subscribers and partner carriers and automatically, semi-automatical ly, ormanually make provisioning changes based on determining that a threshold has been crossedbased on the collected network performance information. 20 FIG. 31 illustrates an embodiment 3100 of the OSI basic reference model of networking that include seven different layers. The reference numerals (3102-3114) on the leftside of the model are used to describe these different layers of the reference model. Each ofthe layers provides protocols for certain types of operations. More specifically, the sevenlayers include: physical layer 3102 (Layer 1), data link layer 3104 (Layer 2), network layer 25 3106 (Layer 3), transport layer 3108 (Layer 4), session layer 3110 (Layer 5), presentation layer 3112 (Layer 6), and application layer 3114 (Layer 7). Typically, physical layer 3102 conveysbit streams, such as bits containing electrical impulses, light or radio signals, through anetwork at the electrical and mechanical level. The physical layer 3102 provides the hardwaremeans for sending and receiving data, including defining cables, cards, physical aspects, data 30 coding, and medium (B8ZS, DS-3, etc.). At the data link layer 3104, data packets are encoded and decoded into bits. The data link layer 3104 further furnishes transmission protocol knowledge and management, and handles errors in the physical layer, flow control, and frame 90 synchronization, including Ethernet, Frame Relay (FR)» ATM, Multi-Protocol Label Switching(MPLS), etc. In the following examples, the network performance information is stored in thedata link layer 3104, and, optionally, the other layers 3106-3114.

The network layer 3106 provides for (i) switching and routing, and (ii) creating logical 5 paths, known as virtual circuits, for transmitting data from node to node within a packetnetwork. The transport layer 3108 provides transparent transfer of data between end systems,or hosts, and is responsible for end-to-end error recovery and flow control. One example of anetwork protocol is Internet Protocol (IP). An exiimple of a transport layer protocol isTransmission Control Protocol (TCP). The session layer 3110 establishes, manages, and 10 terminates connections between applications. The session layer 3110 sets up, coordinates, andterminates conversations, exchanges, and dialogues between the applications at each end of anetwork path. The session layer 3110 further manages session and connection coordination.The presentation layer 3112 provides independence from differences in data representation(e.g., encryption) by translating from application to network format, and vice versa. The 15 presentation layer 3112 transforms data into the form that the application layer 3114 canaccept. Such presentation layer 3112 typically includes text, voice, and video compression.The application layer 3114 also supports application and end-user processes. Some examplesof application layer 3114 applications include email and file transfer applications. Each layerinteracts directly with the layer immediately beneath it and provides facilities for use by the 20 layer above it. In addition, the protocols on each layer enable entities to communicate withother entities on the same layer. FIG. 32 illustrates an embodiment 3200 of various maintenance entities (ME) depictingdefined multiple administrative domains, such as Subscriber Maintenance Entity (SME) 3218,Ethernet Virtual Connection (EVC) ME 3220, Operator ME 3224 and 3226, Network-to- 25 Network (NNI) ME 3222, and User-to-Network (UNI) ME 3228 and 3230. The domains havebeen constructed using Maintenance Entity Group (MEG) - 8 level structures to providelimited views into the quantity and types of information available to each level (domain). Amaintenance entity is a subset of all available maintenance data that has been grouped togetherfor access by a particular network participant, such as a subscriber, Ethernet provider, network 30 operator, or virtual network operator.

The OSI reference model described in FIG. 31 defines specific functionality containedin each of its layers 3102-3114. The principles of the present invention may utilize Ethernet 91 services, which operate in the Data Link Layer 3104 of the OSI reference model. The Ethernetprotocol is identified as ETH Layer 3232 in FIG. 32., where FIG. 32 illustrates the transportlayer 3108 of the OSI model 3100 as TRAN layer 3234.

In one embodiment of the present invention, real-time transmission performance 5 information acquired in the Data Link Layer 3104 is communicated into one or more ofPhysical Layer 3102, Network Layer 3106, Transport Layer 3108, Session Layer 3110,Presentation Layer 3112, and Application Layer 3114. In another embodiment, the real-timetransmission performance information acquired in the Data Link Layer 3104 may becommunicated into other Data Link Layer protocols, such as ATM, MPLS, Frame Relay, or 10 other protocols. This real-time transmission performance information may be used to providereal-time notification of the ETH Layer 3232. This real-time transmission performanceinformation may also be used to complement existing protocols and capabilities to providequicker response time to network changes identified to ETH Layer 3232.

Data Link Layer 3104 from and to Physical Laver 3102. 15 In one embodiment, real-time transmission performance information acquired in the

Data Link Layer 3104 may be communicated to the Physical Layer 3102. In one embodiment,tire degradation of a copper-based link due to induced noise or any other source of impairment,delay, or loss of data could limit the quantity of information that can be carried error-freeacross the link. The transmission performance information carried in the PIP packet is capable 20 of identifying this degradation. This degradation may be reported to the Physical Layer 3102,where a protocol operating on the Physical Layer 3102 realizes the degradation and modifiesthe route to optimize throughput and overcome the impairment, such as rerouting the link to analternative physical copper link or a reduction in the number of Quadrature AmplitudeModulation (QAM) windows or change to another transmission schema all together. 25 Data Link Laver 3104 from and to Data Link Laver 3104.

In one embodiment, the above degradation may be communicated to the Data Link

Layer 3104, where the multiplexed protocols of the Data Link Layer 3104, operating in parallelwith the Ethernet due to physical layer multiplexing and protocol isolation, realize thedegradation and modify their operation to overcome the impairment by conducting an MPLS 30 Fast Re-Route. 92

Data Link Laver 3104 from and to Network Laver 3106.

In one embodiment, the above degradation may be communicated to the Network

Layer 3106, which could alter network traffic routing to reroute packets around the degrading link. This reroute may involve moving the session from one network operator to another 5 network operator. It should be understood that the principles of the present invention may beutilized with any Network Layer (Layer 3) 3106 protocol, including IPv4, IPv6, or otherwise.It should further be understood that the principles of the present invention may be utilized withany protocol operating on any other layer.

Data Link Layer 3104 from and to Transport Laver 3108. 10 In one embodiment, the above degradation may be communicated as round trip delay and other parameters to the Transport Layer 3108, where the TCP Sliding window functionmay be dynamically altered to modify the window size, thereby reducing the amount ofsubsequent retransmitted packets and avoiding congestion. In such an embodiment, suchcommunication allows the adjustment of the window size sooner than current implementations. 15 Data Link Laver 3104 from and to Session Layer 3110.

In one embodiment, the above degradation may be communicated to the Session Layer 3110, where the session management functions could modify schedulers, shapers, or anynetwork element function that provides and contains the Quality of Service (QoS) parameters,thereby dynamically adjusting the quantity of packets in a session. The effect of dynamically 20 adjusting the quantity of packets in the session is that congestion points should experiencerelief as the quantity of packets flowing into a network node or element is reduced.

Data Link Laver 3104 from and to Presentation Layer 3112.

In one embodiment, the above degradation may be communicated to the PresentationLayer 3112, where the presentation protocol could dynamically control a video codec forcing a 25 repeat of the last video frame or reducing frame quality, frame resolution, frame size, framerate or otherwise.

Data Link Laver 3104 from and to Application Laver 3114.

In one embodiment, the above degradation may be communicated to the ApplicationLayer 3114, where notification is generated and communicated to a user indicating that the 30 network is experiencing congestion and to be patient until the congestion clears, try thecommunication later, or try to re-connect using different connection parameters. For example,if a user is engaged in online gaming, the application layer may notify the gamer that the 93 network is slow and to wait before engaging in a fierce battle to avoid the network not having enough bandwidth to facilitate the online action. In another embodiment, the application layer 3114 may determine that the user is a low priority and cut or kill the network connection to the gamer or user. 5 Other uses of data packets including being passed between the Data Link Layer 3104 and Application Layer 3114 may include communications control to manage multiple real-timesessions when the user exceeds available communications resources. Functions, such aspresenting the user with usage statistics of network performance information real-time content(e.g., real-time usage or bandwidth) versus total bandwidth, session usage of real-time 10 bandwidth, and the ability to selectively choose CODEC’s and session types, such as videophone versus a voice-only communications modes, are enabled utilizing the principles of thepresent invention. Load balancing of real-time traffic when multiple paths are available mayalso become user selection modes.

Lavers to MEs. 15 In another embodiment, and continuing with FIGS. 31 and 32, real-time performance information may be communicated from the Physical Layer 3102, Data Link Layer 3104,Network Layer 3106, Transport Layer 3108, Session Layer 3110, Presentation Layer 3112, andApplication Layer 3114 into the MEs (e.g., subscriber ME,EVE ME, and NNI ME of FIG. 32).This real-time information can be used to complement ME information, support real-time 20 modification of network processes and protocols, and assist domain administrators inmanagement of a hybrid network or group of networks, such as a Metro Ethernet Network(MEN). Several descriptions of the use of real-time information flows from protocols ofvarious OSI layers into MEs are described below.

Further, the Metro Ethernet Network Nodes (see FIGS. 32 and 33) may utilize the 25 information contained in a PIP packet to actively determine the best path for each connectionwithin its network. One or more virtual performance tables (VPTs) (FIG. 34) may be createdat the MEN node(s) that inputs information relating to each network node. The MEN nodesmay determine that a particular link goes down at a particular time of day, such as in a carrier’smaintenance window, and, in anticipation of this event, reroute the data traveling on that 30 particular link around it to other links, thereby relieving the congestion on a particular link ornetwork node. Best path metrics may also be used to determine if certain real-time or non-real-time data content needs to be held up for a period of time to assist with relieving the 94 congestion on a particular portion or link of a network. Tables, such as VPTs, may be used by the MEN to anticipate potential congestions on a network and proactively reroute the data on other links to avoid the congestion.

Physical Layer 3234 from and to ETH Layer 3232. 5 In one embodiment, and continuing with FIGS. 31 and 32, the above degradation

occurs on the copper link of an NNI ME 3222, a portion of the circuit that is not Ethernet-based. In this instance, this copper link is providing an end-to-end Virtual Ethernet service asthe NNI portion of the EVC. This degradation may be reported as a change (reduction) in theamount of bandwidth available on the circuit link from the Physical Layer 3102 to the ETH 10 Layer 3232. This information may be included in the appropriate ME domains by a networkelement or node, thereby allowing other network elements (upstream and downstream fromsuch network element) the ability to react to the degradation prior to link failure. Suchcommunication of degradation information provides the ability to try to pre-establish analternative to maintain an end-to-end session in advance of a failure. 15 In another embodiment, a “route flapping” degradation by the Network Layer 3106 may be reported from and to the ETH Layer 3232. “Route Flapping” is a common term todescribe the recalculation of route tables within an network element typically due to a linkhaving marginal connectivity; i.e., conditions are such that the link may “flap” and bemomentarily considered out of service, then naturally recovering and being placed back in

20 service by a network element. This route flapping may occur many times over a time interval.Each time the network element is restored, a route table re-calculation may be requested by theNetwork Layer 3106 routing protocol. A network element could include tlie PIP PMinformation in the appropriate ME domains, thereby allowing other NEs the ability to react tothe degradation prior to link failure and assess real-time stability prior to restoring the link. A 25 potential reaction could be to identify an alternative network operator or network segment,thereby routing around the portion of the network that is “flapping.” Secondarily, thresholdinformation could be communicated to a network of another carrier to allow the other networkto react to the degradation prior to the outage becoming, more severe.

TCP SLIDING WINDOW 30 Transport Laver 3108 to and from ETH Layer 3232.

In one embodiment, a reduction in a “TCP Sliding Window” contained within the

Transport Layer 3108 may be reported to the MEs. This reduction would signal to the ETH 95

Layer 3232 that congestion is occurring somewhere within a virtual circuit over which PIPpackets are being communicated. The congestion may be in the subscriber’s network, where anetwork operator would not otherwise have visibility. In this embodiment, the TCP/IP slidingwindow field is modified in real-time, regardless of any network technologies, thus providing 5 quicker TCP/IP sliding window response to performance issues.

The sliding window field within the TCP/IP protocol is modified to reflect perfonnance changes occurring in the network. This modification may occur at any network node anywherewithin a communication path. The TCP sliding window field modification may beaccomplished by rewriting the specific TCP sliding window field within the TCP/IP packet as 10 it traverses through a network node.

Session Layer 3110 to and from ETH Layer 3232.

In one embodiment, a change in session connection quality by the Session Layer 3110to the ETH Layer 3232 may be reported. The Session Layer 3110 may notify the ETH layer3232 that a QoS parameter, has been modified thereby dynamically adjusting the quantity of 15 packets in the session. A network operator or Ethernet provider may use the QoS informationto manage other EVCs within an MEG, including Connection Admission Control (CAC).

Presentation Layer 3112 to and from ETH PIP flow on Layer 3232.

In another embodiment, a CODEC buffer management algorithm within thePresentation Layer 3112 may communicate to the ETH Layer 3232. Here, a notification may 20 signal that video CODEC buffers have multiple underflow events resulting in repeats of thelast B-frame in an MPEG-4 video, for example. Underflow events are indicative of lost· ordelayed packets. The ETH layer 3232 may use this underflow information to by-pass thedegraded segment by choosing an alternate path.

Application Laver 3114 to and from ETH Layer 3232. 25 In another embodiment, a user program within Ihe Application Layer 3114 could signal to the ETH Layer 3232 that the “Network is Slow.” The complaint may be reacted to by theETH layer 3232, whereby dynamic identification of a degraded segment and an attempt tomodify the session path to circumvent the degradation may be performed.

In another embodiment, instead of directly modifying fields within existing protocols 30 traversing through a network element at any layer in the OS I reference model, an alternativemay include establishing a Vector Performance Table (VPT) within the network element. This 96 VPT may be created and managed as part of a network element operating system and embedded system programming.

In FIG. 33, an embodiment 3300 of an exemplary network element or node is illustrated. In this embodiment, the NE 3302 has four physical interface connections 3304 — 5 3310 that connect to other NEs (not shown) via connections 3318 - 3324. In addition to the physical interface connections 3304-3310 to other NEs, internal interconnections betweenphysical interfaces exist. An example of an internal connection is referenced as 3312. In oneaspect, the NE 3302 further includes a processor 3312 and memory 3314 in accordance withthat described herein. Although not shown, physical or virtual internal connections may exist 10 as point-to-point, point-to-multipoint, or multipoint connections between any or all physicalinterface connections 3304-3310 on a per packet basis. Many currently available NEs mayprovide different internal connection paths, which may result in different packet performanceon a per packet basis. These different performances may be a result specific to internal packethandling processes, such as different types of queuing, scheduling, and rate shaping among 15 other packet process handling. A particular path through a network element may yielddifferent performance measurements than other paths. Other internal architectural structurescould be present, too, that may impact internal performance of a network element. FIG. 34 illustrates an embodiment of exemplary virtual performance tables (VPTs)3402a - 3402n (collectively 3402). The VPT functionality may be predetermined or operator 20 defined and configured via the embedded programming on the NE. The NE vendor may allowthe operator the capability to dynamically size the VPT via configuration parameters. Withinan NE, bi-directional ME performance information is captured and placed into the VPT 3402ain the “current” timestamp 3412. At a later time interval defined as delta t (At), theinformation contained in time stamp 3412 for VPT 3402a is moved to VPT 3402b; the 25 information contained in time stamp 3412 for VPT 3402b is moved to VPT 3402c, and soforth. Alternatively, a new VPT may be created and VPT 3402a may simply become 3402b asa result of the new VPT being created. The information may also be placed in bins or memorylocations or added, summed, averaged, or otherwise summarized or used in calculations, theresult of which is placed in such bins or memory locations. Bins, such as modified Y.1731 30 bins, may be associated with time intervals, MEs, levels of access, operator identifiers, or otherparameters used to identify, communicate, process, collate, or allow access to informationincluded in the bins. Bins that collect network performance information over shorter time 97 intervals may be periodically added into bins that collect network performance information

over longer time intervals. Once the “current” time stamp is empty, bi-directional ME performance information is placed into the “current” tune slot, as illustrated in VPT 3402a. in

essence, a first-in, last-out VPT queue is established. However, other temporal related VPT 5 configurations may be utilized in accordance with the principles of the present invention.

The table size of the VPTs 3402 may be a function of the quantity of memory allocated, the types and quantities of network performance information captured. The networkperformance information may include link number 3414, real-time and total bandwidth usage,packet loss 3416, latency 3418, jitter 3420, delay 3422, real-time application data, non-real 10 time application data, total data, and the At or time stamp 3412 between successive samples.Also, “stitched” network performance information (i.e., network performance informationfrom other network elements) For each NE could be included in the VPT 3402. In one, theVPT 3402 could be dynamically sized to accommodate the data. FIG. 35 describes an exemplary maintenance entity data packet or logical packet flow 15 through a network entity 3502. Bi-directional data packet flows carry network performanceinformation. FIG. 35 further illustrates ingress data packet flows 3508 and 3514 and egressdata flows 3510 and 3512 through the NE 3502. Within the NE 3502, the ME embeddedprogramming determines local NE performance measurements and attaches this information onthe end of the ME or network performance information payload portion of a packet, as 20 discussed further below. The payload may be encapsulated within the envelope of the EthernetProtocol or PIP packet format. In one embodiment, this information could be encapsulatedwithin additional layers of higher protocol information, such as TCP/IP packet protocols.

An exemplary logical structure of the payload portion of the PIP packet is described inFIGS. 36 — 39. FIG. 36 illustrates an exemplary PIP packet payload ingress flow in direction 25 1. FIG. 37 illustrates an exemplary PIP packet payload egress flow in direction 1. FIG. 38 illustrates an exemplary PIP packet payload ingress flow in direction 2. FIG. 39 illustrates thePIP packet payload egress flow in direction 2. In these data flows, the addition of the NE 3502performance information at the NE 3502 egress in each direction is shown. This informationmay be appended to the payload of a packet received on the ingress as the flow is processed 30 through the NE 3502. An end-station, the last device participating in the PIP packet process, may collect all NE performance information from each NE in a communication path, as illustrated in FIG. 40, which depicts PIP packet payload data flows of the end-station (i.e., data 98 flows to and from the end-station). In one aspect, there are two directional paths, as the circuitis full duplex (i.e., transmitting and receiving in both directions concurrently), sometimes onseparate physical facilities. Since the call path in each direction could be different and subjectto differing forces that modify the performance statistics, two bi-directional flows may be used. 5 The performance statistics at each end of the PIP flow may be concatenated and transmitted tothe far end so each end of the transmission or communication path holds both the transmit andreceive network performance information data.

This VPT information may be used locally via new protocols operating outside the PIPpacket or with modifications to existing protocols to allow the use of VPT information. As 10 described above, in one embodiment, rather than directly writing specific network performanceinformation into other OSI Layer protocols, the nelwork performance information may bemade available via the VPT 3402. Each OSI Layer’s protocols may reference any networkperformance information to make enhanced operations decisions.

The VPT 3402 enhances current data flows by capturing not only current data flows, 15 but also providing historical captures over defined time windows that are nA samples deep.The additional samples can enable predictive functions, which can be used to improve thereliability and availability of the session or user experience, perform network maintenance,provision new network hardware or media, design new network configurations, or enhanceinter-network communications. 20 The VPTs 3402 (FIG. 34) may be extended to include the collection of VPT network performance information across a single operator or multiple operators. Conceptually, thiscollection is illustrated in FIG. 41, which depicls an embodiment 4100 of a VectorPerformance Correlation Engine (VPCE) 4102. Individual VPTs 4104a — 4104n (collectively4104) from a network element may be communicated via in-band or out-of-band 25 communication links 4106a — 4106n (collectively 4106) to the VPCE 4102. These VPTs 4104could be transmitted as encapsulated information using common protocols, such as TCP/IP.The entire set of VPTs 4104 including nAt performance samples could be sent by the NE orpolled via the VPCE 4102. Alternatively, each current sample of VPTs 4104 may be sent orpolled and the VPCE 4102 may be utilized to establish and maintain a historical database of 30 the performance samples for each NE.

Once the network performance information is gathered at the VPCE 4102, the network performance information may be processed to provide an encompassing performance 99 management view of one or more networks based on input from each NE. This centralized network performance information store may be used in a variety means such as, but not limited to, Service Level Agreement (SLA) validation, near real-time NE management, predictive network management, and other functions. 5 Customizable algorithms and calculations that use the current and historical network performance information may be developed and included as part of the embeddedprogramming of the operating system of the VPCE 4102. VPCE 4120 may include memory4108, one or more processors 4110, which may include cell processors having two or moreprocessors on a single chip, one or more databases 4112, and one or more I/O ports 4114. The 10 algorithms and calculations may be performed using these computing resources containedwithin the VPCE 4102. Information processed within the VPCE 4102 could be made availableto other network systems (not shown), such as a multimedia Call Control Manager (CCM) orother network management systems using the I/O ports.

In addition, the VPCE 4102 may use the data contained in the VPTs 4104 as historical 15 logs for determining when the performance of a certain link 3414 in a network experiencefailure or deterioration due to congestion or other technical problem. The informationcontained in the PIP packets may contain the historical data rate performance as discussedherein showing the network nodes and links and based on data contained in a particular VPT4104, such as timestamp 3412, detenninations can be made that a particular node or link 20 suffers technical problems, such as congestion during specific times of the day. The VPCEmay also determine the gapping between calls based on this historical network performanceinformation contained in individual or multiple VPTs 3402.

In one embodiment, the VPCE 4102 correlated information is used to create a near real-time exemplary Graphical User Interface (GUI) 4202as illustrated in FIGS. 42a and 42b, which 25 is an illustration of an embodiment 4200 of such a GUI.. In these figures, possible connectionpaths may be illustrated as links 4204a — 4204n (collectively 4204) that connect NE 4206a —4206n. The links 4204n, 4204a, 4204g, 4204e, 4204c, and 4204d are being used to supportend-to-end connectivity. Link 4204b, and 4204-f are alternative circuits available tosupport connectivity, but are currently not carrying traffic. Path 4204e may change color, such 30 as from yellow to red, indicating the link is in severe trouble or congested. The width of theline representing 4204e has been reduced to indicate reduced packet flow. Wider lines mayrepresent greater packet flow. One color may be used to represent real-time application packet 100 flow and another color for non-real-time packet flow within a single path as illustrated. Basedupon this visual notification, the NMS Operator may take steps to route traffic currentlytraversing links 4204e to 4204b or 4204f. Alternatively, these changes could be performedautomatically as described above. The same or similar graphical user interface 4202 can be 5 provided for transmission path segments of a packet network using stitching andcommunicating the VPT or network performance information through in-band signals to theend customer where the information may be displayed to detail the network performancebehavior of the packet transmission paths that the customer is utilizing or being sold. Thisprinciple of communicating network performance information across networks of customers 10 can also be applied ad hoc without the knowledge of the operator with Network-to-Networkinterfaces at the boundaries of the 3rd party service provider to provide each end-point with thenetwork performance information of each network segment. This “man in the middle*’scenario enables tracking of the real-time bandwidth transmission characteristics of customersor other third parties, along with the other PM data. The boundary or segmentation principle 15 can also be utilized across wireless technologies, whereby multiple wireless connectivitysegments are available. In this case, the PIP and VPT tables would provide PM or networkperformance information about the wireless RF route performance. It should be understoodthat using this boundary principle can apply to any technology deployed between two MIP orMEP points. 20 In yet another embodiment, one or more links 4204 may further include indicia representative of the quantity or percentage of real-time application packet flow versus non-real-time packet flow. FIG. 42C represents an enlarged view of an exemplary link 4204i thatdisplays the amount of real-time application flow relative to the amount of non-real-timeapplication flow by showing two different types of indicia, in this case rectangles, relative to 25 each. For example, indicia 4208 may include a different color, cross-hatching, shading,shapes, or other type of indicia that is different than that for indicia 4210. In this example,indicia 4208 indicates the amount of non-real time application flow and indicia 4210 indicatesthe amount of real-time application flow. Further, the general dimensions, such as widths ofthe indicia 4208 and 4210 may reflect the application flows relative to each as well. In FIG. 30 42C, the amount of non-real time application flow is shown as being less than that for the real- time application flow by the indicias 4208 and 4210 having both different hatching and widths.Any indicia and dimension of indicia may be used to readily present this information to a user. 101

In another embodiment, information exchange between the ETH Layer 3232 to andfrom and the OSI layers 3102, 3106 - 3114 may manifest itself in open and closed loopinteractions. Open and close loop systems are well defined in Modem Control Theory textbooks. In summary, in an open loop, information is exchanged without a feedback loop to 5 track a response to the information. In the closed loop manifestation, feedback loops arepresent, thereby providing dynamic control of the response to the information. The principlesof the present invention can use either open or closed loop manifestations. FIG. 43 illustrates an embodiment of an exemplary network implementation 4300. InFIG. 43, NEs 4302a - 4302n (collectively 4302) contain an OSI protocol stack as defined by 10 3102a — 3114a, 3102b - 3114b, 3102c — 3114c, and 3102n ~ 3114n. Additional NEs may exist in the network having similar structures as defined by NEs 4302.

Network Layer through Application Layer, 3106b - 3114b and 3106c - 3114c, may notexist in some types of NEs, such as Ethernet Switches. In other cases, such as with routers,some additional layers above the Data Link Layer 3104b and 3104c may exist. It is the 15 existence of these layers above the Data Link Layer 3104b and 3104c where someembodiments of the present invention take place. The Physical Layer 3102a - 3102n generallyexists in each NE and is included in the embodiments of the present invention.

Bi-directional MEs 4304 and 4306 exist and operate in the OSI reference model DataLink Layer 3104a - 3104n. An end-to-end user communication path may be defined by each 20 of 3114a, 3112a, 3110a, 3108a, 3106a, 3104a, 3102a, 3102b, 3104b, 3104b, 3102b, 3102c,3104c, 3104c, 3102c, 3102η , 3104η, 3106η, 3108η, 3I10n, 3112n, and 3114n. Within NE4302b and 4302c, information flows up from the Physical Layer to the Data Link Layer andthen back down from the Data Link Layer to the Physical Layer as it is processed by each NE.The information flow can either be full duplex (bi-directional paths operating independently 25 from each other at the same time) or simplex (operating in one direction at a time, but in bothdirections) or uni-directional (operating in one direction only). FIG. 44 illustrates another embodiment of an exemplary network implementation 4400.

As described above, bi-directional MEs 4304 and 4306 exist and operate in the OSI reference model Data Link Layer 3104a — 3104n. Within NEs 4302a — 4302n, performance information 30 can be extracted from PIP packets. Once extracted, this information can be sent via communication pathways 4402 - 4408 to the Physical Layer 3102a - 3102n protocols where 102 the operation of these Physical Layer protocols can be modified to react to real-time information provided via the performance information. FIG. 45 illustrates an embodiment of a wireline Digital Subscriber Loop network 4500, including an Ethernet Router/Switch 4514, Ethernet Data Stream 4510, Ethernet Management

5 Stream 4512, Network/Ethemet Management System 4508, and Internet Service Providers(ISPs #1 and //2) 4516a and 4516b. One example of how performance monitoring of non-Ethernet segments may be utilized is in monitoring of broadband access Digital SubscriberLoop (xDSL) connections. The term, “xDSL” generally means DSL technologies, such asADSL, ADSL2, ADSL2+, VDSL, VDSL2, etc. By extracting and inserting relevant DSL 10 performance statistics into the PIP packet, a single management system may have visibility tothe end-to-end performance of a customer’s connection. This insertion could includeappending this performance information to the end of the payload of the PIP packet aspreviously described. This appending may occur at the DSL DSLAM 4504 (assuming theNetwork Connection is Ethernet). The PIP packet containing the DSL network performance 15 information 4506 may then be made available to the Operator’s Network Management System4508.

The additional DSL network performance information improves repair resolution time,as network problems at a subscriber may be quickly identified. The DSL network performanceinformation allows full monitoring and troubleshooting through a single network management 20 system. In addition, dynamic configuration changes based on network performance may bemade by the Network Management System 4508 to optimize circuit performance. Forinstance, if a DSL circuit 4502 suddenly experienced a peak or spike in impulse noise on thenon-Ethernet segment, appropriate diagnostic information may be inserted into the Ethernetmanagement stream and adjustments may be temporarily made by the network management 25 system 4508 to the DSL Signal-to-Noise ratio to compensate for the interference and to ensureline stability. After a given timeframe or due to improving changes in captured performancedata, the line could be re-provisioned by the management system to improve overallperformance. This example is one of dozens of possible configuration changes that may bemade in real-time to optimize DSL circuit performance. 30 Other exemplary network performance information parameters that may be captured and inserted into the management stream include: near-end failures, far-end failures, last statetransmitted (downstream and upstream), actual signal-to-noise ratio, maximum attainable data 103 rate, actual power spectrum density, actual aggregate transmit power, xDSL profile, xDSLlimit PSD mask and band-plan, xDSL Power Spectral Density mask, estimated upstreampower back-off electrical loop length, trellis code use, actual cyclic extension, band number,line attenuation per band, signal attenuation per band, signal-to-noise ratio margin per band, 5 actual data rate (downstream and upstream), previous data rate (downstream and upstream),actual interleave delay (downstream and upstream), actual impulse noise protection, impulsenoise protection report, actual size of Reed-Solomon codeword, actual number of Reed-Solomon redundancy bytes, actual number of bits per second, actual interleaving depth, actualinterleaving block depth, actual latency path, interval number, interval status (valid and 10 complete; invalid or incomplete), forward error correction seconds, errored seconds — line,severely errored seconds — line, loss of signal seconds — line, unavailable seconds — line, fullinitializations, failed full initializations, short initializations, failed short initializations, syncmode, or other capabilities identified in xDSL(e.g., ADSL1, ADSL2, ADSL2+, VDSL2, etc.).

Specifically, the T1.413 Standard defines methods to dynamically adapt the DSL 15 transport stream, the subject matter of which is hereby incorporated by reference. Thesedynamic adaptations are described in the T1.413 standard under the sub-section “On-lineadaptation and reconfiguration using the Overhead Control Channel (AOC)”. In this sub-section the standard defines that the AOC data is carried as overhead bytes in the DSL framingstructure. The actual multiplexing of these overhead bytes into the DSL framing structure 20 depends on the framing structure used (i.e., frill overhead or reduced overhead) and on theallocation of any bearer channel to the fast or interleaved data buffer.

The type and length of an AOC message (except for the acknowledge messages) areidentified by a byte-length header. In particular, the AOC channel sends an all binary zeros“00000000” AOC stuffing pattern in the Idle State, and a valid AOC message always begins 25 with a non-zero byte.

The T1.413 Standard further defines “On-line adaptation - Bit swapping.” Bitswapping enables a DSL system to change the number of bits assigned to a sub-carrier orchange the transmit energy of a sub-carrier without interrupting data flow. An ATU (DSLTermination Unit) may initiate a bit swap. The swapping procedures in the upstream and 30 downstream channels may be independent and may be performed simultaneously. For the bitswap protocol, the “receiver” is the ATU that is receiving the data; it transmits a bit swap(extended or simple) request message and receives the bit swap acknowledge message. The 104 “transmitter” is the ATU that is transmitting the data. It receives a bit swap request (extendedor simple) message and transmits the bit swap acknowledge message.

Bit Swap Request Commands. DSL information or other network performance information may be used to 5 dynamically alter some performance parameters of the Physical Link, such as transmit power.A sub-process may be established in the DSL unit whiich would monitor PIP packets and thenissue the proper AOC Bit Swapping commands (see FIG. 46) to affect the necessaryperformance requirements. FIG. 47 illustrates an embodiment of an exemplary wireless network 4700 that operates 10 in accordance with the principles of the present invention. Many types of wirelesscommunications may benefit from the insertion of performance data into a PIP packet. Byextracting and inserting relevant wireless segment performance information into the PIPpacket, link performance problems may be detected, and traffic could then be rerouted via acentralized management system. Likewise, if the management system determined that a user 15 could achieve greater overall performance by routing traffic in a different manner, that user’straffic may be diverted from the current path, even if that path may have the strongest wirelesssignal. For example, a wireless device 4702 communicates with a wireless access point 4704via a wireless path 4706. In one example, the wireless path 4706 between the wireless device4702 and the wireless access point 4704 has strong signal strength. As the quantity of users on 20 this link increases and performance over the wiireless path 4706 degrades, networkperformance information contained in PIP packets collected by the Network managementSystem (NMS) 4712 may trigger a wireless network management system to send instructionsto the wireless device 4702 to redirect traffic to wireless access point 4708. by redirectingwireless traffic to the wireless access point 4708; performance of the wireless access point 25 4704 may improve and, in response, traffic may be redirected from the wireless access point 4708 to the wireless access point 4704 to accomplish load balancing.

The network performance information indicative of a problem at a wireless access pointmay be appended to the end of the payload of PIP packets. This appending may happen at thewireless routers 4704 and 4708. The PIP packets containing the wireless performance 30 information 4710 would be available to the Network management System, which, in turn, mayinstruct the wireless device 4702 and the wireless access point 4704 to disconnect andreconnect the wireless device 4702 via wireless access point 4708, in one embodiment. This 105 disconnection can override other wireless connectivity parameters, such as signal strength.Alternatively, the NMS 4712 could instruct the wireless devices to switch to a differentchannel. Other similar variations are also possible to re-route or re-channel the wireless device4702. 5 One example of re-routing may be as follows. If the NMS system 4712 determines that connectivity to ISP #2 4516b may provide better performance than ISP #1 4516a, traffic wouldbe dynamically rerouted to ISP #2 4516b based on rules, thresholds, etc., that the NMS 4712could apply to network performance information collected at either or both of the ISPs 4516aand 4516b. Other exemplary variables include: wireless channel, encryption level, and 10 connectivity mode (802.11a, 802.11b, 802.1 lg, 802.11η, WiMax, etc.). The NMS system4712 may constantly monitor the network performance information data flows throughout thewireless network 4700 and evaluates traffic and paths based on the network performanceinformation contained in the PIP packets. In response to determining that one or more nodesegments are underperforming, calls may be rerouted. The NMS system 4712 may evaluate 15 the line state at and between each the connection points within the wireless network 4700. Inone embodiment, the NMS system 4712 may evaluate the core network, including trunksegments, in addition to evaluating the wireless access points 4704 and 4708. The NMSsystem 4712 may retest connections on a periodic basis, such as every 10 seconds or 10milliseconds, for example. 20 Although shown as a portable computer, the wireless devices 4702 may alternatively be a phone, PDA, and/or any wireless device that may use the wireless network 4700 tocommunicate. Although two wireless access points 4704 and 4708 are shown, any number ofwireless access points may be used with the NMS system 4712 and wireless network 4700.

Further, any number of wireless networks 4700 may be used for evaluating and routing 25 wireless calls. For example, NMS system 4702 determines that a particular wireless network ishaving difficulty carrying calls due to congestion or other technical problem, then the NMSsystem 4702 may switch or route its calls to another wireless network. In addition, if the NMSsystem 4702 determines that a particular signal strength from a wireless device to a wirelessaccess point is weak or becomes weaker due to any number of factors, including due to the 30 user increasing distance between himself and the access node, then the NMS system 4702 maychange one or more communications parameters, including encoding, modulation, frequency,and the like to improve or increase the signal strength between the user and the access node. 106

Changing communication parameter^) could be done automatically or manually via a buttonon the user’s wireless device that initiates a request for determining why the signal strength isdegrading or decreasing, and, in accordance with the principles disclosed herein, theinformation derived from the PIP packet may be used to determine these or other solutions to 5 improving the signal or increasing the signal strength. In yet another embodiment, the NMSsystem 4702 may troubleshoot the networks automatically without any user initiation on aperiodic basis to report back on the status of these wireless connections.

Network Layer Example.

The Network Layer 3106 may determine how data is transferred between network 10 devices, route packets according to unique network device addresses, and provide flow andcongestion control to prevent network resource depletion. For purposes of this invention,routing protocols are defined as the protocols used in the implementation of routing algorithmsto facilitate the exchange of routing information between networks. This exchange of routinginformation allows NEs defined as routers on the Network Layer 3106 to build routing tables 15 on a dynamic basis.

In one embodiment, the principles of the present invention provides for the injection ofdynamic link state information obtained from PIP packets into routing algorithms. In FIG. 48,the Data Link Layers 3104a — 3104n may make network performance information in the PIPpacket available to one or more of the Network Layers 3106a - 3106n. The network 20 performance information may include data associated with communications of data packetsincluding real-time content. Data flow over the Data Link Layer may include networkperformance information derived from either of the MEs 4304 and 4306 to Network Layerrouting protocols and routing protocol metrics. This is shown as data flows 4802 - 4808.

In essence, real-time network performance information, such as link failure, link 25 degradation, MEF TRAN failure, Label Switch Path (LSP) ping, trace-route, Virtual CircuitConnection Verification, Bi-Directional Forward Detection, MPLS Fast Reroute, and othersimilar capabilities may be dynamically inserted into Link State routing protocols, therebyforcing recalculation of the route tables, calculation of optimal route paths, and potential LinkState Advertisement (LSA) re-advertisement. LSAs are processes to update neighbor nodes in 30 the event of a Link State change. The LSA process typically creates a short message (i.e., the link-state advertisement) which: 1) identifies the node which originates the LSA; 2) identifies all the other nodes to which it is directly connected; or 3) includes a sequence number, which 107 increases every time the source node makes up a new version of the message. This message isthen communicated throughout the network. In one embodiment, the link-state message iscommunicated to all other network nodes on the network. Typically, each node in the networkis responsible for storing the sequence number of the last link-state message which it received 5 from other nodes. Once the LSA process completes, each node uses this information incalculations for an optimal routing path to other nodes on the network. This information maybe included as routing metric information by Network Layer routing algorithms.

Currently, many common implementations of routing protocols, such as OSPF,establish a link cost to be proportional to the inverse of the link bandwidth. A use of the ME 10 information may be to modify the link cost to represent a larger value; hence, the cost rises andthe use of the link is less likely. Once congestion clears in the ME, the link cost could bereestablished to reflect the normal setting.

Real-time dynamic link information can significantly enhance link state packet routingprotocols. The dynamic link state injection into packet routing protocols may offer the 15 capability to: 1) sample the quality of the physical connection and proactively react to failingconditions, 2) assess changing traffic flows over the connection at regular intervals providingper flow traffic rerouting to accommodate optimal performance, 3) proactively react todegradation conditions affecting the circuit such os creating alternative paths and reroutingtraffic prior to circuit failure, 4) load balancing traffic flows over multiple circuits to 20 accommodate circuits that are operating at less than optimal conditions, 5) improve route re-convergence times, 6) eliminate some route ‘flapping’ conditions and 7) other similar types ofroute enhancing capabilities. FIG. 49 illustrates an embodiment 4900 of the injection of dynamic link stateinformation into Transport Layer protocols and algorithms. In this Transport Layer 25 embodiment, such information flows into and out of MEs 4304 and 4306 as data flows 4902 —4908, In one embodiment, this information flows via PIP packets. FIG. 50 illustrates an embodiment 5000 of a TCP packet in accordance with theprinciples of the present invention. An example of the data flows into the Transport Layer3108 would be a congestion notification via an PIP packet into the Transport Layer 3108a - 30 3108n. TCP is a common protocol that operates in the Transport Layer. In TCP, a sliding window is a variable flow control mechanism to manage the efficiency of transmission on thenetwork. The TCP Sliding window allows a sender to transmit a specified number of data 108 units before an acknowledgement is received or before a specified event occurs, such as a timerexpires. The TCP window function also has a tributaiy effect on the quantity of packets thatcan be transmitted during a time window given that the TCP protocol requires a far-endacknowledgement that a window-size of packets was received prior to transmission of the next 5 packet. When the physical distance between end-points becomes large, the acknowledgementtime becomes a significant contributing factor in the reduction of effective capacity of a callpath. The TCP window can be set to a larger size to increase the transmit versus wait for theacknowledgement window. However, this setting has potential to cause congestion on localLAN segments given the Ethernet collision domain. In this case, the embodiment uses either 10 an ad hoc PIP packet or existing PIP packet if the user is using a static VPN or Point ofPresence protocols, such as SLIP or PPP. In this example the MEP and/or Protocol stackcontains the PIP PM information, which may include a round trip delay measure. With eachTCP session, the TCP protocol can automatically check the PIP PM information anddynamically adjust the TCP window size to meet the line-state conditions. This closed-loop 15 system effectively automates the TCP window setting to the optimal setting for obtainingthroughput performance. It should be understood that the same delay information can be usedto alter the TCP time-out windows. The ME performance information indicative of congestionmay be used to directly adjust (reduce or increase) the TCP Sliding Window 5002, which inturn, reduces the quantity of packets defined by the window that could require retransmission 20 due to tost packets.

In a packet network, an intermediate node that identifies network degradation, delay,congestion, and the like, may capture PIP packet information, which may include event data,and propagate the information to other NEs using PIP packets. At the same time, theintermediate node may also inject this information into the ACK packet flowing from the 25 receiver to the sender. After getting this information, the sender may change the window sizeand follow up with other appropriate action.

Those data packets that may be flowing in the reverse direction, the window sizemodification information can also be passed to the receiver to take appropriate action. In oneaspect, the information contained in PIP packet may be stored externally to the OSI stack or 30 injected into a Transport Layer 3108 device and be stored as a line-state to effect the change ofthe TCP window size, The NEs may have the lower Transport Layer 3108 protocols, and ifnot, then another downstream NE that has the ability may be responsible for this action. 109

At each end of the TCP connection, buffers may be used to manage the data flow. Thismanagement may be in a form of flow control and uses the TCP Sliding Window 5002 toperform this flow control. In the TCP Sliding Window function, a window is defined as themaximum number of unacknowledged bytes that are allowed in any one transmission 5 sequence. The receiver of a packet flow specifies the current receive TCP Sliding Window5002 in every packet sent to the originator. The sender may send up to the amount of specifiedin the TCP Sliding Window 5002 before it has to wait for an update on the TCP SlidingWindow 5002 (from the receiver). It should be understood that the TCP function may bemodified to send a PIP parameter modified window and time-out settings to the far-end during 10 the initiation of the TCP session itself and/or query PIP information stores contained on theend-points

That the sender network node buffers its own sent data until it receivesacknowledgements (ACKs) for the sent data. The TCP Sliding Window 5002 size is typicallydetermined by whatever is the smallest between the Receive Window and the sender's buffer. 15 The TCP Sliding Window 5002 field indicates the range of acceptable sequence numbers,beyond the last segment, that has been received successfully. This value is the allowed numberof octets that the sender of the ACK is willing to accept before an acknowledgement. As theTCP process performs the transmission of a segment of data, it places a copy of the data in aretransmission queue and starts a timer. If an ACK is not received for that segment, or a part 20 of that segment, before the timer runs out, then the segment, or the part of the segment that was not acknowledged, is retransmitted. This embodiment directly modifies values contained inthe TCP Sliding Window 5002 or other portion of TCP Packet 5000 as it traverses through aNE with Layer 4 capabilities.

Within TCP packet 5000, individual code bits flags are identified as fields “U,” "A,” 25 “P,” “R,” *‘S,” and “F” are used to indicate the nature of the header in relationship to the

protocol conversation. For example, such fields include U - Urgent Pointer (URG) 5004, A -Acknowledgement (ACK) 5006, and P - Push function 5008. Push function 5008 causes theTCP sender to push all unsent data to the receiver rather than sending segments when it getsaround to them, (e.g., when the buffer is full). Additional fields typically found in a TCP 30 packet 5000 include: R - Reset the connection (RST) 5010, S - Synchronize sequence numbers(SYN) 5012, and F - End of data (FIN) 5014. 110

Also within TCP packet 5000 are three other fields that may be directly modified toassist in the shaping of the traffic flow between sender and receiver. The first field is theWindow field that indicates the range of acceptable sequence numbers beyond the last segmentthat has been received successfully. A value of the window field represents the allowed 5 number of octets that the sender of the ACK is willing to accept before an acknowledgment.The second field is the Urgent Pointer 5016 that shows the end of the urgent data so thatinterrupted data streams can continue. When the URG bit 5016 is set, the data is given priorityover other data streams. The last field is the Options 5018 that may contain a TCP MaximumSegment Size (MSS) and is sometimes called Maximum Window Size or Send Maximum 10 Segment Size (SMSS).

In one embodiment, the NE 4302b — 4302c supports a protocol stack through the OSITransport Layer 3108. In addition, a NE 4302b - 4302c supporting this embodiment maycontain a set of embedded programming instructions that would react to the ME performanceinformation, establish which fields in the TCP packet would be modified, modify the field 15 values, and send the packet out of the egress interface. Modifications could be made to packetstraversing in any direction (sender to receiver, receiver to sender, or both).

In another embodiment, the OAM information obtained from the Data Link Layer 3102and contained in a PIP packet is used in other types of protocols, such as the User DatagramProtocol (UDP). Since UDP does not use a window or acknowledge packet receipts like 20 TCP/IP, there is no control on the sending rate. Nevertheless, the rate may be controlled bysetting limits on the maximum bandwidth allowed between sites used in other applications andprotocols, such as File Transfer Protocol, Database Storage, and Voice over Internet Protocol(VOIP). Once the network problem is detected, the Data Link Layer 3102 information,derived from the PIP packet, may be injected into Transport Layer 3108 to make the source 25 control the bandwidth. The fault identification process may be refreshed at certain intervals soas to get the current status. Once the fault goes away, this indication may be injected into theTransport Layer 3108 data so that the appropriate NE can take the necessary action and theoriginal transmission rate can resume.

In another embodiment, the information contained in the PIP packets may be 30 communicated to the protocol stacks contained with Network Layer 3106 devices, such asrouters, to convey that there may be the potential for collapse due to congestion. Thiscongestion avoidance may be achieved by packet queuing and/or packet dropping techniques 111 to slow down excessive UDP traffic. Further, Datagram Congestion Control Protocol (DCCP)may be used to add end host TCP-related congestion control behavior to high-rate UDPstreams, such as streaming media. FIGS. 51 - 53 illustrate corresponding exemplary embodiments 5100 - 5300 of data 5 flows to other layers in the OSI reference model.

Further to the discussion above to FIG. 50, FIG. 54 illustrates an embodiment of an exemplary method for adjusting TCP window size when the PIP OAM performance andutilization information indicates a fault in the middle of the network. In this embodiment, afault is detected in the middle of the network and the fault information is injected into the 10 Acknowledgement (ACK) packet before being sent to the packet-sending network device(sender). Upon receiving the information, the sender may change the window size. Typically,a sender starts with an initial TCP window size.

In step 5402, the capability of the network element to process layer 4 information isdetermined. In step 5404, the PIP ME performance and utilization information is captured and 15 sent to the layer 4 embedded programming process in step 5408. In these steps, a QoS ortraffic problem may be identified at a Data Link Layer 3104 node. Also, in step 5408, thesender may receive the ACK packet from the receiver. The fault information may be injectedfrom Data Link Layer 3104 to Transport Layer 3108. This may include embedding thisinformation in the TCP ACK packet In step 5408, the nearest NE with Transport-Layer 3108 20 may also be identified. In step 5416, the sender receives the ACK with fault information. Instep 5410, the embedded program determines if the PIP performance information indicates aperformance degradation. If so, then for each TCP packet received at the ingress of the NE atstep 5412, a determination is made at step 5414 to determine if the TCP state is established. Ifin an opening state or closing state, then the NE may do nothing. If the TCP state is in an 25 established state, then the current window size is viewed and determined if it is at zero (0). Ifit is at zero, then the TCP inherent flow control mechanisms may have already taken care ofthe congestion problem. If the window size is non-zero then the TCP window size 5418 isreduced and the packet is sent out the egress 5420. In step 5418, the sender may makeadditional appropriate changes in the TCP window size, such as increasing the window size. 30 In addition to the embodiments above, FIG. 55 illustrates an embodiment of an exemplary system 5500 and method for shaping network traffic (“network traffic shapingsystem”) that includes using an Ethernet First Mile OAM Packet, PIP packet, or other suitable 112 packet to dynamically change traffic shaping to minimize bursting and packet loss on a packetswitched network. FIG. 55 illustrates a typical packet network 5502 including a broadbandremote access server (BRAS) 5504 and a DSLAM 5506 interconnected across the packetnetwork 5502 in which the network traffic shaping method operates. Typically, the packet 5 network 5502 operates in Data Link Layer 3102 (FIG. 31) of the OSI reference model andtypically includes data link communication devices, or data link layer devices, such as bridgesand switches. Generally, bridges and switches extend the effective length of a LAN bypermitting the attachment of distant stations. It should be understood that the virtual packetpath between the BRAS, DLSAM, DSLAM modem, and User CPE devices may traverse any 10 type of packet network or transport schema.

The packet network 5502 may support Data Link Layer 3102 or Network Layer 3106 network facilitating Data Link Layer 3102 tunnels or any other packet network that supportsData Link Layer 3102 tunneling, such as Asynchronous Transfer Mode (“ATM”). In anotherembodiment of the present advanced fail-over method, LAN switches are used to interconnect 15 multiple LANs.

Some common switching technologies used with the present network traffic shapingsystem 5500 include store-and-forward switching and cut-through switching. Typically, store-and-forward switching waits for an entire frame, or encapsulated packet(s), to be receivedbefore forwarding. In cut-through switching, the switch begins forwarding the frame when 20 enough of the frame is received to make a forwarding decision. It should be understood thatthe BRAS function is a universal edge shaping function that can be distributed throughout thenetwork. The shaping function is normally statically set to a circuit performance level, whichshapes and discards traffic to meet a specific performance parameter regardless of what isoccurring downstream in the network. 25 In addition, the network traffic shaping system 5500 may operate with common bridges including transparent bridging as found in Ethernet environments, source-route bridging ascommonly found in Token Ring environments, and source-route transparent bridging ascommonly found in mixed Ethemet/Token Ring environments.

The BRAS 5504in the instant shaping example, is typically the gateway between the 30 Internet and DLSAMs in the network accessed by DSLAM customers. The BRAS 5504 orshaping entity may contain a MEP. The MEP may track performance or it may obtain networkperformance information of the customer and node or trunk levels from somewhere within the 113 OSI stack. In one embodiment, the BRAS 5504 can use the transmission path stateinformation by detecting transmission Frame Loss via PM information contained within thePIP packets. The PIP session(s) can be both trunk level to a network node, such as a DSLAMor to a customer level. In both cases the trunk or line state PM engine detects that transmitted 5 packets passing beyond the shapers are being dropped in the packet network 5502 oraggregation device going to the DSLAM 5506, the DSL modems, or customer CPE. In oneembodiment, the PIP PM state communicates transmission loss to the traffic shapingmechanisms, a reaction to the transmission path congestion or degradation and can further limitthe transmission rates to alleviate the congestion by modifying (lowering) the shaper windows. 10 In one example, transmission loss causes the network to slowly begin dropping these packetsin the packet network, and the shapers react by perfoiming one or more shaping or schedulingfunctions through the BRAS 5504 to stop the packets from being dropped in the packetnetwork. It is understood as the packet transmission rate increases or frame loss rate decreasesthat the shaping and scheduling functions should gracefully increase the through-put window, 15 thereby returning the circuit state to its normal condition. As a result of these functions, thepackets are dropped before entering the packet network, thus not tying up bandwidth withpackets that are ultimately dropped within the packet network 5502.

In one embodiment, the BRAS 5504 is used to shape DSL traffic of each user 5510 byusing end-to-end signaling outside of TCP flow control to adjust the bursting to eliminate 20 congestion. In one embodiment, the IP protocol flows anticipate lost packets and enablesdropping these packets prior to them being dropped in an internal network, thereby eliminatingcongestion control and reducing network burst traffic, which increases the amount of capacityrequired by the network.

In one aspect, a CPE, such as DSLAM 5506, receives network performance 25 information, via PIP packets, that a percentage of the packets that were sent by the BRAS 5504did not get delivered to the DSLAM 5506. In one embodiment, the modified Y.1731 protocolPM stack is used to transmit the receive PM information performance data from the DSLAM5506 to the BRAS 5504. The protocol performs algorithms to determine the number ofdropped packets not received by the DSLAM 5506. Thus, the DSLAM 5506 contains a 30 Y.1731 stack that correlates this information. There £tre three general ways in which this data is transmitted to the BRAS 5504, a trigger, a push, and a pull, as commonly known to thoseskilled in the art. It should be understood that to accommodate the “forward” shaping at the 114 edge of the network a PIP PM data must be returned to the shaping entity from the far end.The relay of the PM information from MEP to MEP is understood to occur at the node trunklevel, node port level, Customer NID device, and Customer CPE to obtain each subsequentlevel of shaping. 5 The BRAS 5504 performs one or more of several functions to “self-heal” the packet network 5502. For example, the BRAS 5504 can decrease the data transfer rate to each user5510 from the Internet 5508 to the packet network 5502, thus causing the packets to bedropped prior to entering the packet network 5502 and avoiding the packets later beingdropped in the packet network. The BRAS 5504 provides for real-time dynamic network 10 traffic shaping based on session flow performance of the far-end based on the performancedata included in a received packet. Thus, if a CPE, such as the DSLAM 5506, iscommunicating through the PIP packets that packets are being dropped in the packet network5502, the BRAS 5504 relieves the congestion on the packet network by decreasing, via filter5512, session flow performance of the users 5510 at the BRAS 5504 prior to the session flow 15 entering the packet network 5502.

The foregoing exemplary BRAS operation reduces provisioning complexity by addinga “self-healing” traffic shaping mechanism to the network Internet service point. The presentnetwork traffic shaping system also enables a “plug-and-play” traffic adjustment scheme inthat a user may change the network and the BRAS and/or other network elements will 20 dynamically learn the available transport capacity and adjust accordingly without beingmanually configured.

In one embodiment, the network traffic shaping system 5500 includes a nodal masscalling shaping congestion control function for shaping the rate of data traffic through anetwork based on PIP and PM packet information. 'Hie network traffic shaping system may 25 enforce a queue or traffic shaping for an entire access node 5516 or group of traffic in anaccess node 5516. Using the same principles as discussed above, all traffic from an accessnode may be placed into a virtual packet circuit (IP, Mac Address, VLAN, LSP, etc.) and builtinto a scheduler so the overall traffic 5520 may be shaped. Without knowing what is in themiddle of the packet network 5502, the traffic shaping system 5500 can track the packets 30 received at the access node 5516 and use that data at the BRAS 5504 to rate shape the entiregroup 5522 of customers 5510. This rate shaping preserves the cross-utilization of bandwidth 115 between the customers 5510 on access node 5524 and minimizes the packet loss due tobursting and mass calling events that might occur in the network.

In one aspect, the traffic shaping function performed by the network traffic shapingsystem 5500 may be based on cutting the non-real time data flows versus the real-time data 5 flows. In another aspect, the traffic shaping function performed by the network traffic shapingsystem 5500 may be based on QoS requirements and obligations to determine which data flowsto drop and which data flows to keep transmitting. In yet another aspect, the traffic shapingfunction performed by the network traffic shaping system 5500 can also look at the (i)DIFFSERV protocol marking, (ii) recipient’s IP address, and (iii) trunk from which data is 10 being received to determine what flows to shape or drop altogether.

Generally, the present network traffic shaping system 5500 maps everything to a Media

Access Control (MAC) address, an Ethernet virtual circuit, a PPPOE tunnel, a PPPOA tunnelor other similar structures. These locations are considered the egress points. Some or all of thefollowing criteria may be used to determine how data i s to be rate shaped. As discussed above, 15 the present network traffic shaping system 5500 can determine what port the data came from.For example, a determination can be made as to whether the port is an Internet data port or aVOIP data port. In one aspect, the present network traffic shaping system 5500 may determineto drop the traffic coming from one port or shape one port to another port. In a second way,the present network traffic shaping system 5500 may have two virtual circuits going down to 20 the CPE or DSLAM 5506, so it may shape one of them versus the other. In a third Way, thepresent network traffic shaping system 5500 may have a large bandwidth going down to theCPE or DSLAM 5506, but it may use a priority bit marking in the packets to choose whichtraffic to shape or drop. In a fourth way, the present network traffic shaping system 5500 maydetermine the type of packet that is sent through the packet network 5502. For example, if the 25 packet is a real-time VOIP packet and another packet is a FTP packet, then it may decide todrop the FTP packet and transmit the VOIP packet. In yet another example, the presentnetwork traffic shaping system 5500 may determine that a particular user 5510 has a multipleof IP addresses for a particular customer 5510 and decide to not transmit for a period of timeon one of those IP addresses. 30 FIG. 56 illustrates an embodiment of an exemplary user interface for the present network traffic shaping system 5500. The Normal Rate Cap field 5602 contains values relatingnormal rate capacity of a particular session or data flow for either a particular user 5510 (FIG. 116 55) or a node 5516. The Minimal Cap Rate field 5604 contains values for the minimal ratecapacity for a particular session or data flow. The BW Step Down Method field 5606 and theBW Step Up Method field 5608 each contains values and related algorithms relating to theamount of the step-wise reduction or step-wise increase performed by the present network 5 traffic shaping system 5500 when reducing the bandwidth to either the user 5510 or the node5516. The Threshold Loss to Step Down BW field 5610 and Threshold Condition to Step UpBW field 5612 each contains values and related algorithms for initiating the step-wisereduction or step-wise increase of the Step Down Method field 5606 and the BW Step UpMethod field 5608. 10 FIG. 57 illustrates an embodiment 5700 of an exemplary method for shaping the rate of data traffic through a network based on information from the PIP packet of the present networktraffic shaping system 5500 (FIG. 55). In step 5702, the data flow is initiated. In step 5704,the PIP packet data is collected by a CPE or DSLAM 5506 as discussed herein. In step 5706,the PIP packet data is transmitted from the CPE or DSLAM 5506 to the BRAS 5504. In step 15 5708, the present network traffic shaping system 5500 queries whether the Y.1731 type packet loss threshold has been exceeded as described above. If it has, then the present network trafficshaping system 5500 further queries whether the minimum bandwidth threshold has beenreached in step 5710. If it has, then the present network traffic shaping system 5500incrementally reduces the customer or access node shaping window at step 5712. 20 In step 5708, if the Y.1731 type packet loss threshold has not been exceeded, then the present network traffic shaping system 5500 queries whether the maximum threshold has beenreached in step 5714. If the maximum threshold has been reached, then the present networktraffic shaping system 5500 continues at step 5706, where PIP packet data is collected from theCPE or DSLAM 5506 at the BRAS. In step 5716, the present network traffic shaping system 25 5500 queries whether the bandwidth set-up threshold is met. If the set-up threshold has been met, then the network traffic shaping system 5500 incrementally increases bandwidth shapingat step 5718, as described above.

In another embodiment, an Application-Specific Integrated Circuits (ASIC) directspacket data flow and PIP packets based on their real-time and non-real-time content. FIG. 58 30 illustrates an embodiment 5800 of an exemplary Data Link Layer device 5804 and an ASICdevice 5802 that is associated with an incoming network interface for communicating to anoutgoing network interface. The ASIC 5802 may be capable of communicating switched data 117 to an outgoing network interface also associated with the Data Link Layer device 5804. TheASIC 5802 is designed to separate or buffer particular data flows, such as data flows of datapackets including real-time and non-real-time content. The data flows may be performed overa network interface card (NIC) operating in a computer Ethernet port or pluggable 5 fiber/electrical modules.

The Data Link Layer device 5804 includes one or more incoming network interface ormodule 5812a - 5812n (collectively 5812) and one or more outgoing network interface ormodule 5814a - 5814n (collectively 5814). The network interfaces 5812 and 5814 may becapable of handling packet based and other suitable digital signals associated with voice, 10 video, and data transmissions of a packet switched network. In addition, the Data Link Layerdevice 5804 may also include switching unit controllers, processors, memory, and busesinterconnecting them, as known in the art. The network interfaces 5812 and 5814 may also becapable of communicating with other network interfaces, single or multiple, such as TIinterfaces, El interfaces, Integrated Services Digital Network (ISDN) interfaces, SS7 15 interfaces, Optical Carrier level-3 (OC-3) interfaces, other optical interfaces, any other types ofinterfaces, or combinations of these interfaces.

The ASIC 5802 may also include one or more incoming lines 5816a and 5816b(collectively 5816) and one or more outgoing lines 5818a and 5818b (collectively 5818) thatmay in communication with other devices of the packet network. The ASIC 5802 may also be 20 connected directly to these lines 5816 and 5818 or may be connected via a bus of a suitabletype, such as control, synchronization, power, isolation, serial, and the tike. In oneembodiment, the ASIC 5802 may receive uni-directional, bi-directional, or other serial datastreams incoming from the incoming network module 5812. Moving in the opposite direction,the ASCI 5802 may transmit separated or stored real-time data flows and non-real-time data 25 flows through outgoing network modules 3212. In one embodiment, ASIC 5802' may alsocontain a processors) 5806 memory 5808, such as ROM, RAM, EEPROM, Flash, and the like,and coded logic or software 5810 for performing the operations described herein. Memory5808 may store registers, such as sampling registers and static registers based on the type ofdata flow through the ASIC 5802. 30 In one embodiment, the one of the two outgoing data path lines 5818a and 5818b is used for transmitting real-time data flow and the other for non-real-time data flow. Once the 118 two data flows are separated at the ASIC, they can each be transmitted to their respective lines5818a or 5818b for measuring in accordance with the principles ofthe present invention.

In one embodiment, the ASIC 5802 creates “sampling” shift registers with QoS or othermapping/replication functions. In one aspect, the sampling shift registers measure the 5 “buckets” or total volume or amount of either real-time data content or non-real-time contentthrough a Data Link Layer device 5804 or other network device. The ASIC 5802 may measurethe total volume, as opposed to bit transfer rate, of a particular data flow that includes eitherreal-time or non-real-time content. For example, the ASIC 5802 may measure an interval oftraffic from one of these data flows and then measure another interval of traffic in a “round- 10 robin” manner. A static register allocates a bucket per stream, bucket per flow, bucket perlogical connection, bucket per port, and/or bucket per device. Further, the ASIC 5802 maycontain a scheduler modification to provide actual scheduler performance information on whatflows are being served at what rates. Also, it may include a “settable counter trigger” thatcounts when a packet has a specific TOS, QoS, or other marking. Externally, the ASIC 5802 15 may create a “line state” dongle or inline probe that measures, via any method, and generatesthe PIP packet information in both directions for real-time and total bandwidth with otherpeakedness measures. The term “peakedness” means within-the-hour or moment-to-momentvariations in traffic.

Further, the coded logic or software 5810 of the ASIC 5802 may determine packets that 20 contain real-time data content from those that contain, non-real-time data content based on theport or device that transmitted the packet to the ASIC 5802, the payload of the packet, the p-bitof the packet, header information of the packet, or by any other means commonly known tothose skilled in the art and/or described herein. Further, the ASIC 5802 may store the TOS,QoS, or other service information related to a particular customer or user associated with the 25 packet or flow of packets.

In addition, the ASIC 5802 may characterize the “peakedness” or burst in real-time.The real-time data may be characterized as a “state,” thus making it useful for mathematicalcalculations and algorithms to determine the amount of real-time data content transmitted. FIG. 59 illustrates an embodiment of an exemplary method 5900 for determining the 30 amount of real-time data flow and non-real-time data flow with an ASIC 5802. In step 5902, the data is transmitted to the ASIC 5802 in any manner commonly known to those skilled in the art and/or described herein. In step 5904, the ASIC 5902 determines the packets that 119 contain real-time content and packets that contain non-real-time content. In step 5906, the real-time content packets are separated from the non-real-time data packets based on features of the packets or sender information associated with the packets as described herein. In step 5908, the volume of real-time content is measured in buckets or other means. In step 5910, the 5 volume measurements of real-time content may be transmitted to other devices or systems foruse in adjusting these devices and systems to optimize the real-time content flow through anetwork in step 5912. In addition, billing considerations can be made based on the totalvolume of real-time data content measured at the ASIC 5802. FIG. 60 illustrates an embodiment 6000 of a method for using information contained in 10 PIP packets to control packet traffic flow with UDP. In step 6002, a QoS or traffic problem isidentified at a node in the Data Link Layer 3104. In step 6004, a check on NEs Data LinkLayer 3104 capabilities is performed. In step 6006, the fault identification process refreshes atcertain intervals. In step 6008, the NE may control the flow of packet transmission using UDPand buffer size. In step 6010, fault information is injected from the Data Link Layer 3104 into 15 the Transport Layer 3108. In step 6012, the nearest NE with Transport Layer 3108 (UDP) andsufficient buffer space is identified. In step 6014, the NE receives all clear (no fault)information from Data Link Layer 3104. Finally, in step 6016, the NE resumes thetransmission rate using UDP.

The system and methods of the illustrative embodiments allow bandwidth allocation, 20 resource management, and troubleshooting across Ethernet or communications domains.Network performance information about the line state of the Ethernet network may be used inconjunction with or to adjust Connection Admission Control (CAC) policies and devices inreal-time such that bandwidth and services across Ethernet domains are controlled. Thenetwork performance information may also be used to isolate nodes that are failing or sources 25 of trouble in order to make network corrections. The changes, adjustments, fixes, work-arounds that may be implemented are available across communications domains elements withdifferent operators and equipment in a carrier grade Ethernet network. Various access nodes,such as Broadband Remote Access Servers (BRAS), Broadband Digital Loop Carriers(BBDLC), Cable Modem Termination Systems (CMTS), or routers and switches, may use the 30 illustrative methods to manage services and/or bandwidths by tracking both the sharednetwork-side trunk state and the individual subscriber-side line states as a state repository forthe network systems. 120 FIG. 61 is an example of an Ethernet network 6100 in accordance with an illustrativeembodiment of the present invention. FIG. 61 illustrates the Ethernet network 6100 with anumber of access nodes 6101, 6102, 6104, and 6106 in communication with ConnectionAdmission Control (CAC) engines 6108, 6110, 6112, and 6114. Each of these CAC engines 5 6108, 6110, 6112, and 6114, devices, or elements is connected to a data stream 6116 that communicates between and through the access nodes 6101, 6102, 6104, and 6106 using packetstreams 6118, 6120, 6122, and 6124.

Ethernet is a network protocol and local area network (LAN) technology used forsending and receiving data packets across the Ethernet network 6100. CAC engines 6108, 10 6110, 6112, and 6114 control and adjust the connection bandwidth in order to accommodate the necessary communication stream. CAC engines 6108, 6110, 6112, and 6114 may behardware and/or software elements or process performed thereby to take actions duringconnection initiation or re-allocation for strategically controlling congestion. Frequently, theCAC engines 6108, 6110, 6112, and 6114 may be used to determine whether or not to allow a 15 new connection, throttle bandwidth, or to load balance across the Ethernet network 6100. TheCAC engines 6108, 6110, 6112, and 6114 may communicate messages, alerts, alarms,commands, data, and other information with one another. In one embodiment the CACengines may contain transmission path state real-time bandwidth and other PM information.The CAC policy engine may include a threshold trigger based upon the PM information or the 20 CAC engine may dynamically change over-subscription rules for the bandwidth reservationportion of the CAC engine. These new states and state triggers are equivalent to CAC enginestates that may be polled or otherwise interacted with by EMS systems and other protocols. Inanother embodiment, a connection may be accepted only if sufficient resources are available toestablish the connection end-to-end with its required quality of service. For example, in one 25 embodiment, for a new connection to be accepted, the contractual quality of service of existingconnections and customers served by the network may not be adversely affected by the newconnection.

In some cases, CAC engine 6108 may be used to control CAC engine 6114.Provisionally, the CAC engines may be applicable to a port or a transmission path. Each CAC 30 engine 6108, 6110, 6112, and 6114 may specify permissions and authorizations for how andwhen it may be accessed. The permissions may include authentications, passwords, andidentifications so that each CAC engine 6108, 6110, 6112, and 6114 does not have unlimited 121 access to each of the other CAC engines 6108, 6110, 6112, and 6114. For example, if CACengine 6114 receives a bandwidth throttling request from CAC engine 6108, the CAC engine6114 may ensure that the CAC engine 6108 is part of* an authentication list of devices, nodes,EVCs, and elements with permission to adjust bandwidth for CAC engine 6114. 5 The data stream 6116 may include video, data, voice, or other multimedia packet streams. Each of the packet streams 6118, 6120, 6122 and 6124 may represent separateEthernet virtual connections that send and receive packets through the data stream 6116. TheCAC engines may control the packets being placed on the data stream 6116 or being taken offthe data stream 6116 and sent to a user or customer. 10 FIG. 62 is an example of an Ethernet network 6200 in accordance with an illustrative embodiment of the present invention. Ethernet network 6200 may be a particularimplementation of Ethernet network 6100 of FIG. 61. Ethernet network 6200 includesEthernet domains 6202 and 6204, maintenance endpoints 6206, 6208, 6210, and 6212 andmaintenance intermediate endpoints 6214,6216,6218, 6220, and 6222. 15 The Ethernet domains 6202 and 6204 represent Ethernet networks controlled by separate operators that have maintenance end-points 6206, 6208, 6210, and 6212 as defined bythe IEEE 802.1AG standards. The maintenance end-points 6206 and 6208 are in the firstEthernet domain 6202 and the maintenance end-points 6210 and 6212 are in the Ethernetdomain 6204. In an illustrative embodiment of the present invention, information that 20 traverses the entire network between maintenance end-points 6206 and 6212 may be availableat every end-point and intermediate point that is connected between maintenance end-points6206 and 6212. For instance, maintenance intermediate end-point 6216 may have informationfrom itself as well as maintenance intermediate end-points 6214, 6218, 6220, and 6222 andmaintenance end-points 6206, 6208, 6210, and 6212. The information may describe the total 25 packet rate or real-time data packet rate, average packet rates, packet rates for the streams fromaccess node users and any other statistics related to the communications capability and healthof the network. The network performance information may be contained in the Ethernet layer2 real-time packet flows. FIG. 63 is an example of a CAC engine configuration in accordance with an illustrative 30 embodiment of the present invention. FIG. 63 includes data stream 6302, access node 6304,CAC engine 6306, network performance information 6308, user packet stream 6310, customer6311, line state information 6312, and correlation engine 6314. The network performance 122 information 6308 may be updated from the line state information 6312 available on a networknode, store in a table on the network, or extracted from one or more packets in the data stream6302, such as a PIP data packet. The network performance information 6308 may includenetwork statistics, including performance, such as real-time bandwidth, of packets including 5 real-time content. For example, the network performance information 6308 may specifystatistics calculated from the line state information 6312 including a provisioned rate, real-timepacket rate, and average packet rate, real-time and total bandwidth usage. The line stateinformation 6312 may represent the data and information obtained from operationmeasurements as described by the current invention. 10 The user packet stream 6310 is controlled by ihe CAC engine 6306 allowing data to be sent and received between the customer 6311 and the data stream 6302 as determined by thenetwork performance information 6308 and the policies established by the operator of thecommunications network via the CAC engine policy modification based on the Performanceinformation 6308. 15 In one embodiment, the access node 6304 may include a network cache that stores information, such as movies, songs, games, and other data, that may be accessed by the CACengine 6306 and delivered to the data stream 6302 for immediate use by other network users.The network cache may also store network performance information for historical use andsubsequent reference. For example, if a node within the network has repeatedly had problems, 20 the historical data may be used to link the problem to certain events, parameters, or factors.The CAC engine 6306 may determine the adequacy of the data stream 6302 to be able toaccommodate the current movie or other data packets requested by a user on the network. Inaddition the CAC engine 6306 may determine if the data information requested waslegitimately requested and authorized as set by previously determined network policies. 25 FIG. 64 is an example of PIP packet flow of network performance information in accordance with an illustrative embodiment of the present invention. Packet flow 6400 of FIG.64 shows the network performance information obtained using the teachings of the presentinvention. Packet flow 6400 illustrates data and PIP packet flow across access nodes 6402,6404, and 6406, network performance information 6408, 6410, and 6412. The PIP packet flow 30 6400 includes information about the network line state at intermediate points and at end-points in the network. The PIP packet flow may be a particular implementation of data stream 6302of FIG. 63. 123

As the network performance information 6408 is passed through each access node6402, 6404, and 6406 in the packet flow 6400, additional information is added by each accessnode 6402, 6404, and 6406. It is understood that each network segment and path may have aPIP perfonnance and utilization PM flow and measure. For example, as shown, initial network 5 performance information 6408 may be a single PIP packet. Later, as the network performanceinformation reaches the access node 6404, additional network performance information 6410may be added. The network performance information 6410 may be combined into a single PIPpacket or multiple PIP packets may be used. At access node 6406, the PIP packet includesnetwork performance information 6408, 6410, and 6412. Information may be added for any 10 number of access nodes despite the limited examples shown.

As a result, the PIP packets are passed from and to or through each of the access nodes, devices, and other elements of a network communication system. Thus, at any given accessnode the network performance information for each prior node may be easily ascertained andanalyzed as needed. Similar information on the performance of the network may also be 15 available in like manner from a PIP packet flow 6400 going the other direction because PIPpackets may flow in both directions in the network. Alternatively, network performanceinformation may be obtained from or utilized in a central database, EMS server, NOC, CCM orother central resource in communication with a CAC engine or access node, FIG. 65 is an example of stored network performance information associated with 20 access nodes in accordance with an illustrative embodiment of the present invention. FIG. 65details the network performance numbers and statistics that may be available or stored at eachaccess node 6502, such as access nodes 6402, 6404, and 6406 of FIG. 64. In anotherembodiment, the network perfonnance information table may be a compilation of data storedin a central network device, general state engine, or other element or component that is 25 accessible by different nodes and processors within the network. The network performanceinformation table may also be stored in the correlation engine 6314 of FIG, 63. Thecentralized table may alternatively be updated when the network experiences problems, othertables have data overflow, or processing elements are unable to process the networkperformance information fast enough. The correlation engine may send alerts or alarms to 30 access nodes or a network control center to correct or troubleshoot network issues.

The network performance information which may include numbers and statistics may be part of a table or matrix that describes the network in terms of packet count, packet delay, 124 packet loss for the total, real-time, and/or non-real-time flows, total real-time, and non-real-time bandwidth, effective packet rate, jitter, latency, out-of-order packets, quality of service,carrier identification, or other parameters 6500 that describes the important characteristics ofthe end-to-end network. The different numbers, values, and measurements may be used tocalculate in the overall quality of the network by viewing these numbers singularly at eachaccess node or collectively as a network packet loss statistic or other parameter. The networkperformance information may be collected over time to provide an average time-boundnetwork value.

In another instance, numbers, such as the provisioned and or available packetbandwidth rate (pn) and the real-time bandwidth packet rate in use (mn), may be used toprovide a measure to individual CAC engines that may indicate that access to the networkshould be accepted or declined. For instance, if the network performance value: (pn - mn) ispositive, it may indicate that there is capacity on the network for the access node packet streamnumber n. If the value (pn - mn) is negative or zero, it may indicate that the CAC enginedeclines the packet stream from access node n.

In another instance, individual network values such as the real-time bandwidth packetrate (kn) may be averaged over the network to provide a real-time average network packet ratesuch as: ((kl + k2 + kn)/n). The real-time average may be used to provide a real-time usestatistic to an overall monitoring center that may automatically, or through operator assistance,admit or reject additional packet streams through the CAC engines. For example, the real-timeaverage may be used to reject additional packet stresims requesting to join a certain programevent that is oversubscribed at the server. As a result, additional users may be rejected. Itshould be understood that packet rate can be bandwidth in use, packet counts, or a combinationof both.

In another embodiment, a remote server, such as a “video on demand” server, may bededicated to providing video files for any number of customers. In response to a CAC enginereceiving network performance information, via, for example, PIP packets, for node segmentsassociated with the remote server indicative of the real-time bandwidth and other performanceinformation, such as packet loss or congestion, are used to obtain a utilization performancemeasure in relation to the assumed usage and oversubscription rates or other performanceissues associated with receiving content from such remote server. The CAC engine maythrottle the allocation of bandwidth to devices requesting content thereby modifying the 125 bandwidth that may be accessible by each customer, IP address, or other element. Such CACengine may be located at an IP service or access broadband node gateway point used by theremote server or at access points used by CPEs to access content from such remote server overa packet network. For example, CAC engines with the appropriate permissions may be used to 5 throttle the bandwidth of the remote server or to specify priorities. For example, a CAC enginein a remote location may be used to specify that a first CAC engine or a specific networkdevice of high priority may have unlimited bandwidlh access to the remote server, but all otherCAC engines, CPEs, or specific network devices may only have a designated percentage ofavailable bandwidth. 10 FIG. 66 is a flowchart of a process for allocating network resources in accordance with an illustrative embodiment of the present invention. The process of FIG. 66 may beimplemented by an CAC engine.

The process begins by gathering network performance information regarding line andtrunk transmission performance and utilization states (step 6602) and/or other network 15 performance information. The network performance information may include performancenumbers, data, utilization information, or statistical information calculated therefrom regardingreal-time and non-real-time data passing through the data stream of the communicationsnetwork. The network performance information may be gathered using a PIP packets and PMcollection points that has been updated as it reaches each access node within a network, such as 20 an Ethernet network. The network performance arid utilization information gathered in step6602 may also be stored for subsequent analysis.

Next, the CAC engine controls the network resources (step 6604) in response toreceived network performance and utilization information. The network resources arecontrolled based on the performance information particularly for dynamic resource allocation, 25 diagnosis, and troubleshooting. For example, if a node, device, link, access point, or othernode segment is encountering problems, the VOD session controller redirects the CAC engineas it may reroute the IP Service point traffic around the node experiencing problems viaaddressing, or server name response. The CAC engine may also perform load balancingbetween different CAC engines. For example, a single customer may be connected to the 30 communications network through different CAC engines. Based on traffic between thecustomer through the different streams, load balancing may be performed so that bandwidth ismore efficiently utilized across the CAC engines. The same types of balancing is commonly 126 performed using current protocols and applications, such as Bit Torrent. These conventionalprotocols were designed to acquire portions of conte nt, such as a movie from multiple sources,concurrently. These protocols by-pass the rate limiting effect imposed by egress rate shapingfunctions at VOD or content servers. These protocols can have significant affect on the 5 performance of the aggregate path. PIP packets can detect the impacts of the use of these typesof parallel protocols and be used to invoke any of the traffic management functionalitydescribed in accordance with the principles of the present invention.

For purposes of this example, load balancing may refer to a throttling of bandwidth bytwo or more VOD servers responding to CAC engines and/or the routing of traffic or sessions 10 by two or more VOD servers responding to CAC engines transmission path utilization andperformance state information to even out traffic that is directed at two or more access pointsto a network. Thus, a customer network may access a larger packet network, such as theInternet, through connections associated with network access points. The bandwidth of datapassing through each network access point may be controlled by one or more CAC engines 15 working in concert basing load balancing on the transmission state information for the pathsunder the governance of reach CAC engine. To balance the amount of traffic through each ofthe network access points, one or more CAC engines may cause traffic intended for one ofsuch network access points that is approaching full load or a overloaded state to be rerouted orredirected to another network access point that is not experiencing as much traffic. The one or 20 more network access points may be access points to the same network, such as the Internet, ormay alternatively access different packet networks. However, even if such network accesspoints are egresses from a customer network into two different packet networks, both of suchpacket networks may eventually allow a connection to an IP address or other network addresslocated outside of the customer network. For example, one of the network access points may 25 allow egress into a first network that contains the IP address to which a data communication isaddressed, while the other network access point may allow egress to a second network that isthen connected to a third network, that is, in turn, connected to the first network including theIP address to which a communication is addressed. In such a manner, even if network accesspoints and associated CAC managers are not connected to the same external network, both 30 such network access points may allow egress of a data packet in a manner that allows the data packet to eventually be communicated to the target II3 address. 127

Although the foregoing is described generally with respect to rerouting of traffic andthe distribution of new sessions based on actual performance and utilization information of thetransmission paths the system intelligently distributes the bandwidth across two or morenetwork access points by one or more CAC managers, it should be understood that more 5 complicated schemes of load balancing can be utilized that involve algorithms associated withbandwidth reservation and allocation, the throttling of bandwidth, the rerouting of traffic toalternative network access points, known connection paths to an IP address located outside ofthe network through one or more network access points, or any combination of the foregoing.In yet another embodiment, load balancing may include a determination of particular 10 application data included in packets intended to be communicated through network accesspoints such that such load balancing may be accomplished not only with respect to trafficgenerally, but with respect to traffic associated with a particular application or class ofapplications. For example, the most readily apparent example of the need for such specificload balancing may be with regard to the load balancing of real-time session packets associated 15 with applications that perform real-time or near real-time content communications. Thehandling of real-time packets may cause more network performance issues than the handling ofnon-real-time packets as a result of the need to minimize latency and jitter associated with suchreal-time packets. In this case, the goal of the CAC function is to balance the load of real-timetraffic across multiple network paths to optimize network performance. It should be 20 understood that other real-time flows may exist in foe egress trunk of which the CAC enginemay not have knowledge. In order to balance foe real-time traffic, the performance andutilization state information may be present in or accessible to the CAC engine. Thus,complex load balancing schemes that take into account real-time data packets and non-real-time data packets in balancing foe two categories of data packets across one or more network 25 access points by CAC managers that take into account network performance informationregarding external networks and the connection paths available therein may greatly enhancethe user experience associated with applications using real-time content and offer a unique wayto respond to performance issues identified from network performance information in order toaddress such issues and enhance the general performance of the network. 30 In one embodiment, a CAC engine may manage one or more additional CAC engines within the network or in a secondary network in response to received network performanceinformation. For example, a CAC engine may control bandwidth usage in an interconnected 128 network. For example, the network performance information or instructions to a CAC enginelocated in another network may be carried or “piggybacked” from a first network to a secondnetwork, via, for example, PIP packets for allowing a secondary CAC engine to control a CACengine in the first network. 5 In one embodiment, the CAC engine may throttle sessions or restrict the amount of allocated bandwidth (for a smaller codec) based on the amount of bandwidth available for acustomer to access a network (step 6606) in response to received network performanceinformation. Bandwidth requests may be granted, throttled, and bandwidth reserved andallocated based on external or internal factors that are affecting the communications network. 10 For example, bandwidth may be throttled based on interference through a CAC engine thatrelies on a wireless transmission point. The bandwidth through the CAC engine may bethrottled to accommodate the available connection speeds and limiting factors of the wirelesstransmission point. A CAC engine reserves and allocates bandwidth for customers at a network access 15 point. For example, the customer may have a service level agreement or quality of servicerequirement specifying certain parameters and resources for which the customer is paying. Forexample, the customer may have reserved 10 megabits/second for real-time streaming video.If the bandwidth of real-time data packets dedicated for the customer is running at 12megabits/second, the CAC engine may throttle or adjust the customer stream so that only 10 20 megabits/second is provided to the customer. As a result, bandwidth may become available toother customers that are paying more for increased bandwidth or a guarantee that bandwidthwill be available from the communication service provider at any time. Some service levelagreements and quality of service provisions allow for refunds or discounts if available rates orbandwidth levels drop below specified thresholds as provided to the customer by the 25 communication network service provider.

In another embodiment, the additional 2 megabits/second of bandwidth may be allocated on a “best efforts” or similar non-guaranteed basis. As a result, if bandwidth isavailable, the customer may be provided the entire 12 megabits/second, otherwise the customeris provided only the 10 megabits/second that arc; guaranteed the customer. In another 30 embodiment, the customer may be provided the entire 12 megabits/second and is charged a premium rate for the data overage. The customer may also be billed for the amount of time

that the bandwidth used exceeds 10 megabits/second. The updates may be sent from the CAC 129 engine to a billing database. The customer service level agreement may specify different rates, charges, guarantees, quality of service, or service level agreements for both real-time and non- real-time content.

The network performance and utilization information located in the CAC engine may5 also be used to enforce usage limitations, or track customer bandwidth to identify an IP addressthat is monopolizing or overusing resources. Once the customer or IP address found to be aover-using resources has been located via threshold mechanisms set on the line performanceand utilization state information for that line, by the CAC engine or other network process ordevice, the CAC engine may throttle sessions, shut down or alter shaping windows for that 10 customer flow, or send a message to session controllers or customer GUI interfaces to providea usage warning message, or otherwise limit the customer’s access to the network in order topreserve bandwidth and bandwidth availability across the network or meet business objectives.For example, a student in a dorm that is streaming too much real-time data for a movie mayhave their real-time bandwidth limited to provide bandwidth for other students or customers. 15 In another example, the bandwidth percentages or rates available to a customer may be increased in step 6606 in response to received network performance information. Thebandwidth available for customers across the Ethernet network may be dynamically adjustedbased on service level agreements, guarantees, performance representations, type of packets,performance indicators, and other parameters or factors. The bandwidth may be adjusted for 20 access nodes, customers, devices, software applications, and IP addresses. In many cases, thethrottling of step 6606 is performed based on the type of data, including real-time and non-real-time data. For example, a customer may have desired rates and percentages of dedicatedbandwidth for real-time and non real-time data packets. In many cases, real-time Voice overInternet Protocol (VoIP) may be considered a higher priority than regular Internet traffic. Step 25 6606 may also shift bandwidth and network traffic for load balancing between access nodes for better network performance.

Throttling requests and other configuration and maintenance changes implemented by aCAC engine may be implemented by inserting commands or other data in the PIP packet. Byinserting the changes to be made in the PIP packet, the CAC engine may perform the changes 30 in-band and does not need access to an out-of-band communication connection in order toimplement the desired allocation and reservation. Alternatively, allocation and reservation 130 changes made by a CAC engine may occur out-of-band using an alternative communicationline or medium.

The PIP packets may also include data for load balancing between CAC engines, accessnodes, and other communications elements. The network performance information, control 5 signal communication, and/or PIP packets may also be sent to access nodes, CAC engines, anetwork operation center using enhanced messaging service (EMS) or other messagingprotocols. The PIP packets may also specify real-time thresholds, percentages, and parametersthat may be used to regulate the communications network. The PIP packets may addreservation and allocation information that regulates the control process by customer, 10 identification, IP address, or program application. For example, a PIP packet may specify thata CAC engine is to dedicate five percent of available bandwidth to real-time data from IPaddress 128.063.254. FIG. 67 is a flowchart of a process for correcting failure of network resources inaccordance with an illustrative embodiment of the present invention. The process of FIG. 67 15 may be implemented by an access node.

The process begins in step 6702 by gathering network performance information regarding line state. Step 6702 may be performed as previously described in step 6602 of FIG.66. Next, the access node compares thresholds against the network performance informationin step 6704. The comparison of step 6704 may be performed by a correlation engine that is 20 part of the access node or independent from the access node. The network performanceinformation may be compared against a table, matrix numbers, or statistics. The results of thecomparison may be compared against rule-based statistics. The results may be used todetermine the status and performance of the communications network including software andhardware components within the network. 25 The access node determines whether there is an access node experiencing failure in step 6706. The determination may be made based on the comparison of the network performanceinformation to thresholds in step 6704. If there is not an access node experiencing failure, theprocess terminates. If there is an access node experiencing failure in step 6706, the accessnodes corrects the failure for the access node in step 6708 with the process terminating 30 thereafter. In one example, if a problem or failure is detected in step 6706, the problem may be corrected manually or automatically by a network control center. For example, if there is a failure, the access point may send a correction message, alert, or alarm to the network control 131 10 15 20 25 center so that the failing or problem node may be fixed with a software patch, reboot,maintenance order, replacement, or work-around.

The network corrections may occur in any number of ways based on the parameters,designated preferences, rules, and policies. The corrections may be temporary or permanentfixes. The solution to the failure may not always be easily fixed. In one embodiment, thefailure may be corrected by sending a notification to other networks that are dependent on thenetwork. In another embodiment, the traffic may be rerouted through different CAC engines,access nodes, or networks in order to correct network issues.

In another embodiment, the network correction may involve rerouting data throughdifferent CAC engines, networks, and access nodes. The data may be rerouted to preservequality of service and performance of the network. The CAC engine may also request orgenerate trace information for the failing access node. The CAC engine may also request thatthe failing access node or surrounding access nodes or device provide additional information.The additional information may supplement the data provided by the PIP packet to diagnoseand remedy the problem. The CAC engine may also ping the network access nodes todetermine how data and information is flowing within the network and whether each accessnode is available. For example, a number of access nodes may be systematically pinged by theCAC engine to determine which nodes are still responsive.

The CAC engine may broadcast a message out to other access nodes, a NO'C, EMSsystem, CCM, or other network device or process that identifies the failing access node and theassociated problem. The message may specifically indicate the failing node, networkperformance information for the failing node, and a network solution, remedy, or work-around,if applicable. The message may be sent specifically to a network operation center,performance log or table, or a rule based engine that may use network topology to select orprovide solutions for the problem. The rule-based engine may also be used to select the nextstep taken address the problem, such as send a text message to a network administrator.

The network performance information may be archived or stored. The historical datamay be used to reserve and allocate bandwidth changes. The historical data may be used toimplement different control and algorithm changes based on an analysis of the historical data,For example, the historical data may reflect particular network performance issues that occur atspecific times of day or in response to specific events, Such specific network performanceissues can then be addressed by changing the throttling, reservation, and allocation scheme of 30 132 particular CAC managers associated with particular network access points during such times or prior to such events. Thus, a CAC manager may respond to reservation requests associated with a particular network access point during particular times of the day with the allocation of bandwidth to requesting IP addresses that is less than the allocation that is made at other times 5 of day. For example, a CAC engine may respond to bandwidth reservation requests byallocating only 50% of the bandwidth requested in such an allocation request.

Such historical data may also be used to implement particular load balancingalgorithms between different network access points using one or more CAC managers. Forexample, if a particular network access point receives a number of requests for bandwidth 10 allocation associated with real-time data packets for an application using real-time content,such as a video conferencing application, allocations associated with such real-timeapplications may be directed alternatively to a different network access point based on theparticular load balancing algorithm. Thus, historical data related to network performanceinformation can be stored by CAC manager and utilized to change bandwidth throttling, 15 reservation, and allocation algorithms that take into account the level of granularity of networkperformance information obtained by such CAC manager. ln one embodiment, the CAC manager may completely block reservation requestsreceived that are associated with particular IP addresses, applications, or network protocols. Inyet another embodiment, a CAC manager may receive network performance information 20 associated with enhanced levels of jitter experienced in the external network, and in responsethereto, may limit requests to reserve bandwidth that are associated with a SIP protocol or itselfrequest that a particular network device located at a particular IP address utilize a lower ratecodec than may have previously requested before allocating bandwidth in response to suchbandwidth reservation requests.

25 As previously discussed, one CAC engine may communicate with one or more CAC engines located elsewhere within a customer network or even within an external network. Thehistorical data may, therefore, also be used to request or command that alternative throttling,reservation, and allocation algorithms or instructions be utilized by other CAC managers. Forexample, corrections for different nodes, devices, or elements of the communications network 30 may be made to avoid disrupting as few customers and network traffic as possible.

In one embodiment, the CAC engine may access a table of network performance information that is stored within the engine or that is stored remotely within the network or 133 within another communications network. The CAC engine may access data within the table toidentify the problem for troubleshooting purposes.

In one embodiment, network performance information that is obtained using PIPpackets, from any of the tables disclosed herein, or any other manner, may be utilized to 5 change the way in which future network performance information is gathered, collected, oranalyzed. For example, if an analysis of network performance information reveals a problemin a particular network or portion of a network then the frequency at which PIP packets aresent through that portion of the network may be increased. Alternatively, the level of networkperformance information collected by such PIP packets may be increased such that more data 10 and more types of data are collected by PIP packets as they are routed through that network orportion of the network. In yet another embodiment, the normal routing of PIP packets througha network may be changed to avoid a severed communication link, obtain more informationabout a particular portion of the network encountering problems, or to send instructions to suchportion of the network to cause such portion or the notes therein to collect additional 15 information, run diagnostic routines, otherwise troubleshoot the problem, or implementchanges in the ways that network devices within such portion of the network are configured oroperate. For example, a particular network node such as a switch may be instructed to beginparticular congestion control behavior, reroute traffic, or any other appropriate solution orwork-around to a particular problem identified by network performance information.

20 In one embodiment, a network device may be instructed by information within a PIP packet to reboot itself, refresh a routing table, increase the amount of data buffered within suchnetwork device, or otherwise begin, change, or terminate any process or operation within suchnetwork device. In one embodiment, the control regimen for managing PIP packet flow maybe changed entirely in response to a change in network performance information. For 25 example, a network or portion of the network over which PIP packets were being sent everyfive seconds may be increased in frequency to every one second or tenth of a second to collectmore information about a particular network performance problem.

Alternatively, in an embodiment that does not include the use of PIP packets, eachnetwork node may be instructed to update its own internal table or update a table of network 30 performance information used as an essential resource by one or more networks on a morefrequent basis or with increasing levels of detail of network performance information. Inanother embodiment, when PIP packets or other updates regarding network performance 134 information were previously generated only in response to events or triggers or requestsreceived from another network device, such network node may instead be instructed togenerate such PIP packets or other updates on a routine basis until a problem has beenresolved. All the foregoing may be implemented at any time in response to a trigger, or may 5 alternatively be scheduled at a particular time based on an instruction. Similarly, the entire PIPpacket system and/or system of updates regarding network performance information mayrevert back to a normal mode of operation after a network performance problem has beenresolved, if a period of time has elapsed, or it receives additional instructions from a centralnetwork resource or other network noted device. 10 As a result of the foregoing or as the result of other events not described above, the payload of PIP packets may be altered. For example, the PIP packet may be instructed toobtain network performance information at a node that was previously a pass-through nodefrom which network performance information was riot obtained. In another embodiment, PIPpackets may include a payload of specific instructions to a network device at a particular 15 network node to change the operation of such network node. In yet another embodiment, a PIPpacket that was originally instructed only to obtain network performance information withregard to latency, packet loss, and jitter, may instead be instructed to obtain a full descriptionof all network performance information that is available, alternative network performanceinformation, or some level of network performance information between full network 20 performance information and the limited amount of network performance information it wouldnormally receive.

The use of injecting instructions into a PIP packet allows the use of PIP packets asmuch more than a simple reporting system of network performance information. Instead, itallows an inband system for sending instructions, initiating processes, or otherwise configuring 25 the parameters or operation of a network. Although not expressly described herein, theforegoing schemes to modify PIP packets or other methods of reporting network performanceinformation may be directed through a network outside of the packet network (such as LAN)generating the PIP packets or otherwise seeking to obtain network performance information.For example, if a particular network is experiencing delays, packet loss, or jitter in a large 30 number of data packets that originate from a particular outside network, the network canincrease the number of PIP packets that are sent to such outside network in order to obtainnetwork performance information from such network or may request that such outside network 135 generate more of its own PIP packets directed to such network from such outside network. Insuch a manner, the network may ascertain specific performance problems in the outsidenetwork, and if necessary, reroute calls or other data communications through other networksinstead of such outside network. 5 In one embodiment, network performance information may be utilized by a processor to determine the best gateway, access point, network-to-network interface, or other networkegress point for a particular data packet or session of data packets. For example, if a particularnetwork has five different egress points to an outside network, such as a PSTN, such fiveegress points represent five different ways in which a VoIP call or other communication may 10 be routed in order to access such PSTN or other outside network. Within the network itself,there may be five network connection paths between a customer access point making a VoIPcall connection request and each of such egress points. Thus, there may be twenty-fivedifferent routes in which a VoIP call may be connected from the customer access pointsthrough the network in order to communicate with the PSTN or other outside network. An 15 EMS system, inter-network system, CCM, router, or other network device may utilize networkperformance information to determine which of such twenty-five potential connections pathswill offer the best quality of service for the customer attempting to connect the VoIP. Suchnetwork device may also consider which of the twenty-five connection paths least negativelyaffects the performance of the remainder of such network. 20 Such decision may be made in response to known information about network performance outside of the network within the PSTN. For example, if one of the egress pointsis known to be in a portion of the PSTN experiencing significant performance issues, the fivepotential connections associated with such point of* egress may be avoided. Likewise, if tenout of the twenty-five potential connection paths go through a common core switch within the 25 network that is experiencing problems, those ten options can likewise be discarded.

Each of the twenty-five potential network, connection paths between a customer’s access point and the PSTN may be rated, graded, or otherwise compared to determine the bestpossible connection path for either customer quality service and/or general networkperformance. For example, the present application discloses different ways to rate particular 30 node segments of a network by assigning a particular rating to them. Different ratings can be assigned to each node segment based on different criteria such as latency, jitter, packet loss, 136 percentage of real-time traffic, real-time bandwidth, or any other parameter. Thus, each nodesegment may have different grades or ratings in different areas.

Using the one or more grades or ratings for each of the node segments, many differentalgorithms may be used to calculate the best overall path between the customer’s access point 5 to the network and the PSTN or other outside network. For example, ratings associated withjitter could he given one weighting factor and averaged across all node segments locatedbetween the customer’s network access point and the PSTN, which may, in turn, be added toanother weighted factor with respect to latency that represents the average latency grade acrosseach of the node segments between the customer’s network access point and the PSTN. In 10 another embodiment, connection paths between the network access point and the PSTN may beexamined to determine the highest or lowest rating that has been given to any node segmentlocated in such path which such highest or lowest rating being then assigned to the entire path.

In some cases, actual measurements may be utilized to determine whether or not to usea path. For example, the average jitter experienced by a real-time application packet passing 15 through a particular node segment may he added to such averaged jitter of each of the, othernode segments located along such connection path to given overall average jitter that may beexperienced by a real-time application communicating packets along such path. Similarly, theoverall latency of a path can be calculated. The overall rating of the twenty-five possibleconnection paths may be compared to each other either in aggregate or based on different 20 factors and otherwise weighted, filtered, or analyzed, to determine the best possible connectionpath for the customer’s quality of service and/or the general performance of the network.Some algorithms may weight the effect on overall network performances being more importantthan the customer’s quality of service as long as, for example, a particular minimum quality ofservice is reached. Algorithms may also take into account the guarantees or service level 25 agreements that the network provider may have with a particular customer such that thenetwork provider may meet its commitments to the customer and does not have to pay to thecustomer any service level credits, penalties, damages, or expenses.

In one embodiment of the present invention, network performance informationassociated with a particular network that has been collected using PIP packets, retrievals from 30 tables of network performance information, or any other means of obtaining networkperformance information may be utilized by a particular application being executed by a clientof such network. In such an embodiment, information from a table or a PIP packet may be 137 communicated into the particular application using the application’s protocol, an injection fromanother OS I layer into the application layer, an XML interface, or any other suitable means.Such application may then have instructions that execute in response to receiving particularnetwork performance information. 5 For example, an application may be a network computer game wherein the gamer is looking for the best possible connection with the least amount of latency and packet loss that isavailable between the gamer’s client and a remote server hosting the on-line game. Theapplication may obtain network performance information in order to itself calculate such bestconnection or instead query a central network resource, such as a VPCE described earlier, to 10 respond with the best network connection possible. In one embodiment, an application maydisplay network performance information to a user of a client. In another embodiment, anapplication may cause an instruction to be injected into a PIP packet or similar packet to obtainspecific network performance information from a remote network or node segment or to causechanges in a remote network or node segment to enhance the user’s experience of a particular 15 application.

In one embodiment, an application may affirmatively monitor network performanceinformation in order to determine a better network connection than the connection it iscurrently using and switch to such better connection as soon as it is detected. Alternatively, aPIP packet or other instruction may be sent to a particular application in response to network 20 performance issues believed to be caused by such application in order to reduce the amount ofbandwidth utilized by such application, change the operation of such application, terminatesuch application, restart such application, or otherwise change the application to reduce anynegative effects on network performance and/or potentially purchased user experience.

In one embodiment, the network performance information and different levels of detail 25 thereof may be utilized with one or more of the various permission schemes described hereinor any other suitable security-based access to network performance information may be used toshare network performance information between two or more networks. In one embodiment,the information, or a subset thereof, from all networks participating in a global network such asthe Internet may be updated to a central resource and shared among all participants in the 30 global network. In another embodiment, networks may share their network performanceinformation with any network to which they have a nctwork-to-network interface. In another 138 embodiment, a network may combine other network’s network performance information withits own network performance information in a table or other central or distributed resource.

Although the permission’s table described herein relative to FIG. 17B is illustrated anddescribed to show access to different levels of network performance information, a similar 5 table may be utilized to establish what access, testing, instructions, commands, and othercommunications an outside network is permitted to make within a particular networkoperator’s network. For example, a particular network operator may allow an outside networkto manage or send certain instructions to CAC managers, or certain CAC managers, locatedwithin the network operator’s network. In another embodiment, a network operator may 10 permit an outside network to control certain aspects of the operation of a device of suchnetwork operators that forms part of a network-to-network interface with such outside network.In yet another embodiment, the network operator may allow an outside network to control aparticular trunk, node segment, or connection path used solely or primarily to route traffic to orfrom such outside network. 15 In yet another embodiment, particular emergency circumstances may change the level of access that a network operator gives to outside networks over its own network devices andnode segments. For example, a particular emergency event may trigger a command to removeall outside network access to a network operator’s network, whether with respect to receivingnetwork performance information, issuing commands and responses thereto, executing 20 troubleshooting or testing routines, or any other communication inconsistent with normal datacommunication and network operation. In response to another event, a network operator mayconfigure permissions to allow an outside network to obtain enhanced or additional networkperformance information from within a network operator’s network.

Any combination of the foregoing may be utilized to allow interconnected networks 25 and the operators thereof to have increased visibility into the operation of each of theinterconnected networks and offer some measure of allowing each network to control certainaspects of network performance to address problems determined by analyzing such receivednetwork performance information.

In one embodiment, the network performance information is collected, shared, and 30 analyzed automatically by network devices and processes and instructions, commands, furthertroubleshooting, work-arounds, or other solutions may be automatically generated usingsuitable ruled-based engines or algorithms to reroute traffic, throttle or block traffic, change the 139 configuration of network devices, or implement any other change in one or more networks toresolve problems associated with the interconnection of data packets across such networks.Different schemes, automated processes, rules, and algorithms may be utilized with respect toreal-time data packet communication as opposed to non real-time data packet communication 5 or overall data packet communication.

Although network performance information is described as being stored in tables andcarried by PIP packets and otherwise monitored with respect to raw network performanceinformation such as actual measurements of packet loss, latency, bandwidth, real-timebandwidth, jitter, or any other measured or detected item of data associated with network 10 performance, such data may also be stored in tables or communicated via PEP packets afterbeing converted, analyzed, summarized, averaged, or otherwise used in a calculation orstatistical analysis to form derived data. Such derived data shall also be considered networkperformance information.

In one embodiment, statistical information may be kept regarding network performance 15 information that is gathered over a certain interval of time. During such certain interval oftime, many measurements of the same data may be made. Different data can be collected anddetermined at the end of such interval. For example, each measurement itself may be stored,an average of a measurement, such as bandwidth, may be stored, a mode associated with themost frequent bandwidth range or value measured may be stored such as for example, the most 20 frequently measured range of bandwidth.

In. another embodiment, the peak bandwidth or other network performance data may be determined and stored. In yet another embodiment, information may be presented as networkperformance information that establishes the percentage of such interval that the bandwidthwas within a particular range. For example, during a measurement interval of one second, 100 25 measurements may be taken in response, for example, to receiving 100 PIP packets at aparticular network node. Such 100 measurements may be analyzed and: (i) an average packetloss, latency and jitter determined, (ii) a peak latency, packet loss, and jitter determined, (iii) amode or mode range of latency, packet loss, and jitter may be determined, or (iv) any otherstatistical measure of any data collected as a result of the receipt of such PIP packet may be 30 determined.

For the same 100 sets of network performance data generated, percentages and rangesmay be utilized to enhance visibility of the network performance information. For example, if 140 50 of the 100 measurements taken for bandwidth of real-time data packets are determined to bebetween zero and 100 Mbps, 20 measurements are determined to be between a range of 100Mbps and 500 Mbps, and the remaining 20 measurements are determined to be greater than500 Mbps. Network performance information may be generated to indicate that during one 5 second interval of time, less than 100 Mbps of real-time data packets will be received 50% ofthe time between 100 and 500-Mbps of real-time data packets will be received 30% of the time,and greater than 500 Mbps of real-time data packets will be received 20% of the time. Asdescribed above, filtering may also be utilized. For example, the five lowest measured and thefive highest measured amounts of bandwidth may be discarded before doing calculations. In 10 another example, measurements may be tracked as a function of time. In yet anotherembodiment, the rate of change of bandwidth, latency, packet loss, jitter, or any othermeasurement may be utilized to generate indications associated with increasing or decliningcongestion, increasing or declining latency, increasing or declining jitter, or increasing ordeclining packet loss from one interval to the next interval. Thus, for example, network 15 performance information may be generated that shows the degree of change in any datameasured by the node between intervals that may be useful to identify trends in networkperformance or otherwise detect declining network performance or declining traffic through aparticular network element that may be indicative of a problem elsewhere in the network.

In one embodiment, an identifier associated with a graphical element may be included 20 as network performance information such that when network performance information isreceived by a network operation’s center, Internet work system, EMS system, or any othernetwork resource or network device, the graphical element may be displayed as an up arrow,down arrow, blockage symbol, impaired symbol, normal symbol, or any other symbolindicative of network performance information. Thus, for example, an operator of such a 25 resource may see displayed an up arrow indicating that congestion at a particular networkswitch is increasing, which may alert an operator to look more closely at the remainingnetwork performance information associated with one or more network devices.

The previous detailed description is of a small number of embodiments forimplementing the invention and is not intended to be limiting in scope. The following claims 30 set forth a number of the embodiments of the invention disclosed with greater particularity. 141 □’Ewan p-wa , crnxan ;w:n ατα inia^a pnow pnszn irn nr -jaoa,ρνιη ηχΏ- paoana mavia na^’maa np’ioa .zrtwan rwaa mp^an ρπίι7 axnria ......Gtoca uer □innn Pi? »*···»· · sw ***** ^_

♦ Q9 Ua/2312 0420:45«KDS .(mcna nannn) cras&'an nwa

<img img-format="tif" img-content="drawing" file="IL196284AD00021.tif" id="idf0001" />