Method and apparatus for network interface card load balancing and port aggregation
Claim Score by NHIP
Abstract
Disclosed is a process for driving a network interface card. The process includes monitoring the status of a plurality of ports connected between a computer and a network. Detecting a failure in one of the plurality of ports connected to the network. Re-assigning data transmitted over the failed one of the plurality of ports to an active port of the plurality of ports selected in a round robin technique . The process further including receiving data over one of the plurality of ports designated as a primary receiving port. Preferably, when the failed one of the plurality of ports is the primary receiving port, the receiving tasks are assigned to a next active port selected in a round robin technique .

Term
Term ended
Expired 30 June 2017, 9.2 years ago.
- Priority
- Filed
- Granted
- Expired
- Today
36 claims: 5 independent, 31 dependent
- 1A process method for driving a network interface card, comprising:providing a multi-port network interface card on a stand- alone computer, the multi-port network interface card having at least three ports for connecting the stand- alone computer to a one network, the at least three ports being capable of transmitting data from the stand- alone computer to the network as transmit ports at the same time to a remote host attached to the network at an aggregated bandwidth of the at least three ports, a selected port from the at least three ports being capable of receiving data from the network as a primary receiving port, wherein the multi-port network interface card is configured to transmit new data over a selected transmit port while transmitting other data over the other transmit ports at the same time so as to improve load balancing of data from the stand- alone computer;initializing each of the at least three ports, the initializing further including, ascertaining the connection type for the ports, and scanning the indices of a lookup table to determine if the ports are active;creating registry groups by selecting ports from the at least three ports;defining start time-out values to assist in the efficient load balancing of active ports;monitoring a status of the at least three ports between the stand- alone computer and the network;detecting a failure in one of the at least three ports that connect the stand- alone computer to the network;upon detecting the failure in one of the ports, re-assigning data to be transmitted over the failed one of the at least three ports to an active port of the at least three ports, the active port being selected in a round robin technique in a manner such that data transfer load is automatically redistributed over active ports;and receiving data over one of the at least three ports designated as the primary receiving port.
- 13A method for transmitting and receiving data over a network from a server computer system, comprising:providing a multi-port network interface card on the server computer system, the multi-port network interface cad having at least three ports for connecting the server computer system to a network, the at least three ports being capable of transmitting data from the computer as transmit ports at the same time to a remote host at an aggregated bandwidth of the at least three ports, a selected port from the at least three ports being capable of receiving data from the network as a primary receiving port, wherein the multi-port network interface card is configured to transmit new data over a selected transmit port while transmitting other data over the other transmit ports at the same time so as to improve load balancing of data from the computer to the network;connecting the server computer system with the network through the at least three ports for communicating with a plurality of remote hosts coupled to the network;transmitting a load of data from the server computer system through the at least three ports to the plurality of remote hosts, the transmitting further including, retrieving a destination address from the load of data, and determining whether the destination address is listed in a lookup table;detecting, by the server computer system, a failure in one of the at least three ports during a transmit to a selected one of the plurality of remote hosts;and redistributing, by the server computer system, the load of data assigned to the failed one of the at least three ports to an available port of the plurality of ports, the redistributing further including, determining if the available port is free by referring to the lookup table.
- 22A computer readable storage media containing program instructions for driving a multi-port network interface card having at least three ports for connecting a stand- alone computer system to a one network, the computer readable media comprising:program instructions for monitoring a status of the at least three ports of the multi-port network card that connects the computer to the network, the at least three ports being capable of transmitting data from the stand- alone computer system to the network as transmit ports at the same time to a remote host attached to the network at an aggregated bandwidth of the at least three ports, a selected port from the at least three ports being capable of receiving data from the network as a primary receiving port, wherein the multi-port network interface card is configured to transmit new data over a selected transmit port while transmitting other data over the other transmit ports at the same time so as to improve load balancing of data from the stand- alone computer system to the network;program instructions for building a port resolution table indexing connections to at least one remote host attached to the network that is communicating through a selected one of the at least three ports, the building further including instructions for, determining whether a host attached to the network is associated with the port resolution table by referring to a timer parameter;program instructions for periodically updating the port resolution table after a predetermined time-out;and program instructions for performing a link check timer routine, the performing further including instructions for, determining if a receiver port of the at least three ports is active.
- 30A computer readable storage media containing program instructions for driving a network interface card having at least three ports for connecting a stand- alone computer system to a one network, comprising:program instructions for monitoring a status of the at least three ports of the network interface card connected between the stand- alone computer system and the network, the at least three ports being capable of transmitting data from the stand- alone computer system to the network as transmit ports at the same time to a remote host attached to the network at an aggregated bandwidth of the at least three ports, a selected port from the at least three ports being capable of receiving data from the network as a primary receiving port, wherein the multi-port network interface card is configured to transmit new data over a selected transmit port while transmitting other data over the other transmit ports at the same time so as to improve load balancing of data from the stand- alone computer system to the network;program instructions for initializing each of the at least three ports, the initializing further including instructions for, ascertaining the connection type for the ports, and scanning the indices of a lookup table to determine if the ports are active;program instructions for detecting a failure in one of the at least three ports that connect the stand- alone computer system to the network;program instructions for re-assigning data transmitted over the failed one of the at least three ports to an active port of the at least three ports selected in a round robin technique such that data transfer load is automatically redistributed over active ports;and program instructions for receiving data over one of the at least three ports designated as a primary receiving port.
- 33Broadest claimClaim Score 32, narrow(NHIP)An apparatus for driving comprising:a network interface card having at least three ports for connecting a stand- alone computer system to a one network , the apparatus comprising: ;an apparatus for driving the network interface card, comprising: means for monitoring the status of the at least three ports connected between the stand- alone computer system and the network, the at least three ports being capable of transmitting data from the stand- alone computer system to the network as transmit ports at the same time to a remote host attached to the network at an aggregated bandwidth of the at least three ports, a selected port from the at least three ports being capable of receiving data from the network as a primary receiving port, wherein the multi-port network interface card is configured to transmit new data over a selected transmit port while transmitting other data over the other transmit ports at the same time so as to improve load balancing of data from the stand- alone computer system to the network;means for initializing each of the at least three ports, the initializing means further including, means for ascertaining the connection type for the ports, and means for scanning the indices of a lookup table to determine if determining which of the ports are active;means for detecting a failure in one of the at least three ports connected to the network;means for re-assigning data transmitted over the failed one of the at least three ports to an active port one of the at least three ports that has been determined to be active, selected in a round robin technique such that data transfer load is automatically redistributed over active ports;and means for receiving data over one of the at least three ports designated as a primary receiving port.
Independent claims5
97 paragraphs in 4 sections, as filed
BACKGROUND OF THE INVENTION
1. Field of the Invention
This invention relates generally to computer networks, and more particularly to software drivers used for controlling network interface cards.
2. Description of the Related Art
Computers are increasingly being coupled together into computer networks. In typical networks, hardware devices known as network interface cards (NICs) are used by computer systems to enable them to communicate over networks with other computer systems. Commonly, NICs are available as plug in devices that are connected to a computer's interface bus (e.g., PCI Bus), or are built directly into a computer's mother board. Traditionally, a computer's NIC enables communication over a single computer network, such as a Local Area Network (LAN), which may include a multitude of switches and network devices. Through these network devices, remote hosts are capable of communicating with other similarly connected remote hosts or to a server computer system. Through these networks, remote hosts are able to, for example, request data from a server computer system, and the server computer system is able to transfer the requested data to the requesting host over the network. In response to the increased demand for resources stored on networked server computers, many NICs are now capable of communicating over more than one network. In this manner, data stored on one server computer system may be accessed by hosts coupled to different networks (e.g., different LANs).
<figref idref="DRAWINGS">FIG. 1</figref> is a graphical illustration of a server computer system <b>20</b> having a NIC <b>30</b> that is coupled to more than one local area network. In this example, NIC <b>30</b> is a single card having four ports, where each of the four ports is connected to a different network. As such, NIC <b>30</b> is connected to a LAN-1 <b>10</b>, a LAN-2 <b>12</b>, a LAN-3 <b>14</b> and a LAN-4 <b>16</b>, that enables an increased number of users to access data stored on server computer system <b>20</b>. In this arrangement, each port of NIC <b>30</b> is capable of processing received (Rx) data (and requests for data) from remote hosts in addition to transmitting (Tx) data to remote hosts connected to one of the LANs shown in FIG. <b>1</b>. Another traditional way of connecting server computer system <b>20</b> to multiple networks is to install a plurality of single port NICs (one for each network) to the interface bus of the server computer system <b>20</b>. As is the case in multiple port NICs, each port of each NIC is capable of performing receive and transmit functions over its dedicated NIC.
Because NIC <b>30</b> is responsible for providing the communication path between a multitude of remote hosts, the loading experienced by NICs installed on a server computer system <b>20</b> are far greater than the typical loads experienced by NICs installed on the individual remote hosts. By way of example, server computer system <b>20</b> typically receives a few short requests from the remote hosts and then responds by transmitting out a great deal of data to the requesting hosts. In addition, because NIC <b>30</b> is only able to transmit out to each network at a fixed bandwidth (e.g., about 10 Mbps/100 Mbps), the server computer system <b>20</b> will unfortunately be tied up responding to data transmit request for extended periods of time.
Another common problem with conventional NICs is that access to server computer system <b>20</b> may be completely blocked-off to an entire network whenever one port fails to operate properly. By way of example, if port P<sub>1 </sub>fails, all hosts connected to LAN-1 <b>10</b> will be prevented from accessing data stored on server computer system <b>20</b>. As is well known, port failures are common, and are typically caused by network overloads, NIC software driver bugs, NIC link interface failures, and NIC hardware failures. In any event, when a failure occurs, a large number of users attempting to access or transfer data between the server computer system <b>20</b> will be unable to accomplish their networking task. Typically, when a NIC port (e.g., port P<sub>1</sub>) failure occurs, the server computer system <b>20</b> is typically shut down to all network traffic (i.e., LANs <b>10</b>, <b>12</b>, <b>14</b>, and <b>16</b>) while network technicians trouble shoot and fix the NIC port failure. Of course, network repairs may disable a network for several hours or even days.
As businesses continue to intensely network their operations, port failures will unfortunately lead to substantial losses in worker productivity. For example, many companies use a central server to network their employees that may be located in several remote office locations. As such, when the port servicing a particular office goes down, that entire office will be unable access shared data stored in the central server. NIC port failures are also common in large corporations such as, banks, hotel chains, and airlines that critically depend on real-time access to data stored on a central server computer. Consequently, NIC port failures are commonly to blame for significant yearly losses in revenue as well as lower customer satisfaction.
In view of the foregoing, there is a need for a network interface card(s) (NICs) that provide increased load balancing transmit throughput to networks connected to the NIC, while providing resilient backup capabilities that enable continues data transferring connections when port failures occur. Further, there is a need for NIC software drivers to intelligently detect port failures and automatically redistribute data transfer loads over functioning NIC ports.
SUMMARY OF THE INVENTION
Broadly speaking, the present invention fills these needs by providing methods and apparatus for increasing throughput in a load balancing manner over a multi-port NIC. Each port of the multi-port NIC preferably has its own link to the network, hub, or switch, which enables the multi-port NIC to reconfigure automatically to recover from a port failure. The multi-port NIC, driven by a smart NIC driver preferably reconfigures the multi-port driver by redistributing the load (both transmit and receive) over the remaining active ports. It should be appreciated that the present invention can be implemented in numerous ways, including as a process, an apparatus, a system, a device, a method, or a computer readable medium. Several inventive embodiments of the present invention are described below.
In one embodiment, a process for driving a network interface card is disclosed. The process includes monitoring the status of a plurality of ports connected between a computer and a network. Detecting a failure in one of the plurality of ports connected to the network. Re-assigning data transmitted over the failed one of the plurality of ports to an active port of the plurality of ports selected in a round robin technique. The process further including receiving data over one of the plurality of ports designated as a primary receiving port. In a preferred embodiment, when the failed one of the plurality of ports is the primary receiving port, the receiving tasks are assigned to a next active port selected in a round robin technique.
In another embodiment, a method for transmitting and receiving data over a network from a server computer system is disclosed. The method includes connecting a plurality of ports between the server computer system and the network for communicating with a plurality of remote hosts coupled to the network. Transmitting a load of data through the plurality of ports to the plurality of remote hosts. Detecting a failure in one of the plurality of ports during a transmit to a selected one of the plurality of remote hosts. The method further includes redistributing the load of data assigned to the failed one of the plurality of ports to an available port of the plurality of ports. Preferably, data is received through one of the plurality of ports designated as a primary receiving port.
In yet a further embodiment, a computer readable media containing program instructions for driving a network interface card is disclosed. The computer readable media includes program instructions for monitoring a status of a plurality of ports connected between a computer and a network. Program instructions for building a port resolution table indexing connections to at least one remote host that is communicating through a selected one of the plurality of ports. The computer readable media further includes program instructions for periodically updating the port resolution table after a predetermined time-out. Preferably, the at least one remote host is assigned host parameters identifying a media access control address of the at least one remote host, a number assigned to the selected one of the plurality of ports, and a timer.
In still another embodiment, a computer readable media containing program instructions for driving a network interface card is disclosed. The computer readable media includes program instructions for monitoring a status of a plurality of ports connected between a computer and a network. Program instructions for detecting a failure in one of the plurality of ports connected to the network. Program instructions for re-assigning data transmitted over the failed one of the plurality of ports to an active port of the plurality of ports selected in a round robin technique. The computer readable media further includes program instructions for receiving data over one of the plurality of ports designated as a primary receiving port.
In another embodiment, an apparatus for driving a network interface card is disclosed. That apparatus includes means for monitoring a status of a plurality of ports connected between a computer and a network. Means for detecting a failure in one of the plurality of ports connected to the network. Means for re-assigning data transmitted over the failed one of the plurality of ports to an active port of the plurality of ports selected in a round robin technique. The apparatus further includes means for receiving data over one of the plurality of ports designated as a primary receiving port.
Advantageously, the embodiments of the present invention provide a multi-port network interface card having a plurality of ports connected to a single network, and a smart network interface driver for controlling the functions of the multi-port card and interactions with existing server operating systems. As such, it should be appreciated that no changes are required to be made to existing operating systems or other applications to take advantage of the increased throughput and redundancy provided in accordance with the present invention. Further, all internal load balancing and aggregation provides a resilient network interface card that automatically reconfigures itself in case of failure to one or more of the ports, and the re-configuration occurs in real-time without requiring operator intervention. These and other aspects and advantages of the invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the invention.
BRIEF DESCRIPTION OF THE DRAWINGS
The present invention will be readily understood by the following detailed description in conjunction with the accompanying drawings, wherein like reference numerals designate like structural elements, and in which:
<figref idref="DRAWINGS">FIG. 1</figref> is a graphical illustration of a server computer system including a network interface card (NIC) coupled to more than one local area network.
<figref idref="DRAWINGS">FIG. 2</figref> is a pictorial diagram of a server computer system having multi-port NICs connected to a plurality of networks in accordance with one embodiment of the present invention.
<figref idref="DRAWINGS">FIG. 3</figref> shows a more detailed diagram of a multi-port NIC connected to a network in accordance with one embodiment of the present invention.
<figref idref="DRAWINGS">FIG. 4</figref> shows a table containing status, function, ports, MAC addresses, and source addresses of the exemplary network connections illustrated in <figref idref="DRAWINGS">FIG. 3</figref> in accordance with one embodiment of the present invention.
<figref idref="DRAWINGS">FIG. 5A</figref> is an overview flowchart diagram illustrating the method operations associated with implementing a multi-port NIC in accordance with one embodiment of the present invention.
<figref idref="DRAWINGS">FIGS. 5B through 5D</figref> illustrate exemplary graphical user interfaces (GUI) for set-up and configuration of the multi-port NIC in accordance with one embodiment of the present invention.
<figref idref="DRAWINGS">FIG. 6</figref> is a more detailed flowchart diagram of the initialization performed by a smart NIC driver in accordance with one embodiment of the present invention.
<figref idref="DRAWINGS">FIG. 7A</figref> illustrates the method operations associated with performing transmits (Tx) through the various ports of the multi-port NIC in accordance with one embodiment of the present invention.
<figref idref="DRAWINGS">FIGS. 7B and 7C</figref> illustrate a port resolution table (PRT) timing routine performed in accordance with one embodiment of the present invention.
<figref idref="DRAWINGS">FIG. 8</figref> illustrates the method operations performed in selecting a new port when a current port is inactive or is not currently listed in the PRT table in accordance with one embodiment of the present invention.
<figref idref="DRAWINGS">FIG. 9</figref> is a flowchart diagram of the method operations performed in conducting the port resolution table timer routine in accordance with one embodiment of the present invention.
<figref idref="DRAWINGS">FIG. 10</figref> is a flowchart diagram illustrating the method operations associated with performing a link check timer (LCT) routine in accordance with one embodiment of the present invention.
<figref idref="DRAWINGS">FIG. 11</figref> is a block diagram of an exemplary computer system for carrying out the processing according to the invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
An invention is described for methods and apparatus for increasing throughput in a load balancing manner over a multi-port NIC. The present invention also provides a smart NIC driver for managing loads over the multi-port NIC and redistributing loads over active ports when failures occur with a given port of the multi-port NIC. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be obvious, however, to one skilled in the art, that the present invention may be practiced without some or all of these specific details. In other instances, well known process operations have not been described in detail in order not to unnecessarily obscure the present invention.
In one embodiment, the multi-port NIC is well suited for server computer systems that are required to transfer large amounts of data over a network in response to numerous short data requests. Preferably, the multi-port NIC of the present invention has each of its ports linked to the network, hub, or switch, and each port acts as an independent NIC having an associated bandwidth. Accordingly, the multi-port NIC of the present invention is capable of simultaneously transmitting data over each of the ports to a particular remote host located within a network with an aggregated bandwidth. By way of example, if each port of the multi-port NIC has a bandwidth of 100 Mbps, and the multi-port NIC is a 4-port card, the total aggregated bandwidth can be 400 Mbps, or higher for NICs with additional ports. Further, when ever a port of the multi-port NIC fails, the smart NIC driver that controls the operability of the multi-port NIC will automatically redistribute the load over the remaining ports that remain active.
<figref idref="DRAWINGS">FIG. 2</figref> is a pictorial diagram of a server computer system <b>120</b> being connected to a plurality of networks <b>110</b>a-<b>110</b>n in accordance with one embodiment of the present invention. Server computer system <b>120</b> has a random access memory (RAM) block <b>122</b> that has loaded therein an operating system <b>124</b> and a smart NIC driver <b>126</b>. Generally, operating system <b>124</b> can be any suitable operating system including, Unix, DOS, Microsoft Windows 95, Microsoft Windows NT, Netware, Macintosh OS, etc. RAM block <b>122</b> is shown in communication with a PCI bus <b>128</b>, which is also used to couple one or more multi-port NICs <b>130</b>. In one embodiment, one multi-port NIC <b>130</b> is used to connect to a distinct network (e.g., LAN).
Preferably, the present invention uses a “single” instance of the smart NIC driver <b>126</b> to control each port of the multi-port NIC <b>130</b>. In this manner, the operating system <b>124</b> is capable of interfacing with the smart NIC driver <b>126</b> to control the multiple ports of a multi-port NIC <b>130</b> in a manner that is transparent to the operating system <b>124</b>. That is, although smart NIC driver <b>126</b> is controlling multiple ports to a single network, its interface with existing operating systems will be essentially identical to software drivers used to conventionally control data flow through a single port. Therefore, the smart NIC driver <b>126</b> is seamlessly able to communicate with any of the well established network protocols controlled by operating system <b>124</b>, including TCP/IP, Netware and NetBEUI.
In the example of <figref idref="DRAWINGS">FIG. 2</figref>, one multi-port NIC <b>130</b> is shown having ports P<sub>1</sub>, P<sub>2</sub>, P<sub>3 </sub>and P<sub>4 </sub>connected to a Network-A <b>110</b>a, that may have a multitude of hosts <b>132</b> connected to hubs, switches and routers within Network-A <b>110</b>a. It should be understood that server <b>120</b> is capable of being networked to a number of separate networks by coupling associated multi-port NICs <b>130</b> to the PCI bus <b>128</b>. Further, although only four ports are shown associated with each multi-port NIC <b>130</b>, it should be understood that any number of ports may be added to suit the throughput needs of a particular system. As mentioned above, because each port of the multi-port NIC <b>130</b> has the data transfer capabilities of an independent NIC card, the transmit (Tx) throughput is aggregated by the number of ports in the multi-port NIC <b>130</b>. Further, in one embodiment, the multi-port NIC <b>130</b> may include 12 internal independent NIC cards that make up 12 associated ports. In still another embodiment, the multi-port NIC <b>130</b> may include “3” 4-port cards to produce a combined number of 12 associated ports. In any event, each of the described ports should be understood to have the capabilities of an independent NIC. As an example, in one embodiment, multi-port NIC <b>130</b> can function to transmit about 1,200 Mbps (about 1.2 gigabits) by including 12 ports, each having data transfer rates of 100 Mbps.
<figref idref="DRAWINGS">FIG. 3</figref> shows a more detailed diagram of a multi-port NIC <b>130</b> connected to a network <b>110</b> in accordance with one embodiment of the present invention. As shown, multi-port NIC <b>130</b> has ports P<sub>1</sub>, P<sub>2</sub>, P<sub>3</sub>, and P<sub>4 </sub>connected to node inputs on a switch <b>140</b> located within network <b>110</b>. Further, each port of Multi-port NIC <b>130</b> has an associated Ethernet media access control (MAC) address assigned to each port. As such, when port P<sub>1 </sub>communicates with switch <b>140</b>, switch <b>140</b> will know which port of multi-port NIC <b>130</b> transmitted the data to switch <b>140</b> by inspecting the source address “SA” of any received packet. In a similar manner, if ports P<sub>2</sub>, P<sub>3 </sub>or P<sub>4 </sub>transmit data through switch <b>140</b>, switch <b>140</b> will be able to ascertain the MAC address of each of those respective ports transferring data to network <b>110</b>.
To manage packet switching, switch <b>140</b> typically creates routing tables and stores packet data in queues before routing the data to its destination, which may be any one or more Hosts <b>132</b>a through <b>132</b>n, that may be connected to switch <b>140</b> of network <b>110</b>. Although only one switch has been illustrated for ease of description, it should be understood that typical networks include a multitude of network devices interconnecting a plurality of hosts through many hubs, switches, repeaters, bridges and routers.
<figref idref="DRAWINGS">FIG. 4</figref> shows a table containing status, function, ports, MAC addresses, and source addresses of the exemplary network connections illustrated in <figref idref="DRAWINGS">FIG. 3</figref> in accordance with one embodiment of the present invention. In this embodiment, port P<sub>1 </sub>is designated as a primary receiving (Rx) port as well as a transmitting (Tx<sub>1</sub>) port, while ports P<sub>2</sub>, P<sub>3 </sub>and P<sub>4 </sub>are designated as transmit ports Tx<sub>2</sub>, Tx<sub>3 </sub>and Tx<sub>4</sub>, respectively. As mentioned above, each port will also have an associated MAC address that is appended to outgoing packets as a source address “SA”. Accordingly, when packets are transmitted through port P<sub>1</sub>, each packet will necessarily have the SA-<b>1</b> associated with MAC-<b>1</b>.
It is important to note that each one of the ports P<sub>1</sub>, P<sub>2</sub>, P<sub>3</sub>, and P<sub>4 </sub>may transmit data through switch <b>140</b> at the same time in a load balanced manner which advantageously enables multi-port NIC <b>130</b> to increase its transmit throughput. For example, assume that ports P<sub>1</sub>, P<sub>2 </sub>and P<sub>4 </sub>are currently transmitting packets through switch <b>140</b> of network <b>110</b>, and when a new transmit request is received for a new host (currently not in the PRT table) by multi-port NIC <b>130</b>, the smart NIC driver <b>126</b> of <figref idref="DRAWINGS">FIG. 2</figref> will examine each of the transmit ports Tx<sub>1</sub>, Tx<sub>2</sub>, Tx<sub>3 </sub>and Tx<sub>4</sub>. Based on this examination, the smart NIC driver <b>126</b> will determine weather any port is currently free (i.e., not transmitting data), and then assign the new transmit request to the next free port, chosen in a round robin manner. In this example, port P<sub>3 </sub>is free, and therefore smart NIC driver <b>126</b> will assign the new data transmit request to port P<sub>3</sub>. On the other hand, if no free port remained, the smart NIC driver <b>126</b> would be configured to assign the new packet transmit request to the least busiest port (i.e., the port having the lightest load) currently transmitting to network <b>110</b>.
As mentioned above, port P<sub>1 </sub>is preferably designated as the primary receive (Rx) port assigned all receiving tasks of packet data coming from Hosts <b>132</b>a through <b>132</b>n shown connected to network <b>110</b>. Because NICs resident in server computers are typically requested to transmit out substantially more data than it is required to receive, only one port is actually necessary for efficiently performing the receive functions of multi-port NIC <b>130</b>. However, if port P<sub>1 </sub>fails for any reason, smart NIC driver <b>126</b> will automatically re-assign the receive (Rx) tasks to the next active port (in a round robin orientation) of multi-port NIC <b>130</b>. Once assigned to the next active port, the port that is now assigned with the receiving task will also be assigned the Ethernet MAC address of port P<sub>1</sub>. In this manner, all upper layer protocols will continue sending data to the same destination address “DA” associated with the Ethernet MAC address of P<sub>1</sub>.
It is important to point out that because the primary Rx's Ethernet MAC address is assigned the port taking over the receiving functions, all external hosts connected to the network <b>110</b> will be unable to tell a difference in the internal and transparent re-assignment of server <b>120</b> receiving functions. As such, the internal re-assignment of the primary Rx Ethernet MAC address is transparently controlled by the smart NIC driver <b>126</b>, and all that external hosts understand is that the multi-port NIC <b>130</b> is continuing to function smoothly and uninterrupted.
In one embodiment, if the primary Rx Ethernet MAC address is re-assigned to another port, the port newly assigned with the receiving Rx functions will also transmit (Tx) using the source address “SA” of the primary. By way of example, assuming that the primary Rx port P<sub>1 </sub>were to fail, and the next active port was port P<sub>2</sub>, then when port P<sub>3 </sub>transmits a packet over the network, that transmitted packet will have the source address SA-<b>1</b> associated with P<sub>1</sub>. As will be described in greater detail with reference to <figref idref="DRAWINGS">FIG. 10</figref>, the smart NIC driver <b>126</b> is set to perform a link check timer (LCT) routine for determining the status of all ports of the multi-port NIC <b>130</b>. During the LCT routine, if the primary Rx comes up (i.e., returns to an active state) from the its failed state, the smart NIC driver <b>126</b> will perform logical operations to ascertain whether it is proper to re-assign the receiving tasks to the primary Rx port P<sub>1</sub>. In any event, the smart NIC driver <b>126</b> is well suited to detect failures in any of the multiple ports (including the primary Rx port), and then perform load balancing to re-assign the load to the remaining active ports of the multi-port NIC <b>130</b>.
In a like manner, if any one of the transmit ports, e.g., port P<sub>3 </sub>fails at any point during a transmission, the load assigned to port P<sub>3 </sub>will be redistributed in a round robin manner to the next active and free port of multi-port NIC <b>130</b>. However, if all ports that remain active are currently busy (i.e., not free), then the load of port P<sub>3 </sub>will be assigned to the least busiest of the remaining ports, thereby enabling efficient load balancing over the remaining active ports.
<figref idref="DRAWINGS">FIG. 5A</figref> is an overview flowchart diagram illustrating the method operations associated with implementing a multi-port NIC <b>130</b> in accordance with one embodiment of the present invention. The method begins at an operation <b>502</b> where a multi-port NIC <b>130</b> is connected to a network as shown in FIG. <b>3</b>. By way of example, if multi-port NIC <b>130</b> is a 4-port card, then each port of multi-port NIC <b>130</b> will be connected to a node of the network <b>110</b>, which may include a switch <b>140</b>, a hub, a bridge, a router, a repeater or other network interconnection device. In a preferred embodiment, multi-port NIC <b>130</b> is preferably installed to a PCI bus <b>128</b> of a server computer system <b>120</b> or other network device requiring the ability to transmit at increased load balancing throughput while ensuring superior fail-over resiliency in case one or more ports fail to operate properly.
Once multi-port NIC <b>130</b> has been installed into a computer and connected to a network, the method proceeds to an operation <b>504</b> where a multi-port NIC <b>130</b> configuration panel is called up in the form of a graphical user interface (GUI). The method then proceeds to an operation <b>506</b> where port configuration is performed for each port in the multi-port NIC <b>130</b> to ascertain the bandwidth (e.g., 10 Mbps, 100 Mbps, etc.) of each port. By way of example, as shown in <figref idref="DRAWINGS">FIG. 5B</figref> below, a port configuration GUI <b>520</b> is shown having an available network port window <b>524</b> listing all ports that may be connected to the multi-port NIC <b>130</b>. In one embodiment, the ports displayed in available network ports window <b>524</b> are provided from an operating system registry which holds all available network ports that have been previously installed.
By default, the connection type for the available network ports is set to “auto detect default connection” which is configured to ascertain the type of connection associated with each of the particular ports. However, if the user desires to manually change the connection types, the connection types window <b>522</b> may be modified through the drop-down list box to make an appropriate change. Once the changes are made, the apply icon button is preferably selected to implement the newly entered connection types.
Referring back to <figref idref="DRAWINGS">FIG. 5A</figref>, once each port has been configured for bandwidth in <figref idref="DRAWINGS">FIG. 5B</figref>, the method proceeds to an operation <b>508</b> where grouping of selected configured ports is performed to attain desired load balancing for the groups. By way of example, once port configuration has been performed in FIG. <b>5</b>B and the OK icon has been selected from ports configuration GUI <b>520</b>, a port aggregation GUI <b>530</b> is presented to the user as shown in FIG. <b>5</b>C. In port aggregation GUI <b>530</b>, the user is able to create groups by selecting ports from the available network ports list <b>534</b> and then adding and removing ports through buttons <b>542</b>.
By way of example, when a port is selected from the available port list <b>534</b> and the ADD button <b>542</b>a is clicked, the selected port will be shifted to a group ports list <b>536</b>. Of course, ports may also be removed from the group ports list <b>536</b> by clicking remove button <b>542</b>b after a port is selected from the group ports list <b>536</b>. Once the selected ports are moved in or out of the group ports list <b>536</b>, the user may name that group in a groups window <b>540</b>, where a link check time-out is provided. As will be described in greater detail below, the link check time-out (LCT) is used to check the activity of the selected ports during operation. In the example shown, the LCT time is set to 10 seconds, however, the LCT timer may be set to any number of seconds between about 1 second and about 20 seconds.
If the user selects the status icon <b>548</b> within the port aggregation GUI <b>530</b>, a port aggregation window <b>540</b> of <figref idref="DRAWINGS">FIG. 5D</figref> is provided to graphically display whether any of the ports within a particular group is active or inactive. For example, window <b>544</b> illustrates that Group <b>1</b> (having ports P<sub>1</sub>, P<sub>2 </sub>and P<sub>3</sub>) has port P<sub>2 </sub><b>544</b>a in an inactive state, which is indicated by a cross-out icon, or may be indicated by any other graphical indicator signaling inactivity. However, port P<sub>1 </sub>and port P<sub>3 </sub>are shown currently active. To ascertain the load activity of a particular port, the user may simply select the icon of a particular port, e.g., port P<sub>3</sub>, and data for the number of bytes received per second and the number of bytes transmitted per second, are displayed to the user in status window <b>546</b>.
Returning to <figref idref="DRAWINGS">FIG. 5A</figref>, once the ports are configured and the groups are configured for load balancing in operations <b>506</b> and <b>508</b>, the method proceeds to an operation <b>510</b> where the smart NIC driver <b>126</b> initializes the ports and groups selected by the user through the graphical user interfaces. Once smart NIC driver <b>126</b> has completed its initialization task, the method may proceed to an operation <b>512</b> where the multi-port NIC <b>130</b> transmits and receives over the network through the control of the smart NIC driver <b>126</b>. In a preferred embodiment, the install and set-up of load balancing for the multi-port NIC <b>130</b> may be conducted through the above described GUIs loaded in a Windows based machine.
<figref idref="DRAWINGS">FIG. 6</figref> is a more detailed flowchart diagram of operation <b>510</b> of <figref idref="DRAWINGS">FIG. 5</figref> in accordance with one embodiment of the present invention. The initialization of the ports and the groups begins at an operation <b>602</b> where the smart NIC driver <b>126</b> reads the operating system registry entries for each port and initializes the ports. For example, the ports configured for each group in <figref idref="DRAWINGS">FIG. 5C</figref> are initialized to ascertain the connection types for the associated ports. In one embodiment, the initialization also includes the transmission of a packet by the transmitting ports other than the primary Rx (i.e., Tx<sub>2</sub>, Tx<sub>3 </sub>and Tx<sub>4</sub>) to the primary Rx port in order to verify that the transmitting ports are indeed connected to one network. By way of example, assume that the network administrator erroneously connected one port, e.g., port P<sub>2 </sub>that is shown connected to network A <b>110</b>a of <figref idref="DRAWINGS">FIG. 2</figref>, to network N <b>110</b>n, then the Rx port would not receive the transmitted packet sent by Tx<sub>2</sub>. As such, if the packet is not received from Tx<sub>2</sub>, then the smart NIC driver <b>126</b> would known during the initialization process that port P<sub>2 </sub>is not connected to network A <b>110</b>a, and therefore the smart NIC driver <b>126</b> will not send packets through port P<sub>2</sub>.
Next, the method proceeds to an operation <b>604</b> where the smart NIC driver <b>126</b> creates registry groups from the registry values associated with the groups configured through the graphical user interfaces. Once the smart NIC driver <b>126</b> creates the groups as shown in <figref idref="DRAWINGS">FIG. 5D</figref>, the method proceeds to an operation <b>606</b> where the smart NIC driver <b>126</b> initializes the configuration values and start time-out values associated with the multi-port NIC <b>130</b>. By way of example, the start time-out values include a port resolution table (PRT) timer and a link check timer (LCT). As will be described in greater detail below, the PRT time is used to march through a PRT table and remove entries for those hosts that are not in use for a predetermined period of time, and the LCT timer is used to monitor the activity of all ports for another predetermined period of time. As such, these timers assist in efficient load balancing among active ports of the multi-port NIC <b>130</b>. Once the configuration values and start time-out values are initialized in operation <b>606</b>, the method proceeds to operation <b>512</b> of FIG. <b>5</b>.
<figref idref="DRAWINGS">FIG. 7A</figref> illustrates the method operations associated with performing transmits (Tx) through the various ports of the multi-port NIC <b>130</b> in accordance with one embodiment of the present invention. The method begins at an operation <b>702</b> where the multi-port NIC <b>130</b> receives a request from an upper layer to transmit packets to a networked remote host. By way of example, the application layer which may be a Ping application, an FTP application, or a Telnet application that communicates with the smart NIC driver <b>126</b> that enables communication with the multi-port NIC <b>130</b> (to perform the desired transmit requests).
The method then proceeds to an operation <b>704</b> where the destination Ethernet media access control (MAC) address is retrieved from packets that are to be transmitted over the network. By way of example, if a data packet is requested to be transmitted to a remote Host A, then the destination address “DA” of the Host A will be retrieved from the packet that is to be transmitted. The method then proceeds to a decision operation <b>706</b> where it is determined whether the retrieved Ethernet MAC address is currently listed in a PRT table.
By way of example, <figref idref="DRAWINGS">FIG. 7B</figref> illustrates a port resolution table (PRT) in accordance with one embodiment of the present invention. PRT table is essentially a lookup table that lists hosts that may be connected to a port of the multi-port NIC <b>130</b> at a certain point in time. By way of example, when a packet is initially assigned to be transmitted to a particular Host <b>132</b> having an associated destination address “DA,” the smart NIC driver <b>126</b> will examine the 6th byte of the destination address “DA” to obtain an index number to an array of 255 entries in the port resolution table (PRT).
If the examination of the 6th byte of the destination address “DA” of the packet to be transmitted returns, for example, a number 3, then the entire destination address “DA” of the packet being transmitted is compared to the MAC destination address of any hosts linked to the entry indexed in the PRT table. In this example, only one Host C <b>132</b>c is linked to index number 3, and therefore the comparison of the DA of the packet being sent and the MAC destination address of Host C <b>132</b>c should match up. However, as will be described below, if there are multiple hosts doubly linked to the same index number (i.e., both hosts have the same 6th byte) then the comparison is done for those multiple hosts that are doubly linked. Although destination and source addresses are typically 6 bytes long, comparison are preferably performed based on 8 bytes (two words) at one time to improve processing efficiency. That is, additional zero values are added to the 6 bytes of the SA and DA to ensure that comparisons are based on two words. It should be appreciated that this lookup table comparison technique is well suited to reduce the amount of processing required by the CPU of the computer containing the multi-port NIC <b>130</b>, thereby reducing the number of possible CPU interrupts.
Assuming that the first packet to be transferred through multi-port NIC <b>130</b> (i.e., when none of the ports are currently in use) is a packet destined for Host C <b>132</b>c as described above, then the first available port P<sub>1 </sub>will be selected in a round robin selection technique. As pictorially shown in <figref idref="DRAWINGS">FIG. 7B</figref>, Host C <b>132</b>c (being linked to index number 3 of the PRT table) has an associated parameters box <b>750</b> identifying port P<sub>1 </sub>as the selected port, a timer set to “1,” indicating current transmission, a HOST C MAC destination address of Host C <b>132</b>c, a pointer to index number <b>3</b> at one end, and a pointer to null at the other end. The pointer to null indicates that there are no other hosts sharing the same 6th byte of their MAC destination addresses. However, Host A <b>132</b>a and Host B <b>132</b>b do share the same 6th byte of their MAC destination address and therefore, are coupled in a doubly-linked manner, where a parameters box <b>752</b> of Host A <b>132</b>a points to index <b>70</b> at one end, and points to Host B at the other end. In a like manner, a parameters box <b>754</b> of Host B <b>132</b>b points to Host A <b>132</b>a at one end and points to null at the other end.
Continuing with the example of <figref idref="DRAWINGS">FIG. 7B</figref>, the second packet transfer performed by Multi-port NIC <b>130</b> was designated for Host A <b>132</b>a, and the next succeeding data transfer was directed for Host B <b>132</b>b. Because port assignment is carried out in a round robin technique, ports P<sub>2 </sub>and P<sub>3 </sub>will be assigned to Host A <b>132</b>a and Host B <b>132</b>b, respectively. In a like manner, when Host N <b>132</b>n is requested as the designated remote host, the smart NIC <b>126</b> will select the next available port, which is port P<sub>4 </sub>as shown in a parameters box <b>756</b> associated with Host N <b>132</b>n. It should be understood that data transfers may be performed simultaneously through each of the transmit ports and therefore, are shown having a number “1” in the timers of parameter boxes <b>750</b>, <b>752</b>, <b>754</b>, and <b>756</b>.
As will be described with reference to <figref idref="DRAWINGS">FIG. 9</figref> below, a PRT timer routine is controlled by the smart NIC driver <b>126</b> which scans all indices of the PRT table and associated connected hosts to determine whether they are or are not actively transmitting data. In an initial scan, if the timer is currently set to “1,” then the timer will be changed to “0” if the port is inactive. After a predetermined time-out, the smart NIC <b>126</b> will again scan all indices (i.e., from 0 to 255) of the PRT table and associated hosts, and again change all timers that are currently set to “1,” to “0.” However, if the timer is already set to “0,” the smart NIC driver <b>126</b> will remove the connection to that remote host, and thereby make the port assigned to that host free.
Returning to <figref idref="DRAWINGS">FIG. 7A</figref>, if it is determined in operation <b>706</b> that the MAC destination address of a desired remote host is currently listed in the PRT table of <figref idref="DRAWINGS">FIG. 7B</figref>, then the method will proceed to a decision operation <b>708</b> where it is determined if the port for the Ethernet MAC address is active. For example, if the host is inactive due to a link failure or port failure, it will be considered in-active in operation <b>708</b>. As shown in <figref idref="DRAWINGS">FIG. 7B</figref>, if data is to be transmitted to a particular remote host having an associated Ethernet MAC address, and a port was previously assigned to transmit to that remote host, all future packets destined for that remote host will preferably be queued up for transmission through that port. By way of example, because a connection exists in Host C <b>132</b>c through port P<sub>1</sub>, all data targeted for Host C will continue to pass through port P<sub>1 </sub>until a failure occurs.
Alternatively, if it is determined in operation <b>708</b> that the port for the Ethernet MAC address is not active, the method will proceed to an operation <b>710</b> where a new port is selected for transmission to a desired remote host. Once a new port is selected from the remaining active ports (or least busiest) in a round robin technique, the method will proceed to an operation <b>712</b> where the packet(s) are transmitted over the port to the desired Ethernet MAC address of the recipient host.
<figref idref="DRAWINGS">FIG. 8</figref> illustrates the method operations performed in selecting a new port as described in operation <b>710</b> of <figref idref="DRAWINGS">FIG. 7A</figref> in accordance with one embodiment of the present invention. This method begins at an operation <b>802</b> where selection of a new port is performed by proceeding to the next active port in a round robin orientation. For example, if multi-port NIC <b>130</b> is a 4-port card and selection begins at port P<sub>1</sub>, then the next port in a round robin orientation will be port P<sub>2</sub>, then port P<sub>3</sub>, then port P<sub>4 </sub>and then back to port P<sub>1</sub>. Once the next active port is selected in operation <b>802</b>, the method proceeds to a decision operation <b>804</b> where it is determined if the next port in the round robin orientation is free. That is, the port is free if it is currently not transmitting packet data to a remote host, and if it is not indexed to the PRT table of FIG. <b>7</b>A. If the next port is free, then the method will proceed to an operation <b>808</b> where the free active port is selected for transmission.
On the other hand, if it is determined that the next port is not free in operation <b>804</b>, the method will proceed to an operation <b>806</b> where it is determined whether all ports have been checked and been determined to have been busy. If all ports have been not been checked and been determined to have been busy, the method will proceed back to operation <b>802</b> where the method proceeds to select the next active port in a round robin orientation. Once the next active port is selected, the method will again proceed to decision operation <b>804</b> where it is determined if this next port is free. If it is again determined to not have been free, then the method will again proceed to decision operation <b>806</b> where it is again determined whether all ports have been checked and are busy.
Once all ports have been checked and have been ascertained to be busy, the method will proceed from operation <b>806</b> to operation <b>810</b>, where the least busiest port is selected for transmission. Once the least busiest port has been selected for transmission, the method returns to operation <b>712</b> of FIG. <b>7</b>A. In a like manner, if one port was ascertained to be free in operation <b>808</b>, that port would be selected and the method would proceed to operation <b>712</b> of FIG. <b>7</b>A. In sum, the selection of a new port is performed in a load balancing manner that first determines whether there are any free ports that may be used for transmission, and if there are no free ports, the method will proceed to select the least busiest port for transmission.
<figref idref="DRAWINGS">FIG. 9</figref> is a flowchart diagram of the method operations performed in conducting a port resolution table (PRT) timer routine in accordance with one embodiment of the present invention. Once the multi-port NIC <b>130</b> has been initialized by the smart NIC driver <b>126</b> of <figref idref="DRAWINGS">FIG. 2</figref>, the method proceeds to an operation <b>902</b> where the routine marches through the PRT table entries beginning with entry 0 as shown in FIG. <b>7</b>B. Once the method is at entry 0 in the table, the method will proceed to a decision operation <b>904</b> where it is determined whether there is a host associated with the current PRT table. As shown, entry 0 in PRT table does not currently have a linked host, and therefore the method will proceed back to operation <b>902</b>.
In operation <b>902</b>, the method will then proceed to the next entry in the PRT table. The next entry is entry number 2, which again does not satisfy the condition of operation <b>904</b>, therefore, the method will again proceed to operation <b>902</b> where the method moves to entry number 3. Once at entry number 3, it is determined in decision operation <b>904</b> that there is a host associated with the current PRT table entry. At this point, the method proceeds to a decision operation <b>906</b> where it is determined whether the time of the parameters box <b>750</b> is currently set to “0.” In this example, the timer is currently shown set to “1,” and therefore the method proceeds to operation <b>910</b> where the PRT timer is re-set to “0” as shown in parameters box <b>750</b>a.
On the other hand, if it is determined in operation <b>906</b> that the timer was already set to “0,” then the method would proceed to operation <b>908</b>, where the host entry would be deleted from the PRT table as shown in FIG. <b>7</b>C. The removal of Host C <b>132</b>c from the PRT table essentially means that port P<sub>1 </sub>was idle for at least a predetermined PRT timer delay time-out. In one embodiment, the PRT timer delay (TIMEOUT) is calibrated for performance optimization to a time of between about ¼ second and 1 second, and most preferably about ½ second. Of source, these preferred optimizations may vary depending on network loading as well as other factors.
Once Host C <b>132</b>c entry is removed from the PRT table as shown in <figref idref="DRAWINGS">FIG. 7C</figref>, the method will proceed to an operation <b>912</b> where it is determined whether there is a pointer to a next doubly linked host. As shown in <figref idref="DRAWINGS">FIG. 7B</figref>, the parameters box <b>752</b> of host C <b>132</b>c do not point to a next doubly linked host, and therefore the method will proceed to an operation <b>914</b>. In operation <b>914</b> it is determined whether all of the 255 entries have already been analyzed. Because in this example all 255 entries have not yet been analyzed, the method will proceed back up to operation <b>902</b> where the method proceeds to the next PRT table entry. Once at the next PRT table entry, the method will continue looping through operations <b>904</b> and <b>902</b> until a host is associated with a current PRT table entry. As mentioned above, the next PRT table entry having a host associated with it is entry number <b>70</b> as shown in FIG. <b>7</b>B. At this point, it is determined if the timer of the parameters box <b>752</b> of Host A <b>132</b>a is set to 0. Because the timer is set to “1,” the method will proceed to operation <b>910</b> where the PRT timer is set to “0.”
Once the timer is set to “0,” the method will proceed to decision operation <b>912</b> where it is determined if there is a pointer to a next doubly linked host. Because Host B <b>132</b>b is doubly linked to Host A <b>132</b>a, the method will proceed back to operation <b>906</b> where it is determined if the timer of the parameters box <b>754</b> is set to “0.” Because the timer is currently set to “1,” the method will proceed to operation <b>910</b> where the timer is set to “0.” On the other hand, if the timer had been set to 0 as shown in parameters box <b>754</b>a, the method would have proceeded to operation <b>908</b> where the host would be deleted from the PRT table as shown in FIG. <b>7</b>C. This process will therefore continue until all 255 entries have been analyzed and the method will proceed to operation <b>916</b>, where the routine will wait for a PRT timer delay (TIMEOUT). Once the TIMEOUT is complete, the method will again proceed through the PRT timer routine and commence marching through each of the entries beginning with 0, and checking the timers of each host that is currently linked (or doubly-linked) with an entry of the PRT table.
As mentioned above, if the timer of any host was previously set to 0 in a prior PRT timer routine, that host will be deleted as an entry to the PRT table as described with reference to operation <b>908</b> of FIG. <b>9</b>. As such, when a host is deleted from the PRT table, the port number allocated to that host will likewise become available when a new packet transfer operation is requested to a new remote host not currently linked to the table, or the same port link may again be re-established.
<figref idref="DRAWINGS">FIG. 10</figref> is a flowchart diagram illustrating the method operations associated with performing a link check timer (LCT) routine in accordance with one embodiment of the present invention. The link check timer routine begins at an operation <b>1002</b> where all ports of the multi-port NIC <b>130</b> are scanned to determine their status. By way of example, each port is examined to ascertain whether it is active or inactive. Once all ports have been scanned to determine their status in operation <b>1002</b>, the method proceeds to a decision operation <b>1004</b> where it is determined whether the primary Rx port is active. As mentioned in <figref idref="DRAWINGS">FIG. 4</figref> above, port P<sub>1 </sub>is preferably designated as the primary Rx port, which is assigned with all receiving tasks while it is active.
Therefore, if it is determined that the primary Rx port is active, the method will proceed to a decision operation <b>1010</b> where it is determined if the primary Rx port is functioning as the receiver. By way of example, if the primary Rx port had previously failed for any reason, the receiving tasks would have been assigned to the next available active port. However, assuming that the primary Rx port is the receiver, the method will proceed to an operation <b>1016</b> where the LCT timer is re-started.
On the other hand, if it is determined that the primary Rx port is not currently the receiver in operation <b>1010</b>, the method would proceed to an operation <b>1014</b> where the primary Rx port would be designated as the receiver, because it was determined in operation <b>1004</b> that the primary Rx port was active. Then, once the primary port Rx is designated as the receiver, the method would proceed to operation <b>1016</b> where the LCT timer would be restarted. Referring back to operation <b>1004</b>, if it is determined that the primary Rx port is not active, then the method would proceed to a decision operation <b>1006</b>.
In decision operation <b>1006</b>, it is determined whether the primary Rx port is currently designated as the receiver. By way of example, assuming that the primary Rx port was functioning as the receiver, and then failed to operate properly (i.e., not active) in operation <b>1004</b>, then the receiving tasks would have to be assigned to another port. Therefore, if the primary Rx port is currently the receiver and it is no longer active, the method would proceed to an operation <b>1012</b>, where the next active port (selected in a round robin orientation) would be designated as the receiver. On the other hand, if it was determined in operation <b>1006</b> that the primary Rx port was not the receiver, the method would proceed to a decision operation <b>1008</b> where it is determined if the port designated as the receiver is active.
By way of example, assuming that the primary Rx port was not functioning as the receiver, and the port assigned all receiving functions also became inactive as determined in operation <b>1008</b>, the method would proceed to operation <b>1012</b> where the next active port would be designated as a receiver. On the other hand, if the port designated as the receiver in operation <b>1008</b> was active, the method would proceed to operation <b>1016</b> where the LCT timer would be re-started as described above. Once the LCT timer is restarted in operation <b>1016</b>, the method will proceed to an operation <b>1018</b> where a link check timer delay (TIMEOUT) would commence. In one embodiment, the LCT time-out is preferably set by the user to be between about 1 second and about 10 seconds, and most preferably, about 2 seconds. Of course, these LCT time-out parameters may be modified outside the preferred times to fit well within networks of varying load requirements. After the TIMEOUT of operation <b>1018</b>, the method will proceed through the link check timer routine again as described above.
Tables A through D below illustrate the re-assignment of receiver functions to the active ports when the port assigned the receiving functions fails to operate due to software errors, hardware errors, network over-loading errors, or a combination thereof. In table A below, all ports are currently active, and the primary Rx port P<sub>1 </sub>is assigned all receiving functions. As such, each port will have its own MAC address, and transmits will be performed with their own source addresses attached to the packet.
<tables id="TABLE-US-00001" num="00001"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="4"><colspec colname="offset" colwidth="21pt" align="left" /><colspec colname="1" colwidth="112pt" align="left" /><colspec colname="2" colwidth="28pt" align="center" /><colspec colname="3" colwidth="56pt" align="center" /><thead><row><entry /><entry namest="offset" nameend="3" rowsep="1">TABLE A</entry></row><row><entry /><entry namest="offset" nameend="3" align="center" rowsep="1" /></row><row><entry /><entry /><entry>MAC</entry><entry>Source</entry></row><row><entry /><entry>Ports</entry><entry>Address</entry><entry>Address</entry></row><row><entry /><entry namest="offset" nameend="3" align="center" rowsep="1" /></row></thead><tbody valign="top"><row><entry /><entry>Port P<sub>1 </sub>(Active) (Primary Rx)</entry><entry>MAC-1</entry><entry>SA-1</entry></row><row><entry /><entry>* Receiver</entry></row><row><entry /><entry>Port P<sub>2 </sub>(Active)</entry><entry>MAC-2</entry><entry>SA-2</entry></row><row><entry /><entry>Port P<sub>3 </sub>(Active)</entry><entry>MAC-3</entry><entry>SA-3</entry></row><row><entry /><entry>Port P<sub>4 </sub>(Active)</entry><entry>MAC-4</entry><entry>SA-4</entry></row><row><entry /><entry namest="offset" nameend="3" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>
When port P<sub>1 </sub>fails for any reason, the smart NIC driver <b>126</b> will look to the next active port, and assign the receiving functions to that port. As illustrated in Table B below, port P<sub>2 </sub>is now the receiver. As mentioned above, because port P<sub>2 </sub>is now the receiver, it will be assigned the MAC address of port P<sub>1</sub>, and when port P<sub>2 </sub>transmits out, it will transmit packets with the source address SA-<b>1</b> of port P<sub>1</sub>.
<tables id="TABLE-US-00002" num="00002"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="4"><colspec colname="offset" colwidth="21pt" align="left" /><colspec colname="1" colwidth="112pt" align="left" /><colspec colname="2" colwidth="28pt" align="center" /><colspec colname="3" colwidth="56pt" align="center" /><thead><row><entry /><entry namest="offset" nameend="3" rowsep="1">TABLE B</entry></row><row><entry /><entry namest="offset" nameend="3" align="center" rowsep="1" /></row><row><entry /><entry /><entry>MAC</entry><entry>Source</entry></row><row><entry /><entry>Ports</entry><entry>Address</entry><entry>Address</entry></row><row><entry /><entry namest="offset" nameend="3" align="center" rowsep="1" /></row></thead><tbody valign="top"><row><entry /><entry>Port P<sub>1 </sub>(Down) (Primary Rx)</entry><entry>MAC-1</entry><entry>SA-1</entry></row><row><entry /><entry>Port P<sub>2 </sub>(Down)</entry><entry>MAC-1</entry><entry>SA-1</entry></row><row><entry /><entry>* Receiver</entry></row><row><entry /><entry>Port P<sub>3 </sub>(Active)</entry><entry>MAC-3</entry><entry>SA-3</entry></row><row><entry /><entry>Port P<sub>4 </sub>(Active)</entry><entry>MAC-4</entry><entry>SA-4</entry></row><row><entry /><entry namest="offset" nameend="3" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>
If after an LCT timer routine it is determined that port P<sub>2 </sub>is no longer active (operation <b>1008</b>), then the smart NIC driver <b>126</b> will designate the next active port as the receiver. As shown in Table C below, the next active port is port P<sub>3</sub>. Because port P<sub>3 </sub>is now the receiver, it will be assigned the MAC address of the primary Rx port P<sub>1</sub>, and any transmits performed from port P<sub>3 </sub>will have the source address SA-<b>1</b>.
<tables id="TABLE-US-00003" num="00003"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="4"><colspec colname="offset" colwidth="21pt" align="left" /><colspec colname="1" colwidth="112pt" align="left" /><colspec colname="2" colwidth="28pt" align="center" /><colspec colname="3" colwidth="56pt" align="center" /><thead><row><entry /><entry namest="offset" nameend="3" rowsep="1">TABLE C</entry></row><row><entry /><entry namest="offset" nameend="3" align="center" rowsep="1" /></row><row><entry /><entry /><entry>MAC</entry><entry>Source</entry></row><row><entry /><entry>Ports</entry><entry>Address</entry><entry>Address</entry></row><row><entry /><entry namest="offset" nameend="3" align="center" rowsep="1" /></row></thead><tbody valign="top"><row><entry /><entry>Port P<sub>1 </sub>(Down) (Primary Rx)</entry><entry>MAC-1</entry><entry>SA-1</entry></row><row><entry /><entry>Port P<sub>2 </sub>(Down)</entry><entry>MAC-2</entry><entry>SA-2</entry></row><row><entry /><entry>Port P<sub>3 </sub>(Active)</entry><entry>MAC-1</entry><entry>SA-1</entry></row><row><entry /><entry>* Receiver</entry></row><row><entry /><entry>Port P<sub>4 </sub>(Active)</entry><entry>MAC-4</entry><entry>SA-4</entry></row><row><entry /><entry namest="offset" nameend="3" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>
When the link check timer routine is performed again after the predetermined timeout, the smart NIC driver <b>126</b> will proceed to decision operation <b>1004</b> where it will be determined that the primary Rx port P<b>1</b> is again active as shown in Table D below. As such, the primary Rx port will be once again designated as the receiver as described in operation <b>1014</b>, after it is determined that the primary Rx port was not the receiver in operation <b>1010</b>, and shown in Table C above.
<tables id="TABLE-US-00004" num="00004"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="4"><colspec colname="offset" colwidth="21pt" align="left" /><colspec colname="1" colwidth="112pt" align="left" /><colspec colname="2" colwidth="28pt" align="center" /><colspec colname="3" colwidth="56pt" align="center" /><thead><row><entry /><entry namest="offset" nameend="3" rowsep="1">TABLE D</entry></row><row><entry /><entry namest="offset" nameend="3" align="center" rowsep="1" /></row><row><entry /><entry /><entry>MAC</entry><entry>Source</entry></row><row><entry /><entry>Ports</entry><entry>Address</entry><entry>Address</entry></row><row><entry /><entry namest="offset" nameend="3" align="center" rowsep="1" /></row></thead><tbody valign="top"><row><entry /><entry>Port P<sub>1 </sub>(Active) (Primary Rx)</entry><entry>MAC-1</entry><entry>SA-1</entry></row><row><entry /><entry>* Receiver</entry></row><row><entry /><entry>Port P<sub>2 </sub>(Down)</entry><entry>MAC-2</entry><entry>SA-2</entry></row><row><entry /><entry>Port P<sub>3 </sub>(Active)</entry><entry>MAC-3</entry><entry>SA-3</entry></row><row><entry /><entry>Port P<sub>4 </sub>(Active)</entry><entry>MAC-4</entry><entry>SA-4</entry></row><row><entry /><entry namest="offset" nameend="3" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>
The present invention may be implemented using any type of integrated circuit logic or software driven computer-implemented operations. By way of example, a hardware description language (HDL) based design and synthesis program may be used to design the silicon-level circuitry necessary to appropriately perform the data and control operations in accordance with one embodiment of the present invention. By way of example, a VHDL® hardware description language available from IEEE of New York, N.Y. may be used to design an appropriate silicon-level layout. Although any suitable design tool may be used, another layout tool may include a hardware description language “Verilog®” tool available from Cadence Design Systems, Inc. of Santa Clara, Calif.
The invention may also employ various computer-implemented operations involving data stored in computer systems. These operations are those requiring physical manipulation of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. Further, the manipulations performed are often referred to in terms, such as producing, identifying, determining, or comparing.
Any of the operations described herein that form part of the invention are useful machine operations. The invention also relates to a device or an apparatus for performing these operations. The apparatus may be specially constructed for the required purposes, or it may be a general purpose computer selectively activated or configured by a computer program stored in the computer. In particular, various general purpose machines may be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct a more specialized apparatus to perform the required operations. An exemplary structure for the invention is described below.
<figref idref="DRAWINGS">FIG. 11</figref> is a block diagram of an exemplary computer system <b>1100</b> for carrying out the processing according to the invention. The computer system <b>1100</b> includes a digital computer <b>1102</b>, a display screen (or monitor) <b>1104</b>, a printer <b>1106</b>, a floppy disk drive <b>1108</b>, a hard disk drive <b>1110</b>, a network interface <b>1112</b>, and a keyboard <b>1114</b>. The digital computer <b>1102</b> includes a microprocessor <b>1116</b>, a memory bus <b>1118</b>, random access memory (RAM) <b>1120</b>, read only memory (ROM) <b>1122</b>, a peripheral bus <b>1124</b>, and a keyboard controller <b>1126</b>. The digital computer <b>1100</b> can be a personal computer (such as an IBM compatible personal computer, a Macintosh computer or Macintosh compatible computer), a workstation computer (such as a Sun Microsystems or Hewlett-Packard workstation), or some other type of computer.
The microprocessor <b>1116</b> is a general purpose digital processor which controls the operation of the computer system <b>1100</b>. The microprocessor <b>1116</b> can be a single-chip processor or can be implemented with multiple components. Using instructions retrieved from memory, the microprocessor <b>1116</b> controls the reception and manipulation of input data and the output and display of data on output devices. According to the invention, a particular function of microprocessor <b>1116</b> is to assist in the port aggregation and load balancing tasks controlled by smart NIC driver <b>126</b> of multi-port NIC <b>130</b>.
The memory bus <b>1118</b> is used by the microprocessor <b>1116</b> to access the RAM <b>1120</b> and the ROM <b>1122</b>. The RAM <b>1120</b> is used by the microprocessor <b>1116</b> as a general storage area and as scratch-pad memory, and can also be used to store input data and processed data. The ROM <b>1122</b> can be used to store instructions or program code followed by the microprocessor <b>1116</b> as well as other data.
The peripheral bus <b>1124</b> is used to access the input, output, and storage devices used by the digital computer <b>1102</b>. In the described embodiment, these devices include the display screen <b>1104</b>, the printer device <b>1106</b>, the floppy disk drive <b>1108</b>, the hard disk drive <b>1110</b>, and the network interface <b>1112</b>. The keyboard controller <b>1126</b> is used to receive input from keyboard <b>1114</b> and send decoded symbols for each pressed key to microprocessor <b>116</b> over bus <b>1128</b>.
The display screen <b>1104</b> is an output device that displays images of data provided by the microprocessor <b>1116</b> via the peripheral bus <b>1124</b> or provided by other components in the computer system <b>1100</b>. The printer device <b>1106</b> when operating as a printer provides an image on a sheet of paper or a similar surface. Other output devices such as a plotter, typesetter, etc. can be used in place of, or in addition to, the printer device <b>1106</b>.
The floppy disk driver <b>1108</b> and the hard disk drive <b>1110</b> can be used to store various types of data. The floppy disk drive <b>1108</b> facilitates transporting such data to other computer systems, and hard disk drive <b>1110</b> permits fast access to large amounts of stored data.
The microprocessor <b>1116</b> together with an operating system operate to execute computer code and produce and use data. The computer code and data may reside on the RAM <b>1120</b>, the ROM <b>1122</b>, or the hard disk drive <b>1110</b>. The computer code and data could also reside on a removable program medium and loaded or installed onto the computer system <b>1100</b> when needed. Removable program mediums include, for example, CD-ROM, PC-CARD, floppy disk and magnetic tape.
The network interface <b>1112</b> is used to send and receive data over a network connected to other computer systems. An interface card or similar device and appropriate software implemented by the microprocessor <b>1116</b> can be used to connect the computer system <b>1100</b> to an existing network and transfer data according to standard protocols.
The keyboard <b>1114</b> is used by a user to input commands and other instructions to the computer system <b>1100</b>. Other types of user input devices can also be used in conjunction with the present invention. For example, pointing devices such as a computer mouse, a track ball, a stylus, or a tablet can be used to manipulate a pointer on a screen of a general-purpose computer.
The invention can also be embodied as computer readable code on a computer readable medium. The computer readable medium is any data storage device that can store data which can be thereafter be read by a computer system. Examples of the computer readable medium include read-only memory, random-access memory, CD-ROMs, magnetic tape, optical data storage devices. The examples listed in the prior sentence are examples of computer readable storage media. The computer readable medium can also be distributed over a network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.
Although the foregoing invention has been described in some detail for purposes of clarity of understanding, it will be apparent that certain changes and modifications may be practiced within the scope of the appended claims. In addition, it should be understood that the various processing functions described above may be implemented both in silicon as hardware integrated circuits, or as software code that may be stored and retrieved from any suitable storage medium. Accordingly, the present embodiments are to be considered as illustrative and not restrictive, and the invention is not to be limited to the details given herein, but may be modified within the scope and equivalents of the appended claims.
Contents4
15 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15
Every citation, both waysCites: the store holds 54 of 55
| Document | Relation | Office | Cited during |
|---|---|---|---|
| US2012117366A1 | Cited by | United States of America | Pre-grant |
| US2005281191A1 | Cited by | United States of America | Pre-grant |
| US10156991B2 | Cited by | United States of America | Applicant |
| US10809917B2 | Cited by | United States of America | Applicant |
| US2014016470A1 | Cited by | United States of America | Pre-grant |
| US9491084B2 | Cited by | United States of America | Search report |
| US9614768B2 | Cited by | United States of America | Search report |
| US8412919B2 | Cited by | United States of America | Search report |
| US4658396A | Cites | United States of America | Search report |
| US4694487A | Cites | United States of America | Search report |
| US4847837A | Cites | United States of America | Search report |
| US5179550A | Cites | United States of America | Search report |
| US5239537A | Cites | United States of America | Search report |
| US5289458A | Cites | United States of America | Search report |
| US5329520A | Cites | United States of America | Search report |
| US5335227A | Cites | United States of America | Search report |
| US5361250A | Cites | United States of America | Search report |
| US5390188A | Cites | United States of America | Search report |
| US5410535A | Cites | United States of America | Search report |
| US5430726A | Cites | United States of America | Search report |
| US5448559A | Cites | United States of America | Search report |
| US5450403A | Cites | United States of America | Search report |
| US5459714A | Cites | United States of America | Search report |
| US5479608A | Cites | United States of America | Search report |
| US5490252A | Cites | United States of America | Search report |
| US5526492A | Cites | United States of America | Search report |
| US5586121A | Cites | United States of America | Search report |
| US5596723A | Cites | United States of America | Search report |
| US5598536A | Cites | United States of America | Search report |
| US5603029A | Cites | United States of America | Search report |
| US5606559A | Cites | United States of America | Search report |
| US5606664A | Cites | United States of America | Search report |
| US5646936A | Cites | United States of America | Search report |
| US5666488A | Cites | United States of America | Search report |
| US5708779A | Cites | United States of America | Search report |
| US5710549A | Cites | United States of America | Search report |
| US5734811A | Cites | United States of America | Search report |
| US5742587A | Cites | United States of America | Search report |
| US5742761A | Cites | United States of America | Search report |
| US5754552A | Cites | United States of America | Search report |
| US5764895A | Cites | United States of America | Search report |
| US5790554A | Cites | United States of America | Search report |
| US5793746A | Cites | United States of America | Search report |
| US5822300A | Cites | United States of America | Search report |
| US5828569A | Cites | United States of America | Search report |
| US5828879A | Cites | United States of America | Search report |
| US5838681A | Cites | United States of America | Search report |
| US5982744A | Cites | United States of America | Search report |
| US6009077A | Cites | United States of America | Search report |
| US6047323A | Cites | United States of America | Search report |
| US6067619A | Cites | United States of America | Search report |
| US6163543A | Cites | United States of America | Search report |
| US6381218B1 | Cites | United States of America | Search report |
| US6393483B1 | Cites | United States of America | Search report |
| US6535491B2 | Cites | United States of America | Search report |
| US6628649B1 | Cites | United States of America | Search report |
| US6665733B1 | Cites | United States of America | Search report |
| US6681261B2 | Cites | United States of America | Search report |
| US6711171B1 | Cites | United States of America | Search report |
| US6757297B1 | Cites | United States of America | Search report |
| US6763438B2 | Cites | United States of America | Search report |
| US6789118B1 | Cites | United States of America | Search report |
| Matrox, Matrox 100 Multiport NIC, pp. 1-3, 1998. | Non-patent | – | Search report |
| 3COM, Core Builder 5000 Distributed Management Module User Guide, May 1997. | Non-patent | – | Search report |
| 3COM Core Builder 5000 Distributed Management Module Commands Guide, May 1997. | Non-patent | – | Search report |
| 3COM CoreBuilder 5000 Etherflex Module User Guide, May 1997. | Non-patent | – | Search report |
| Matrox, Matrox 100 Multiport NIC, pp. 1-3, 1998. | Non-patent | – | Search report |
| 3COM, Core Builder 5000 Distributed Management Module User Guide, May 1997. | Non-patent | – | Search report |
| 3COM Core Builder 5000 Distributed Management Module Commands Guide, May 1997. | Non-patent | – | Search report |
| 3COM CoreBuilder 5000 Etherflex Module User Guide, May 1997. | Non-patent | – | Search report |
2 members in 1 office
Priority claims6
| Document | Office | Kind | Date |
|---|---|---|---|
| 88479497 | United States of America | A | |
| 88479497 | United States of America | A | |
| 70446403 | United States of America | A | |
| 08884794 | – | – | – |
| US19970884794 | – | – | – |
| US20030704464 | – | – | – |
Members2
| Document | Office | Kind | |
|---|---|---|---|
| US6393483B1 | United States of America | B1 | |
| USRE41397EThis record | United States of America | E |
92 transactions on the USPTO file
Allowed after 2 non-final rejections, 1 final rejection and 1 appeal.
- Non-final rejections
- 2
- Final rejections
- 1
- RCEs
- 0
- Appeals
- 1
Over time
Point at a mark for the transactionTransactions
| Event | Code | |
|---|---|---|
| Email NotificationEML_NTR | EML_NTR | |
| Change in Power of Attorney (May Include Associate POA)PA.. | PA.. | |
| Correspondence Address ChangeC.AD | C.AD | |
| Email NotificationEML_NTR | EML_NTR | |
| Change in Power of Attorney (May Include Associate POA)PA.. | PA.. | |
| Correspondence Address ChangeC.AD | C.AD | |
| Post Issue Communication - Certificate of CorrectionN423 | N423 | |
| Post Issue Communication - Certificate of CorrectionN423 | N423 | |
| Recordation of Patent Grant MailedPGM/ | PGM/ | |
| Email NotificationEML_NTR | EML_NTR | |
| Issue Notification MailedAllowedWPIR | WPIR | |
| Receipt into PubsR1021 | R1021 | |
| Dispatch to FDCD1935 | D1935 | |
| Dispatch to FDCD1935 | D1935 | |
| Application Is Considered Ready for IssuePILS | PILS | |
| Issue Fee Payment VerifiedN084 | N084 | |
| Issue Fee Payment VerifiedN084 | N084 | |
| Issue Fee Payment ReceivedIFEE | IFEE | |
| Electronic ReviewELC_RVW | ELC_RVW | |
| Email NotificationEML_NTF | EML_NTF | |
| Mail Notice of AllowanceAllowedMN/=. | MN/=. | |
| Notice of Allowance Data Verification CompletedAllowedN/=. | N/=. | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Email NotificationEML_NTR | EML_NTR | |
| Mail-Petition Decision - GrantedMPTGR | MPTGR | |
| Petition Decision - GrantedPTGR | PTGR | |
| Petition EnteredPET. | PET. | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Date Forwarded to ExaminerFWDX | FWDX | |
| Mail-Petition Decision - DismissedMPTDI | MPTDI | |
| Petition Decision - DismissedPTDI | PTDI | |
| Response after Non-Final ActionA... | A... | |
| Request for Extension of Time - GrantedXT/G | XT/G | |
| Petition EnteredPET. | PET. | |
| Mail Non-Final RejectionNon-final rejectionMCTNF | MCTNF | |
| Non-Final RejectionNon-final rejectionCTNF | CTNF | |
| Examiner Interview Summary Record (PTOL - 413)EXIN | EXIN | |
| Examiner Interview Summary Record (PTOL - 413)EXIN | EXIN | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Appeal Brief Review CompleteAPBR | APBR | |
| Date Forwarded to ExaminerFWDX | FWDX | |
| Appeal Brief FiledAP.B | AP.B | |
| Request for Extension of Time - GrantedXT/G | XT/G | |
| Notice of Appeal FiledN/AP | N/AP | |
| Request for Extension of Time - GrantedXT/G | XT/G | |
| Date Forwarded to ExaminerFWDX | FWDX | |
| Mail-Petition Decision - GrantedMPTGR | MPTGR | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Oath or Declaration Filed (Including Supplemental)C602 | C602 | |
| New or Additional Drawing FiledC614 | C614 | |
| Response after Final ActionA.NE | A.NE | |
| Petition EnteredPET. | PET. | |
| Mail Examiner Interview Summary (PTOL - 413)MEXIN | MEXIN | |
| Examiner Interview Summary Record (PTOL - 413)EXIN | EXIN | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Mail Final Rejection (PTOL - 326)Final rejectionMCTFR | MCTFR | |
| Final RejectionFinal rejectionCTFR | CTFR | |
| Date Forwarded to ExaminerFWDX | FWDX | |
| Cleared by OIPE CSRL194 | L194 | |
| Supplemental ResponseSA.. | SA.. | |
| Cleared by OIPE CSRL194 | L194 | |
| Miscellaneous Incoming LetterLET. | LET. | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Date Forwarded to ExaminerFWDX | FWDX | |
| Supplemental ResponseSA.. | SA.. | |
| Examiner Interview Summary Record (PTOL - 413)EXIN | EXIN | |
| Examiner Interview Summary Record (PTOL - 413)EXIN | EXIN | |
| Date Forwarded to ExaminerFWDX | FWDX | |
| Response after Non-Final ActionA... | A... | |
| Mail Notice of Informal or Non-Responsive AmendmentNINA | NINA | |
| Date Forwarded to ExaminerFWDX | FWDX | |
| Informal or Non-Responsive Amendment after Examiner ActionA.I. | A.I. | |
| Response after Non-Final ActionA... | A... | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Mail Non-Final RejectionNon-final rejectionMCTNF | MCTNF | |
| Non-Final RejectionNon-final rejectionCTNF | CTNF | |
| IFW TSS Processing by Tech Center CompleteTSSCOMP | TSSCOMP | |
| Preliminary AmendmentA.PE | A.PE | |
| Miscellaneous Incoming LetterLET. | LET. | |
| Preliminary AmendmentA.PE | A.PE | |
| Preliminary AmendmentA.PE | A.PE | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Mail-Petition Decision - GrantedMPTGR | MPTGR | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Notice of Reissue Published in Official GazetteNRE. | NRE. | |
| Application Is Now CompleteCOMP | COMP | |
| Cleared by OIPE CSRL194 | L194 | |
| Application Dispatched from OIPEOIPE | OIPE | |
| Preliminary AmendmentA.PE | A.PE | |
| Petition EnteredPET. | PET. | |
| Initial Exam Team nnIEXX | IEXX |
11 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| AssignmentAS | AS | |
| AssignmentAS | AS | |
| AssignmentAS | AS | |
| AssignmentAS | AS | |
| AssignmentAS | AS | |
| AssignmentAS | AS | |
| Fee paymentFPAY | FPAY | |
| Certificate of correctionCC | CC | |
| Fee payment procedurePAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITYFEPP | FEPP | |
| AssignmentAS | AS | |
| Certificate of correctionCC | CC |
Numbers
- Publication
- RE041397
- Publication, DOCDB
- RE41397
- Publication, EPODOC
- USRE41397E
- Application
- 10704464
- Application, DOCDB
- 70446403
- Application, EPODOC
- US20030704464
Titles
- English
- Method and apparatus for network interface card load balancing and port aggregation
Classification
- CPC, 3
- H04L41/22
- H04L41/0668
- H04L43/0817
- IPC, 2
- G06F15 16
- H04L12 24
- USPC, 9
- 709226000
- 370230000
- 370359000
- 370419000
- 370463000
- 709239000
- 709250000
- 710200000
- 714004100