Low latency communications

Low latency communication

Technical field

[0001] Embodiments of the present invention generally relate to communication networks and, more particularly, to detecting and retransmitting data packets that are dropped on a communication network.

Background technique

[0002] Conventional packet-based networks are designed to support reliable data transfer over unreliable communication networks. While such networks can achieve high performance at a reasonable cost, both system designers and end users expect that some packets will occasionally be discarded. Although there are some recovery mechanisms for retransmission of dropped packets, there is still a long delay and application performance is compromised. ^

[0003] In some instances, dropped packets can cause a delay of approximately 25 〇 - 300 ms. Various solutions have been developed for time-delay-critical operations such as block access storage. However, due to the significant shortcomings of these solutions, these solutions have not been widely implemented. For example, private networks have been developed that rarely drop data packets, such as Data Center Ethernet ("DCE"), InfiniBand, and the like. However, these implementations currently have size limitations, require specialized switches, and are otherwise expensive to implement. Therefore, these private networks have not been widely deployed.

Summary of the invention

Embodiments of the present invention generally relate to systems and methods for reducing packet delay that greatly improve prior art communication networks.

The features and advantages of the embodiments are set forth in the description which follows.

[0006] For example, systems and methods are provided for reducing communication latency. A communication session with the remote device is initiated. One or more message packets are sent to the remote device. Subsequently, it is determined if the additional message packet is scheduled to be sent to the remote device. In the event that no additional message packets are scheduled to be sent to the remote device, two lost detection packets are appended to the last message packet. The last message packet and the loss detection packet are sent to the remote device. Based on the acknowledgment message, it is further determined whether the last message packet was successfully received by the remote device. In the event that the message packet is not successfully transmitted, a retransmission of the lost message packet is performed.

DRAWINGS

Other embodiments, details, advantages and modifications will become apparent from the following detailed description of the preferred embodiments. _

1 is an overview of a network system including network elements that implement embodiments of the present invention and/or interact with embodiments of the present invention.

2 is a block diagram of a computer server/system in accordance with an example embodiment of the present invention.

FIG. 3 illustrates an example packet flow in accordance with an example embodiment of the present invention.

4 shows a flow chart for reducing communication latency in accordance with an example embodiment of the present invention.

[0012] FIG. 5 is a packet timing diagram of a sequence of exceptions of request packets and response packets exchanged between a client and a server, in accordance with an example embodiment.

[0013] FIG. 6 is a packet timing diagram of an example sequence including message packets exchanged between a client and a server, in accordance with an example embodiment.

detailed description

[0014] Systems and methods are provided for reducing communication latency. Embodiments of the present invention append one or more loss detection packets to a message packet (e.g., a session layer request or response) if no other message packets are scheduled to be sent in the communication session. Lost detection packets are used to quickly identify the loss of time sensitive packets and/or other important packets. If any loss detection packet in the lost detection packet is actually lost or discarded within the communication channel, it has little or no effect on the application, since the lost detection packet is typically a bait that does not carry latency-critical application data (decoy ) grouping. However, if the network discards the last data packet or the penultimate data packet in the communication session stream, the receipt of the loss detection packet triggers a retransmission process of the applicable communication protocol being used in the communication network.

[0015] In various embodiments, a communication session with a remote device is initiated. One or more message packets are sent to the remote device. Subsequently, it is determined if any additional message packets are scheduled to be sent to the remote device. In the event that no additional message packets are scheduled to be sent to the remote device, the loss detection packet is appended to the last message packet. The last message packet and the loss detection packet are sent to the remote device. Based on the receipt of the acknowledgment message, it is further determined whether the last message packet was successfully received by the remote device. In the event that the last detected packet was not successfully received, a retransmission of the last message packet is performed. Thus, the example embodiment can more easily detect lost message packets and thereby increase the throughput of the communication network.

[0016] Embodiments of the present invention provide systems and methods for reducing communication latency while using existing and widely implemented communication networks. Another technique known as tail loss detection ("TLP") has also been proposed. In the TLP, the sender sets a timeout for the acknowledgment from the recipient. If the sender does not receive an acknowledgment within a predetermined time, the sender resends the first unacknowledged packet. The TLP has the advantage that only the sender makes any changes, but has the disadvantage that the sender cannot set its timer for a short duration. The timer must be set to at least twice the duration of the connected smooth round trip time to account for the delayed acknowledgement. In addition, setting and testing short deadline timers is expensive. Finally, the TLP is sensitive to the delay acknowledgment algorithm used by the receiver. In contrast, embodiments of the present invention recover significantly from packet loss faster, and there is no overhead to set and cancel timers, and there is no requirement for the receiver's delayed acknowledgement algorithm.

[0017] In various embodiments, the term "message packet" is used and can be understood by one of ordinary skill to include request packets, response packets, and any packet data exchanged between communication devices on the network. For example, a message packet can include a protocol data unit ("rou") or a session layer pdu. Within a layered system, message packets may include information exchanged between entities of a given layer via services provided by the next lower layer to coordinate their joint operations. In another example, the message packet can represent a session layer message as well as a transmission control protocol segment.

1 is a diagrammatic view of a network system 100 that includes network elements that implement embodiments of the present invention and/or interact with embodiments of the present invention.

[0019] The network system 100 includes a client device 102, a network 114, and a service provider network 122 that is configured to exchange packet data with the server 116 using one or more communication protocols. Within client device 102, packet data may be generated by one or more applications 104. Example applications include block access storage, audio and/or video streaming, audio and/or video telephony, application streaming, and the like. Moreover, the packet data can be encoded, decoded, or otherwise configured using, for example, one or more network protocol layers 110 that provide communication functionality in accordance with various communication protocols.

[OOM] The server 116 is configured to exchange packet data with the client device 102. For example, server 116 can be configured to respond to requests from client device 102. In addition, client device 102 can communicate packet data via server 116 with a corresponding backend server 124 in service provider network 122. Backend server 124 may include various servers for data storage, block access storage, cloud computing, and the like. At the server side, for example, one or more network protocol layers 120 that provide communication functionality in accordance with various communication protocols can be used to encode, decode, or otherwise configure packet data. [0021] Each of client device 102 and server 116 may be any electronic device for communication, such as a smart phone, tablet, laptop, server, television, and the like. Different types of packet data from pure data to real-time media can pass through the network system 100. In general, some data can be sensitive to throughput, while other data, such as real-time media, can be sensitive to latency. Within the Transmission Control Protocol ("TCP") and Internet Protocol ("IP"), commonly referred to as TCP/IP, the server 116 can quickly detect lost packets as long as the client device 102 continues to generate requests. For example, by using the TCP/IP fast retransmission algorithm, the server 116 detects the loss of the packet and confirms to the client device 102 the last received sequence number. If the client device 102 receives multiple (eg, three) acknowledgments of the same serial number, the client device 102 retransmits the missing packets. If the client device 102 pauses while sending the request, the loss of the last packet or the penultimate packet in the session stream prevents the server 116 from receiving the complete request until the TCP/IP retransmission timer expires, which is usually It is a few hundred milliseconds (for example, 300ms). The retransmission timer is typically set to have such a long duration because it should be longer than the delayed acknowledgment timer (eg, 150 ms). Therefore, occasional delays that are orders of magnitude (eg, thousands of times) higher than the usual delay occur frequently.

[0022] Network 114, service provider network 122, and other components of network system 100 may utilize various communication protocols including the Internet, Ethernet, Wide Area Network ("WAN"), fabrics in storage systems, Wired and wireless protocols, as well as other known or anticipated network systems. Moreover, these embodiments enable time-delay critical applications to use conventional networks, such as Ethernet networks. The communication protocol can include TCP/IP. Internet RFC 793, incorporated herein by reference, generally describes TCP/IP functionality, and Internet RFC 5681, also incorporated herein by reference, generally describes TCP congestion control. Although the embodiments of the present invention are generally described in connection with TCP/IP, the present invention is not limited thereto, but can be applied to various network types.

[0023] As an alternative to direct implementation within TCP/IP, other implementations in conjunction with TCP/IP may be used. For example, a remote direct memory access ("RDMA") implementation can be layered over TCP/IP, such as the Internet Wide Area RDMA Protocol ("iWARP"). RFC 5040, which is incorporated herein by reference, generally describes RDMA functionality. Without implementing an embodiment of the present invention, thousands of times the latency of the normal situation can occasionally occur due to a lost packet iWARP request. Thus, embodiments can be applied to iWARP to provide more reliable communication and consistently improve communication latency.

[0024] Alternatively, an Internet Small Computer System Interface ("iSCSI") can be used. RFC 3720, incorporated herein by reference, generally describes iSCSI functionality. For iSCSI, as long as the iSCSI implementation asserts a push acknowledgment ("PSH") at the end of each request and response, the loss detection packet can be appended as a TCP/IP session option. In addition, architectural non-volatile memory ("NVMe") AVMe storage protocols can be implemented on RDMA architectures such as Inf iniBand, RDMA ("RoCE") on converged Ethernet, and iWARP.

[0025] In yet another alternative, the Network File System ("NFS") protocol can be implemented over TCP/IP. In an NFS embodiment, an NFS-based client device can insert one or more null protocol requests, and the NFS-based server can insert one or more null protocol responses to generate a loss detection packet. Therefore, the loss detection packet prevents the NFS-based client from experiencing a very long delay due to the expiration of the TCP retransmission timer due to the loss of the data packet.

Most Ethernet network interface controllers ("NICs") and TCP offload engines ("TOEs") implement a set of stateless offloads. For example, typical offloading includes checksum offload, segmentation offload, receive side scaling, and receive side aggregation ("RSC", also known as large receive offload ("LR")). Software implementation of generalized receive offload ("GR0"). Embodiments of the invention may be implemented in conjunction with any of these offload mechanisms. For example, the RSC merges the packets before receiving the TCP/IP stack to receive the packets. If the RSC combines two loss detection packets or merges the last packet of the flow with the loss detection packet, the receiving side will not transmit a sufficient number of duplicate ACK packets to trigger fast retransmission. To overcome this problem, the interoperation of the lost detection packet with the RSC requires setting the last packet of the stream and the PSH bits on the two loss detection packets.

[0027] For implementations above the TCP/IP level, embodiments may control or modify the packet structure of the session to ensure that the TCP implementation generates multiple loss detection packets when needed. Conventional TCP/IP socket application programming interfaces ("APIs") do not provide control over the partitioning of application flows across packets. However, embodiments may be applied within a kernel space application that can impose control over packet partitioning, in a user space application that uses a proprietary TCP/IP implementation, such as the Intel Data Plane Development Kit ("DPDK"), or at a high level of implementation. Application on a dedicated NIC for protocols such as iWARP or iSCSI. Each of these network environments provides sufficient control over the placement of data in the packet to generate a loss detection packet. [0028] Regardless of the communication protocol used, when no additional requests are scheduled, in some embodiments, small loss detection packets can be appended to the request, each loss detection packet being encoded in a single separate packet. If the network 114 discards the data packet including the loss detection packet, then a retransmission process (eg, TCP/IP fast retransmission) is triggered. RFC 6298, incorporated herein by reference, generally describes a retransmission timeout process. In such an implementation, the loss detection packet can be discarded. Alternatively, loss detection packets can be used to transmit non-time sensitive non-critical data and/or other data. If a lost detection packet is used to transmit packet data, the loss detection packet is not easily discarded.

2 is a block diagram of a computer server/system (ie, system 10) in accordance with an embodiment of the present invention. System 10 can be used to implement any of the network elements shown in Figure 1 as needed to achieve any of the functions of the embodiments of the invention disclosed in detail below. Although illustrated as a single system, the functionality of system 10 can be implemented as a distributed system. Additionally, the functionality disclosed herein can be implemented on separate servers or devices that can be brought together by a network. Additionally, one or more components of system 10 may not be included and alternative configurations may be used. For example, for the functionality of server 116 of FIG. 1, system 1 may be a server that generally does not require display 24 or one or more other components shown in FIG.

[0030] System 10 includes a bus 12 or other communication mechanism for communicating information, and a processor 22 coupled to bus 12 for processing information. Processor 22 can be any type of general purpose or special purpose processor. System 10 also includes a memory 14 for storing information and instructions to be executed by processor 22. Memory 14 may include random access memory ("RAM"), read only memory ("ROM"), static storage (such as a magnetic or optical disk), or any other combination of non-transitory or computer readable media. System 10 also includes a communication device 20, such as a network interface card, to provide access to the network. Thus, the user can interface directly with system 10 or remotely via a network or any other method.

[0031] The computer readable medium can be any available media that can be accessed by the processor 22, and the computer readable media includes volatile and nonvolatile media, removable and non-removable media, and communication media. Communication media may include computer readable instructions, data structures, program modules or other data in a modulated data signal (such as a carrier or other transport mechanism) and includes any information delivery.

[0032] The processor 22 can also be coupled via a bus 12 to a display 24, such as a liquid crystal display ("LCD"). Keyboard 26 and cursor control device 28, such as a computer mouse, can also be coupled to bus 12 to enable a user to interface with system 10 as desired.

[0033] In one embodiment, memory 14 stores software modules that provide functionality when executed by processor 22. The module includes an operating system 15 that provides operating system functionality to system 10. The module also includes a loss detection module 16 for providing a bait or loss detection packet and all other functions disclosed herein. In an example embodiment, the loss detection module 16 may implement the server 116 of FIG. 1 in conjunction with one or more of the remaining elements of FIG. System 10 can be part of a larger system, such as the added functionality of a "flash storage system" from Oracle Corporation. Thus, system 10 can include one or more additional functional modules 18 to include additional functionality. Database 17 is coupled to bus 12 to provide centralized storage for loss detection module 16 and additional function modules 18.

[0034] In one embodiment, the loss detection module 16 and/or the additional function module 18 may include an establishment module that establishes a communication session between the client device and the server using one or more loss detection packets. For example, the loss detection module 16 can be configured to append one or more loss detection packets to the message packet if no other message packets are scheduled to be sent in the communication session. Loss detection packets are used to more quickly identify loss of time sensitive message packets and/or other important message packets. The loss detection packet is used to trigger a retransmission process of the applicable communication protocol being used in the communication network.

FIG. 3 illustrates an example packet flow in accordance with an example embodiment of the present invention.

As shown in FIG. 3, data packet stream 300 includes message packets 310.0-310 • N and loss detection packets 320.1 and 320.2. Each packet may include a destination address 311, 321, a source address 312, 322, a sequence number 313, 323, and data payloads 314, 324. For loss detection packets 320_1 and 320_2, including data payload 324 is optional. For example, loss detection packets 320.1 and 320.2 can be used to transmit non-critical data and/or other data that is not time sensitive. Alternatively, the loss detection packets 320.1 and 320.2 may be simplified bait packets that are only used to determine whether the message packets 310.0-310. N have been discarded.

[0037] Depending on the communication protocol being used, the client side device and the server side device of the communication session negotiate the format of the loss detection packet. In TCP/IP, for example, when a TCP/IP client sends a data packet with the PSH flag set, the TCP/IP communication protocol can append multiple (eg, two) additional one-byte misses that the receiving server silently discards. Detect the packet. It may be preferable to use two loss detection packets to ensure that a sufficient number of acknowledgment messages are received to trigger a fast retransmission. Using two or more loss detection packets does not hinder network performance because of their small size. In such an implementation, each of the lost packets is prevented from containing one byte in order to increase the TCP/IP sequence number and to cause a packet loss retransmission algorithm. In TCP/IP, the sequence numbers 313, 323 are calculated as the last sequence number added to the length of the last packet.

While the embodiment describes the use of additional loss detection packets 320.0 and 320.1, the impact of their use on network performance is negligible. Modern Ethernet NICs and T0E easily handle millions of packets per second. Therefore, the Ethernet NIC or T0E can easily handle additional loss detection packets without compromising the data transmission rate.

FIG. 4 illustrates a flow diagram of a function 400 for reducing communication latency, in accordance with an example embodiment of the present invention. In some examples, the functions of the flowchart of FIG. 4 are implemented by software stored in a memory or other computer readable or tangible medium and executed by a processor. In other examples, the functionality may be by hardware (eg, by using an application specific integrated circuit ("ASIC"), a programmable gate array ("PGA"), a field programmable gate array ("FPGA"), etc.) or hardware and software. Any combination to perform.

At 410, function 400 initiates a communication session between the client device and the remote device. Embodiments of the invention may be used in conjunction with various communication protocols. These communication protocols may include TCPAPjDMAdSCS: [NVMe over fabrics on architecture, RDMs on architecture (such as InfiniBand, RoCE, and iWARP), and other communication protocols. [0041] Once the communication session has been established, function 400 transmits one or more message packets from the client device to the remote device. The request and response data packets exchanged between the client device and the remote device can be part of various applications such as block access storage, audio and/or video streaming, audio and/or video telephony, application streaming, and Any other response/request type application.

[0042] Subsequently, at 430, function 400 (eg, loss detection module 16 of FIG. 2) determines if any additional message packets are scheduled to be sent to the remote device. In the event that one or more additional message packets are scheduled to be sent to the remote device, function 400 returns to 420 for transmitting the message packet from the client device to the remote device. Otherwise, in the event that no additional message packets are scheduled to be sent to the remote device, at 440, one or more loss detection packets are appended to the last message packet. Here, the last message packet and the loss detection packet are sent to the remote device. T〇〇43] At 4f50, function 400 further determines if the last message packet was successfully received by the remote device. Function 4 may determine whether the lost detection packet was successfully received based on the acknowledgment message. In the case where the lost detection packet is not successfully received, at 4f5f5, retransmission of the last message packet and the loss detection packet is performed. For example, if the client device receives multiple (eg, three) acknowledgments with the same sequence number, then the client device retransmits the message packet that is considered missing. The last message packet can be retransmitted within 100 microseconds or less of its previous transmission. For example, the retransmission process can include TCP/IP fast retransmission. Otherwise, in the event that the lost detection packet is successfully transmitted, function 460 terminates the communication session at 460.

[0044] FIG. 5 is a packet timing diagram 500 of an example sequence including request packets and response packets exchanged between client device 502 and server 516, in accordance with an example embodiment. FIG. 5 includes network elements, such as client device 502 and server 516, as described herein with respect to FIG. 1 (client device 102 and server 116 in FIG. 1).

At 510, client device 502 sends one or more configuration request packets 520 (eg, message packets) to server 516. In turn, server 516 responds to configuration request packet 520 with one or more configuration response messages 521. The configuration request packet 520 and the configuration response packet 5M enable the client 502 and the server 516 to establish a communication session 523. Embodiments of the invention may be utilized in connection with various communication protocols such as redundancy, 1/1, 13031, architectural NVMe, architectural RDMA (such as IrrfiniBancURoCE and iWARP), and other communication protocols.

[0046] Once the communication session 523 is established, the client device 502 sends one or more request packets 524 to the server 516. In turn, server 516 responds to request packet 524 with one or more response messages 525. Request packet 524 and response message 525 may be part of various applications such as block access storage, audio and/or video streaming, audio and/or video telephony, application streaming, and any other response/request application.

[0047] Subsequently, the loss detection module of client device 502 determines if any additional request packets are scheduled to be sent to the remote device. If no additional request packets are scheduled, one or more loss detection packets are appended to the last request packet 526. Here, the last request packet and loss detection packet 256 is sent to the server 516. Finally, the loss detection module of client device 502 further determines whether loss detection packet 526 was successfully received by server 516. For example, when multiple (eg, three) acknowledgments of the same sequence number are received, the client device 502 determines if the loss detection packet 526 was not successfully received. In the event that the loss detection packet 526 is not successfully received, at 528, the last request packet and the appended loss detection packet are retransmitted. Here, for example, a TCP/IP fast retransmission algorithm can be used.

FIG. 6 is a packet timing diagram 600 including an example sequence of message packets exchanged between client device 602 and server 616, in accordance with an example embodiment. FIG. 6 includes network elements, such as client device 602 and server 616, as described herein with respect to FIG. 1 (client device 102 and server 116 in FIG. 1).

[0050] Each message packet 610, 620, 630 may include fields for a sequence number ("SN" in FIG. 6) and a length ("L" in FIG. 6) (such as the number of bytes). The acknowledgement message 640, 660, 680 also includes the sequence number of the acknowledged message packet or the last message packet received by the server 616. Moreover, each loss detection packet 650, 670 can include a field for a sequence number and length (eg, one byte). Although not shown in FIG. 6, each of the various message packet types includes a field for the source address, the destination address, and other fields as described above in FIG.

[0051] Client device 602 can send message packets 610, 620, 630 to server 616. Client device 602 also transmits loss detection packets 650, 670. In turn, server 616 responds to message packets 610, 620, 630 with one or more acknowledgment messages 640, 650, 660. Here, in this particular example, message packet 630 is discarded during transmission to server 616. Thus, client 602 receives multiple acknowledgments of message packet 620. When multiple (e.g., three) acknowledgments of the same sequence number are received, client device 602 determines if the message packet (e.g., message packet 620) was not successfully received. The receipt of the third repeated acknowledgement message results in a fast retransmission of the message packet (including message packet 630) with the sequence number after 1040.

[0052] In the example packet sequence 600, after receiving the message packets 610, 620, the server 616 acknowledges them with the acknowledgment packet 640, indicating that the server 616 has received the sequence number 1040. Server 616 has never received message packet 630. Upon receiving the loss detection packet 650 with sequence number 1117, the server 616 sends a duplicate acknowledgement packet 660, again indicating that it only received data up to the sequence number 1040. Client device 602 sends a loss detection packet 670. Upon receiving the loss detection packet 670, the server 616 sends another duplicate acknowledgement packet 680 for the sequence number 1040. Client device 602 has now received three acknowledgments for sequence number 1040 and responded by retransmitting the missing packet message(s) and/or loss detection packet after sequence number 1040. By applying an embodiment of the present invention, although message packet 630 is lost, client device 602 transmits message packets 610, 620, 630 to server 616 with minimal delay.

[0053] The various embodiments described herein can be implemented using a combination of hardware and software. For example, an embodiment can be implemented using a C language application that generates one or more messages from a client device to a server via a TCP socket. In this example, the C language application sets the TCP_N0DELAY parameter to disable the standard Nagle algorithm that aggregates small transfers into packets. The C language application generates messages from the client device to the server and waits for a one-byte response. Alternatively, the client device can generate two virtual one-byte packets. The sample pseudo code used to send the loss detection packet is as follows:

[0054] while (there are PDUs to send) then

[0055] sending a packet of the next PDU;

[0056] transmitting two lost detection packets;

[0057] if (received an acknowledgment packet indicating a lost packet) then [0058] retransmitting the missing packet;

[0059] As described herein, systems and methods are provided for reducing communication latency. Embodiments of the present invention append one or more loss detection packets to a message packet if no other message packets are scheduled to be sent in the communication session. Lost detection packets are used to more quickly identify the loss of time sensitive packets and/or other important packets. If any loss detection packet in the lost detection packet is actually lost or discarded within the communication channel, it has little or no effect on the application since the lost detection packet is typically a bait packet. However, if the network drops the last data packet or the penultimate data packet in the communication session stream, the loss detection packet is used to trigger the applicable communication protocol being used in the communication network (eg, at 100 microseconds or Retransmission process in less time. [0060] It will be apparent to those skilled in the art that various modifications and changes can be made to the addition of the loss detection packets of the present disclosure without departing from the spirit or scope of the present disclosure. Therefore, it is intended that the present disclosure cover the modifications and the modifications