Transporting operations of arbitrary size over remote direct memory access
15 claims: 3 independent, 12 dependent
- 1コンピューター実行可能命令を格納するコンピューター読み取り可能記憶媒体であって、プロセッサーによって前記命令を実行すると、リモート・ダイレクト・メモリー・アクセス(RDMA)によるデーター動作を使用してデーターを交換する方法を実行し、 前記プロセッサーがクライアント内に設けられ、 前記方法が、 前記クライアントに対し サーバーとの接続を作るステップと、 前記サーバーとの接続をネゴシエートするステップであって、前記ネゴシエー ト が、前記接続において前記サーバーが受信する最大バイト数を決める、ステップと、 前記サーバーへの接続に関連するデーターを送るパケット数を決定するステップと、 前記データーの断片化を使用するか否か判断するステップと、 前記データーの断片化を使用しないとき、第1プロトコル・パケットを前記データーと共に前記サーバーに送るステップと、 前記データーの断片化を使用するとき、 送信バッファーにおける最初のバイトを、前記第1プロトコル・パケットにおけるデーターとして送るように初期化するステップと、 前記第1プロトコル・パケットにおいてDataLengthフィールドおよびRemainingDataLengthフィールドを設定するステップであって、RemainingDataLengthフィールドは前記 サーバー が未だ受信していない断片化メッセージのバイト数に等しい、ステップと、 前記第1プロトコル・パケットを前記サーバーに送るステップと、 少なくとも1つの第2プロトコル・パケットに対して、 前記送信バッファー内にあるバイトを、 前記 少なくとも 1つの 第2プロトコル・パケットにおいてデーターとして送るように初期化し、 前記少なくとも1つの第2プロトコル・パケットにおける前記RemainingDataLengthフィールドを設定し、 前記少なくとも1つの第2プロトコル・パケットを前記サーバーに送る、ことを含む命令を繰り返すステップと、を含む、コンピューター読み取り可能記憶媒体。
- 2請求項1記載のコンピューター読み取り可能記憶媒体において、断片化を使用するか否か判断する前記ステップが、前記データーを送るためのパケットの数が1よりも大きいか否か判断するステップを含む、コンピューター読み取り可能記憶媒体。
- 3請求項2記載のコンピューター読み取り可能記憶媒体において、前記接続に関連するデーターを前記サーバーに送るためのパケット数を決定する前記ステップが、前記サーバーが前記接続において受信する最大バイト数が、送られるデーターのバイト数未満であるか否か判断するステップを含む、コンピューター読み取り可能記憶媒体。
- 4請求項1記載のコンピューター読み取り可能記憶媒体において、前記サーバーとの接続をネゴシエートするステップが、 受信をプリポストするステップと、 プロトコル・ネゴシエート要求を作るステップと、 前記プロトコル・ネゴシエート要求を前記サーバーに送るステップと、 ネゴシエート満了タイマーを設定するステップと、 前記ネゴシエート満了タイマーが満了する前に、前記プロトコル・ネゴシエート要求に対する応答が受信されたか否か判断するステップと、 前記ネゴシエート満了タイマーが満了する前に前記プロトコル・ネゴシエート要求に対する応答が受信されない場合、 前記サーバーとの接続を落とす ステップと、 前記ネゴシエート満了タイマーが満了する前に前記プロトコル・ネゴシエートに対する応答が受信された場合、プロトコル・ネゴシエート応答を受信するステップと、 前記プロトコル・ネゴシエート応答の有効性を判断するステップと、 前記プロトコル・ネゴシエート応答を処理するステップであって、前記プロトコル・ネゴシエート応答が、前記プロトコル・パケットを送るための少なくとも1つのクレジットを前記 クライアント に供給する、ステップと、を含む、コンピューター読み取り可能記憶媒体。
- 5請求項4記載のコンピューター読み取り可能記憶媒体であって、更に、 前記少なくとも1つの第2プロトコル・パケットを送るための十分なクレジットが存在する場合、前記少なくとも1つの第2プロトコル・パケットを送るステップと、 前記少なくとも1つの第2プロトコル・パケットを送るための十分なクレジットが存在しない場合、少なくとも1つの追加クレジットを前記サーバーに要求するステップと、 十分なクレジットが受信されたか否か判断するステップと、 十分なクレジットが受信されない場合、前記接続を終了するステップと、 十分なクレジットが受信された場合、前記少なくとも1つの第2プロトコル・パケットを送るステップと、を含む、コンピューター読み取り可能記憶媒体。
- 6リモート・ダイレクト・メモリー・アクセス(RDMA)によってサーバー・メッセージ・ブロック(SMB/SMB2)を使用してデーターを 、送信側と受信側との間で 交換するように構成されたシステムであって、 RDMAメッセージによってデーターを転送するように動作可能なRDMAネットワーク・インターフェース・カード(RNIC)と、 プロセッサーによって実行可能なコンピューター・プログラム命令を格納するように動作可能なメモリーと、を含み、前記プロセッサーが、前記RNICおよび前記メモリーと通信して、カーネルを実行するように動作可能であり、前記カーネルが、 第1プロトコル を実施する第1 クライアントを含み、この 第1 クライアントが、 2つ以上のRDMAメッセージにおいて断片化SMBデーターを伝達し、各RDMAメッセージがRemainingDataLengthフィールドを含み、RemainingDataLengthフィールドは前記受信側が未だ受信していない断片化メッセージのバイト数に等しく、 クレジットを交換するために受信バッファーを作る、ように動作可能であり、前記クレジットが、 前記 送信側が 前記 受信側に送ることができるRDMAメッセージ数を決定する、システム。
- 7請求項6記載のシステムであって、更に、前記第1 クライアント と通信可能なRDMAインターフェースを含み、前記RDMAインターフェースが、前記RNICを介してRDMA接続を開くように動作可能である、システム。
- 8請求項7記載のシステムであって、更に、 第2プロトコル を実施する第2 クライアントを含み、この第2 クライアント が、 RDMAダイレクト・データー転送を実行することをサーバーに命令するためにSMB2リード/ライト要求を作り、 前記SMB2リード/ライト要求を前記第1 クライアント に送るように動作可能であり、 前記第1 クライアント が、更に、 前記SMB2リード/ライト要求を第1プロトコル・データー転送パケットにカプセル化し、 前記第1プロトコル・データー転送パケットを前記サーバーに送るように動作可能である、システム。
- 9請求項8記載のシステムにおいて、前記SMB2リード/ライト要求が、RDMAチャネル記述子情報を特定するフィールドを含み、前記第1プロトコル が SMBダイレクト・プロトコルである、システム。
- 10請求項9記載のシステムにおいて、前記RDMAチャネル記述子情報が、誘導情報を含み、前記サーバーが、前記誘導情報を決定し、前記誘導情報によって特定されたメモリー・アドレスに対して、エンコードされていないアプリケーション・データーのRDMAライトまたはRDMAリードを実行する、システム。
- 11リモート・ダイレクト・メモリー・アクセス(RDMA)によってサーバー・メッセージ・ブロック(SMB/SMB2)を使用してデーターを 、送信側と受信側との間で 交換する接続を作るためのコンピューター実装方法であって、 第1SMBダイレクト・データー転送パケットを 前記受信側が 受信するステップであって、前記第1SMBダイレクト・データー転送パケットが、RemainingDataLengthフィールドおよびSMB2データーを含み、RemainingDataLengthフィールドは前記受信側が未だ受信していない断片化メッセージのバイト数に等しい、ステップと、 前記RemainingDataLengthフィールドが非ゼロであるか否か判断するステップと、 RemainingDataLengthフィールド前記がゼロであるとき、前記第1SMBダイレクト・データー転送パケットにおける前記SMB2データーを処理するステップと、 前記RemainingDataLengthフィールドが非ゼロであるとき、再組み立てバッファーを前記接続に割り当てるステップと、 前記SMB2データーを前記第1SMBダイレクト・データー転送パケットから前記再組み立てバッファーにコピーするステップと、 少なくとも1つの他のSMBダイレクト・データー転送パケットを受信するステップと、 前記SMB2データーを、前記少なくとも1つの他のSMBダイレクト・データー転送パケットから前記再組み立てバッファーにコピーするステップと、を含む、コンピューター実装方法。
- 12請求項11記載の方法において、前記SMBダイレクト・データー転送パケットが、CreditsRequested、CreditsGranted、DataOffset、およびDataLengthから成るグループからの1つ以上のフィールドを含む、方法。
- 13請求項12記載の方法において、前記CreditsGrantedフィールドが、前記SMBダイレクト・データー転送パケットの送信側にどれだけのクレジットが供給されたかを示す、方法。
- 14請求項11記載の方法において、クレジットが、前記送信側によって送ることができるRDMAメッセージの数に対する値である、方法。
- 15請求項11記載の方法であって、更に、 前記少なくとも1つの他のSMBダイレクト・データー転送パケットが最後のパケットであるか否か判断するステップと、 前記少なくとも1つの他のSMBダイレクト・データー転送パケットが最後のパケットである場合、前記再組み立てバッファーにおける前記SMB2データーを処理するステップと、 前記少なくとも1つの他のSMBダイレクト・データー転送パケットが最後のパケットではない場合、少なくとも1つの他のSMBダイレクト・データー転送パケットを受信し、前記少なくとも1つの他のSMBダイレクト・データー転送パケットからの前記SMB2データーを、前記再組み立てバッファーにコピーするステップを繰り返すステップと、を含む、方法。
Independent claims15
91 paragraphs, as filed
Conventional technology
[0001] Server Message Block (SMB) or its version, eg, a file access protocol such as SMB2, is used primarily to provide shared access to files and miscellaneous communication between nodes in a network. Applications-Can act as layered network protocols. Previously, SMB or SMB2 ran on Transmission Control Protocol (TCP) transport and traditional network infrastructure. Although SMB2 was very successful as a protocol for general purpose remote file access, SMB2 was not widely adopted for remote file access, which requires high throughput and low latency file input / output.
<p num="0002"> [0002] Remote Direct Memory Access (RDMA) is direct memory access from the memory of one computer to the memory of the other computer and does not require the operating system of the other computer. This direct transfer enables high-throughput, low-latency data transfer over the network and is especially useful in performance-critical deployments. When an application makes an RDMA read or write request, application data is delivered from the direct source memory buffer to the destination memory buffer using an RDMA-capable network adapter, with a central processing unit in this transfer. Does not require a (CPU) (or simply called a processor) or operating system. These RDMA transfers reduce latency and enable high speed message forwarding. Unfortunately, SMB2 isn't working with RMDA, so the benefits of RDMA aren't being exploited by systems that use SMB2.</p><p num="0003"> [0003] While addressing specific issues in this context, this disclosure is not intended to be limited to solving these specific issues.</p>
<p num="0004"> [0004] In general, embodiments are for achieving data operations, such as SMB2 operations (or other versions of SMB operations or file access protocol operations, for example), in addition to RDMA transport (atop). Regarding protocol and processing. In embodiments, the protocol definition specifies a new message for negotiating an RDMA connection and, for example, a new message for transferring SMB2 data using the negotiated connection. In one embodiment, the protocol for achieving SMB2 operation in addition to RDMA transport is the SMB direct protocol. However, other embodiments also provide other SMB protocols, SMB protocol versions, or other data operating protocols without departing from the gist and scope of the present disclosure. for). According to one embodiment, the new SMB direct message can include new header information. This new header information can include, but is not limited to, one or more of the following: CreditsRequested, CreditsGranted, Flags, Reserved, RemainingDataLength, DataOffset, and DataLength. This header information is used because RDMA transport supports receiving fixed-size messages by the receiver, and SMB2 messages are very large, ranging in size from about 100 bytes to over 1 million bytes. This is because even large messages can change widely. The SMB2 protocol has been modified to recognize the presence of RDMA capabilities, while the SMB direct protocol adds a new layer to the networking stack to logically request or respond to multiple individual RDMA messages. In conjunction with, it makes it possible to address both fixed-size constraints on RDMA messages and the uncertain size requirements inherent in SMB2 messaging. Changes to the SMB2 protocol and the addition of the SMB direct protocol allow direct transfer of data between peer memories. In embodiments, the SMB2 server can use RDMA to read from or write to the client's memory to perform direct data placement. The server performs an RDMA write to the client to complete the SMB2 read and an RDMA read to complete the SMB2 write. While the SMB direct protocol allows direct transfer of SMB / SMB2 between peer memories, the SMB direct protocol can be adapted to other protocols according to one embodiment. According to embodiments, SMB direct protocol bidirectional peer-to-peer</p><p num="0005"> [0005] RDMA transport also limits the number of messages that can be processed at any given time. Again, the value is fixed only by the receiver. To meet this RDMA requirement, embodiments provide the ability for peers to exchange or allocate "credits". A credit is a number requested and granted by each mutual peer in each protocol header and specifies the number of RDMA messages that the sender can send to the receiver. Credits are dynamic and are managed independently by each peer. The rules for managing and creating sufficient credits available to perform an SMB2 exchange are set out by the protocol in the embodiments disclosed herein.</p><p num="0006"> [0006] In an embodiment, the supply of independent two-way credits allows each peer to send a request or response without explicit negotiation or prior knowledge and consent by the receiving peer. Can be. By sending sequentially in relation to RDMA, you can send unexpectedly large messages without error in RDMA processing and without resorting to inefficient negotiated transfers when such a situation occurs. It can also be exchanged.</p><p num="0007"> [0007] According to embodiments, additional messages can also be used to negotiate protocol versions and other parameters. The negotiation request message can include, for example, fields for CreditRequested, Reserved, MinVersion, MaxVersion, OutboundSendSize, MaxInboundSendSize, and so on. On the other hand, the negotiation response message sent in response to the request message can include, for example, fields for CreditRequested, CreditsGranted, Version, Reserved, Status, OutBoundSendSize, InboundSendSize, and the like. These parameters support capability negotiation, resource end-to-end optimization, and compatibility with future improvements in the protocol.</p><p num="0008"> [0008] This abstract is provided to introduce in a simplified form a selection of concepts further described below in the detailed description chapter. This abstract is not intended to identify the essential features of the claimed subject matter, nor is it intended to be used at all to limit the scope of the claimed subject matter.</p>
[0011] The embodiments of the present disclosure can be further explained in an easy-to-understand manner by referring to the accompanying drawings. In drawings, similar reference numbers refer to similar items.<figref num="1">FIG. 1 shows an example of a logical representation of an environment or system in which SMB2 messages are (over) exchanged by RDMA according to an embodiment of the present disclosure.</figref><figref num="2A">FIG. 2A shows an example of a logical representation of a client system that sends SMB2 messages by RDMA according to an embodiment of the present disclosure.</figref><figref num="2B">FIG. 2B shows an example of a logical representation of a server system that receives SMB2 messages via RDMA according to an embodiment of the present disclosure.</figref><figref num="3A">FIG. 3A shows a logical representation of the message sent or received when exchanging data using SMB2 messages by RDMA according to an embodiment of the present disclosure.</figref><figref num="3B">FIG. 3B shows a logical representation of the message sent or received when exchanging data using SMB2 messages by RDMA according to an embodiment of the present disclosure.</figref><figref num="3C">FIG. 3C shows a logical representation of the message sent or received when exchanging data using SMB2 messages by RDMA according to an embodiment of the present disclosure.</figref><figref num="3D">FIG. 3D shows a logical representation of a message sent or received when exchanging data using SMB2 messages by RDMA according to an embodiment of the present disclosure.</figref><figref num="3E">FIG. 3E shows a logical representation of the message sent or received when exchanging data using SMB2 messages by RDMA according to an embodiment of the present disclosure.</figref><figref num="4A">FIG. 4A shows a flow diagram showing the operating characteristics of a process that negotiates communication using SMB2 by RDMA according to an embodiment of the present disclosure.</figref><figref num="4B">FIG. 4B shows a flow diagram showing the operating characteristics of a process that negotiates communication using SMB2 by RDMA according to an embodiment of the present disclosure.</figref><figref num="4C">FIG. 4C shows a flow diagram showing the operating characteristics of a process that negotiates communication using SMB2 by RDMA according to an embodiment of the present disclosure.</figref><figref num="5A">FIG. 5A shows a flow diagram showing the operating characteristics of a process of exchanging data using SMB2 by RDMA according to an embodiment of the present disclosure.</figref><figref num="5B">FIG. 5B shows a flow diagram showing the operating characteristics of a process of exchanging data using SMB2 by RDMA according to an embodiment of the present disclosure.</figref><figref num="5C">FIG. 5C shows a flow diagram showing the operating characteristics of a process of exchanging data using SMB2 by RDMA according to an embodiment of the present disclosure.</figref><figref num="5D">FIG. 5D shows a flow diagram showing the operating characteristics of a process of exchanging data using SMB2 by RDMA according to an embodiment of the present disclosure.</figref><figref num="6">FIG. 6 shows a flow diagram showing the operating characteristics of a process of exchanging data using RDMA direct data transfer according to an embodiment of the present disclosure.</figref><figref num="7">FIG. 7 shows an example of a calculation system that can realize the embodiment of the present disclosure.</figref>
[0018] Hereinafter, the present disclosure will describe the embodiments in more detail with reference to the accompanying drawings. Specific embodiments are shown in the accompanying drawings. However, other forms can also be embodied in many different forms, and inclusion of specific embodiments in the present disclosure is limited to embodiments described herein. Do not interpret it as. Conversely, the embodiments shown in the drawings are through and complete and are included to provide disclosure that fully conveys the intended scope to those skilled in the art. Dashed lines may be used to indicate any component or behavior.
[0019] Generally speaking, embodiments relate to systems, methods, and protocols for exchanging SMB2 data via RDMA connections. RDMA has advantages when transferring data. For example, RDMA transfers from one memory to the other device or system memory. These transfers do not require a processor and therefore reduce the overhead involved in transmitting the data. In addition, because transport management does not require a processor, RDMA transfers data in fewer clock cycles. That is, RDMA provides low latency, high bandwidth connectivity.
[0020] Generally, a version of SMB, such as SMB2, is an application-layer network protocol used to provide shared access for files and miscellaneous communications between nodes in a network. SMB and SMB2 are examples of file access operating protocols. SMB2 allows the transfer of data in exchanging messages between a client (s) and a server (s). The description of some embodiments herein refers to SMB2. However, in other embodiments, any virgin of SMB or other file access operating protocol may be used without departing from the gist or scope of the present disclosure.
[0021] In general, due to differences in behavior, SMB2 (or SMB) no longer operates with RDMA and instead uses TCP to transport data. The present embodiment creates a system and protocol for exchanging SMB2 messages with RDMA-capable network protocols. First, clients and / or servers use SMB2 components to enable each other's capabilities or abilities) is discovered. Clients and / or servers can be collectively referred to as peers. In one embodiment, the request is sent by the client asking for information about the server. The responding server is the number of network interface cards (NICs) that the server has, the Internet Protocol (IP) address of the NIC), the speed of the NIC, whether the NIC is RDMA capable, and / or , Probably answer with information about the IP address. In embodiments, the requesting client can use this information to determine how to interface with the server.
[0022] If the server is RDMA capable, the RDMA capable client can then negotiate an RDMA connection with that server in accordance with embodiments of the present disclosure. The new interface, referred to in embodiments as the RDMA interface, is the interface between the RDMA-capable network adapter and other system components (including SMB clients). In addition, a new module called SMB Direct Client / Server is added to the stack. First, the SMB direct client pre-posts reception, perhaps in response to a capability request, by reserving at least a portion of the memory buffer to receive messages from other peers. The SMB direct client can then send an SMB direct negotiate request packet. An SMB direct negotiate request initiates the process of creating an SMB direct connection between peers. According to embodiments, the SMB direct negotiate request includes one or more fields that determine how the RDMA connection works.
[0023] In the embodiment, part of the SMB Direct Negotiate request is a "credit" request. When RDMA transfers directly from one memory to another, the receiving peer reserves a space in the memory buffer for the transmitting peer to contain the transferred data. Memory buffers are allocated on a block-by-block basis (eg, 1 Kbyte block). In the embodiment, this allocation is set so that the block size does not change after the SMB direct connection is made. In other embodiments, no allocation is set, in which case the block size can change. If allocation is set, none of the transfers to the memory buffer can exceed the block size. To transfer SMB2 data that is larger than the block size, the sender sends two or more packets. These are stored in two or more allocations of memory buffers. The sending peer requests credit to reserve memory allocation. The number of credits required is determined by local policy and is not necessarily affected by the composition of the message. Each credit represents a block of memory buffer and therefore a message that the requester can send to other peers.
[0024] In embodiments of the present disclosure, the server can send a response, eg, an SMB direct negotiate response, to the requester. This SMB direct negotiate response also includes various fields that define the RDMA connection. In response to this request, the SMB Direct Negotiate response provides a number of credits to the client. In addition, SMB direct negotiate responses can also request credits to reserve allocations in the client's memory buffer. In an embodiment, an RDMA connection is made by exchanging SMB direct negotiate requests and SMB direct negotiate responses. You can then exchange SMB2 data over this connection.
[0025] According to embodiments, exchanging SMB2 data over an RDMA connection involves the transfer of at least one packet. If the SMB2 data sent is smaller than the block size, then only one packet, called the SMB direct data transfer packet, can be sent. However, if the SMB2 data to be sent is larger than the block size, one embodiment can send two or more SMB direct data transfer packets. RDMA handles packet reception according to the sequence. That is, the second packet received in communication is placed immediately after the first received packet. To take advantage of this RDMA, SMB direct data transfer packets include a field that tells the total amount of SMB2 data sent and a field that tells how many packets are left after the current packet. In this way, the SMB direct peer can determine when the SMB transfer is complete and can reassemble the SMB2 data. The SMB direct protocol, therefore, in embodiments, overcomes the problems associated with transferring SMB2 data by RDMA, while preparing for rapid transfer of SMB2 with low latency and low overhead.
FIG. 1 shows Example 100 of a logical environment or system that exchanges SMB2 data over an RDMA connection according to an embodiment disclosed herein. Connected peers (also called client 102 and server 106) 102 and 106 are RDMA over an RDMA connection. SMB2 data can be moved between NICs (RNICs) 108a and 108b over network 104. The connecting peer may be any computer system, as described, for example, with respect to FIG. Connected peers are shown in Figure 1 as client / application server 102 and file server 106. However, these peers are only presented as an example. In embodiments, any type of client (s) or server (s) can act as a connecting peer. That is, an RDMA connection can be, for example, between multiple clients, multiple servers, server farms (s), server clusters (s), message servers (s), or clients. It may be between (s) and the server (s). The client / application server 102 and the file server 106 are presented by way of example only for the purpose of understanding the teachings of the embodiments disclosed herein.
[0027] While SMB2 data is being moved over network 104, network 104 may be as described, for example, with reference to FIG. The network 104 is shown as a separate network, but may be any type of network commonly understood by those skilled in the art. According to one embodiment, the network may be a global network (eg, the Internet or the World Wide Web, ie the "web" for short). It may also be a local area network, such as an intranet or a wide area network. According to embodiments, communication over network 104 follows one or more standard packet-based formats, such as H.323, IP, Ethernet®, and / or ATM.
[0028] Further, in embodiments, the RNIC 108 (a and / or b) may be any network interface card, network adapter, and / or network interface controller that supports RDMA. There are various distributors that offer RNIC. For example, iWARP or InfiniBand is a network protocol that supports RDMA. RNIC can support RDMA, allowing direct transfer of data from memory 110a to memory 110b and vice versa. These data transfers do not require or include the monitoring of processor 112a or 112b. That is, according to embodiments of the present invention, RDMA transfers are high bandwidth, low latency, and low overhead. The processor 112 and memory 110 may be, for example, as described with respect to FIG.
[0029] FIG. 1 shows a typical environment for exchanging SMB2 data via RDMA, and FIG. 2A shows peers transmitting or receiving SMB2 data over an RDMA connection, according to embodiments disclosed herein. An example 102 is shown. In this example, peer 102 is a client and / or application server. The various components of client 102 can include hardware and / or software according to embodiments of the present disclosure. However, for purposes of illustration, the components will be described below as software modules. In embodiments, client 102 includes, but is limited to, one or more of kernel 202a, at least one application 220, memory buffer 222a, one or more timers 224a, and / or one or more settings 226a. It is not done. In embodiments, the "kernel" is the core of the operating system, where memory, files, and peripheral devices are managed, applications are triggered and launched, and system resources are allocated.
[0030] In embodiments, kernel 202a is a WIN32® file application programming interface (API) or equivalent 204, SMB2 module (shown as SMB2 client 208), SMB direct module (SMB). Can include, but is not limited to, one or more of the direct client 214a) and the RDMA interface 216a. The SMB direct module and RDMA interface 216a are introduced to perform the methods and processes described herein. Describe the modules, components, and / or interfaces as follows:
In embodiments, the WIN32® file API 204 can be an application between kernel 202a and one or more user applications 220. In one embodiment, the WIN32® file API204 is MICROSOFT. A set of APIs available in the WINDOWS® operating system. Almost all WINDOWS® programs interact with WINDOWS® to perform their functions. Embodiments of the WIN32® File API 204 provide access to resources available to the WINDOWS® system, such as, for example, file systems, devices, and / or error handling. The WIN32® file API 204, according to embodiments, can eliminate the need for access to features added to the kernel. In embodiments, the WIN32® file API 204 also allows the system to perform actions on remote files that use the underlying file access functionality. On the other hand, the basic file access function uses various networking capabilities to deal with remote access.
[0032] In an embodiment, the SMB2 client 208 manages communication between the application and the interface provided by the SMB2 server 212b. Communication between the operating system and the device driver is primarily via I / O request packets (IRPs), as the speed at which the device operates may not match the operating system. These packets are similar, for example, to network packets or WINDOWS® message packets. These packets are passed from the operating system (s) to specific drivers and between drivers. In an embodiment, the SMB2 client 208 can redirect I / O requests to network resources and compose SMB messages to communicate over the network. The SMB2 client 208 propagates the SMB packet to the SMB direct client 214a in order to exchange the SMB packet or data with the RMDA. Similarly According to one embodiment, the SMB2 server 212b can also use IRP to send file requests from the SMB2 client 208 to server storage.
[0033] In an embodiment, SMB Direct Module 214 is an instance created from the SMB Direct Network Provider Interface (NPI) in kernel 202. SMB Direct Module 214 exposes the API (called SMB Direct NPI) to SMB2 Client and SMB2 Server Modules. The SMB2 client / server module uses this SMB direct NPI to make requests to send or receive data over the SMB direct connection. According to embodiments, the SMB direct module 214 implements the SMB direct protocol and is located between the SMB2 client / server module and the underlying RDMA interface. SMB Direct NPI enables the SMB Direct Protocol. SMB Direct NPI, among other things, creates and destroys SMB direct connections, sends and receives data over SMB direct connections, registers / unregisters memory, and performs RDMA read / write operations with peers. Can be run over an SMB direct connection, receive a notification when the SMB direct connection is disconnected by a peer, and serialize / deserialize SMB2 packets for transmission over the SMB direct connection. An SMB direct module 214 is created to perform these tasks. The SMB direct module 214 can manage the transmission and withdrawal of SMB2 data stored by the RDMA protocol from the memory buffer 222. That is, the SMB direct module 214 simply converts the data from SMB2 data to RDMA and vice versa from RDMA to SMB2. The SMB direct module 214 communicates with another new module, the RDMA interface 216, to perform the operations of the embodiments of the present disclosure.
[0034] According to an embodiment, the SMB direct module 214 performs various functions. The SMBDirectReceiveEvent callback function notifies the SMB2 client 208 or SMB2 server 212B that a message has been received on the SMB direct endpoint. The SMBDirectDisconnectEvent event callback function notifies the SMB2 client 208 or SMB2 server 212b that the connection at the endpoint has been lost. The SMBDirectAcceptEvent event callback function notifies the SMB2 server 212b that an incoming connection at the listening endpoint has been accepted. The SMBDirectListen function listens for incoming connections at a given local address (listen) for) Create a listener endpoint. The SMBDirectCloseEndpoint function closes the endpoint and frees any associated resources. The SMBDirectConnect function connects the endpoint to a remote SMB direct transport address. The SMBDirectDisconnect function disconnects the endpoint from the remote SMB direct transport address. The SMBDirectSend function sends a buffer of data to a remote SMB direct peer. The SMBDirectSend function sends a buffer of data to a remote SMB direct peer. The SMBDirectRegisterMemory function allows the SMB2 client 208 to register a memory buffer for use in RDMA read / write operations. The SMBDirectUnregisterMemory function unregisters the memory buffer previously registered by the SMBDirectRegisterMemory function. The SMBDirectRdmaRead function causes RDMA to read data directly from the memory of the remote peer to which the endpoint is connected. The SMBDirectRdmaWrite function causes RDMA to write data directly to the memory of the remote peer to which the endpoint is connected. These functions and their operations will be described in relation to FIG.
[0035] In one embodiment, the RDMA interface 216 is a new interface for interfacing with the distributor-specific RDMA function of the RNIC. RDMA interface 216 can provide access to the functions of the RDMA device. Functions of the RDMA device include listening to the port for receiving SMB direct packets and can supply SMB2 data to the SMB direct module 214. In one embodiment, the RDMA device can include a kernel mode RDMA module to manage communication over the RDMA connection. In addition, RDMA devices can also include an RDMA access layer and extensions to access and listen to ports that send RDMA messages. According to embodiments, the proxy driver can interface with the RNIC hardware driver.
[0036] In embodiments, the user application 220 may be any software executed by the processor for the user or other software. Examples of user application 220 include web browsers, email, and the like. These user applications 220 interface with kernel 202a to send and receive data, in particular from remote storage locations such as file server 106.
[0037] In one embodiment, the memory buffer 222 may be any type of memory, as described with reference to FIG. Memory buffer 222 can be used to receive SMB direct messages and / or SMB2 data sent within a message, and assembles outgoing SMB direct messages before sending these messages. Can be used for (stage). That is, according to the embodiment, the memory buffer 222 can be divided into blocks as described below.
[0038] In the embodiment, the timer 224 is a set of clocks that can count down from a predetermined time to 0. That is, timer 224 represents the stored data and the clock function executed by the processor. The expiration of timer 224 can trigger one or more functions on the RDMA interface or by the SMB direct module 214. In other embodiments, the timer may count from 0 to a threshold, or perform some other type of counting operation. Some of these timers can include a SendCreditGrantTimer. This is a timer that starts when SendCreditCount reaches zero and operates in timer section 224. Peers connected apart must grant additional Send Credits before this timer expires. The SendCreditGrantTimer can also regulate the amount of time a client / server waits for a peer to grant additional SendCredits. When the client / server discovers that the packet cannot be sent to the peer because the value of SendCreditsCount is zero, the client / server sets a timer that expires after a predetermined number of seconds, according to embodiments. To do. If this timer expires before SendCredits is available, the client / server will disconnect. In one embodiment, an idle connection timer regulates the amount of time the client / server waits for a peer to send a packet. If no packets have been received from the peer during the last few seconds, the client / server keeps alive. Send request) to the peer and set the KeepAliveResponse timer. If no response is received before the KeepAliveResponse timer expires, the connection will be disconnected according to the embodiments of the present disclosure.
[0039] According to embodiments of the present invention, the KEEPALIVE_REQUESTED flag can also be set for any SMB direct data transfer over an SMB direct connection. In one embodiment, the KEEPALIVE_REQUESTED flag is a request from the receiving peer to respond to the transmitting side as soon as possible so that the transmitting side knows that the receiving side is still connected and ready to respond. One or more systems can attempt to time out the connection without exchanging messages. Therefore, messages with the KEEPALIVE_REQUESTED flag can stay connected. In an alternative embodiment, messages set with the KEEPALIVE_REQUESTED flag can be used to request or receive credits.
[0040] In one embodiment, the configuration 226 store stores and retrieves data related to SMB message exchange by RDMA. The setting 226 can be stored in any type of memory or storage device as described with respect to FIG. As an example, these settings are the number of seconds the credit replenishment timer expires after being set, the maximum size of SMB direct data transfer packets that a peer may receive from other peers, or the peer grants to other peers. It is possible to include a limit on the number of credits to be sent.
[0041] The SMB_DIRECT_ENDPOINT structure can, in embodiments, be an opaque structure that represents an SMB direct endpoint. SMB direct endpoints are similar in functionality to, for example, network sockets. According to one embodiment, the SMB2 client 208 or SMB2 server 212b cannot directly access the members of this structure, but can access it through the SMB direct module 214. The SMB_DIRECT_ENDPOINT structure can contain data for various operations. MwReleaseList is a list of memory windows that can be released and returned to remote connected peers. These memory windows are associated with completed RDMA read / write operations. ReceiveCreditCount is the number of received credits that are to be granted to the remote connecting peer. The endpoint host can have at least this number of receptions pending at the endpoint. SendCreditCount is the number of outgoing credits currently held by the endpoint host. A remote-attached peer can have at least this number of receptions pending at the endpoint. PendingRdmaReadCount is the number of RDMA read operations that have started but have not yet completed at this endpoint. PendingRdmaReadLimit is the maximum number of RDMA read operations that can be pending at the endpoint at the same time. The DeferredInitiatorOpQueue is, in the embodiment, a queue of initiator operations that has been postponed because the endpoint resources for issuing them are not currently available. NdkQp is a queue pair that represents the receive and initiator request queues. object). NdkReceiveCq is the NDK reception completion queue. Receive request completions are placed in this queue. NdkReceiveQueueCapacity is the maximum number of receive requests that can be put on hold at the endpoint at the same time. NdkInitiatorCq is the NDK initiator completion queue. Send, bind, fast register, read, write, and invalidate request completions are, for example, queued in this queue. In an embodiment, the NdkInitiatorQueueCapacity is the number of send, join, fast register, read, write, and invalidation requests that can be put on hold at the endpoint at the same time.
[0042] FIG. 2B shows example 106 of a peer transmitting or receiving SMB2 data over an RDMA connection, according to an embodiment disclosed herein. In this example, peer 106 is a file server. The various components of file server 106 can include software and / or hardware. Although these components are described as software modules in the embodiment, other embodiments also include other types of modules. In embodiments, file server 106 includes, but is limited to, kernel 202b, NTFS232, memory buffer 222b, one or more timers 224b, and / or one or more of one or more settings 226b. Not. Some of these functions may be the same as or similar to those described in FIG. 2A.
[0043] According to embodiments, of kernel 202b (and / or as described with respect to FIG. 2A), input / output (I / O) manager 206b, SMB2 server 212b, SMB direct server 214b, and RDMA interface 216b. Includes, but is not limited to, one or more of them. Some of these functions may be the same as or similar to those described in FIG. 2A.
[0044] In an embodiment, the SMB2 server 212b is a driver that implements the server side of the SMB2 protocol relative to the MICROSOFT® server. Other embodiments also include other types of servers. The SMB2 server 212b can start and use the SMB to exchange data, supply data, and receive data from I / O manager 206b. In an embodiment, the SMB2 server 212b can send or receive SMB2 data over a network connection. The SMB2 server 212b functions to communicate over the server network. That is, according to the embodiment, the SMB2 server 212b supplies SMB2 data for transmission and receives SMB2 data from network transmission.
[0045] In embodiments, the New Technology File System (NTFS) 232 may be a standard file system. NTFS provides metadata and advanced data to improve performance, reliability, and disk space utilization, and for additional enhancements such as security access control lists (ACLs) and file system journaling (journaling). Includes support for using structures. NTFS works to organize and store file data for one or more clients. This data file can be supplied to the client via communication with the client, similar to SMB2 data transfer by RDMA.
[0046] According to an embodiment, SMB Direct makes an RDMA connection to exchange SMB2 data. This protocol creates an RDMA connection through a negotiation process. After peers 102 and 106 negotiate an RDMA connection, either peer 102 or 106 can send SMB2 data over the RDMA connection.
[0047] According to the embodiment, the first SMB direct negotiate request packet 300 is shown in FIG. 3A. On the other hand, in the embodiment, the associated SMB direct negotiate response packet 336 is shown in FIG. 3B. Further, according to the embodiment, the SMB direct data transfer packet 338 used to transfer the data is shown in FIG. 3C. Each message or packet can be, for example, created, transmitted, stored, and / or received. In embodiments, each of these packets or messages can include parts or fields that store different data.
Moving on to FIG. 3A, the SMB Direct Negotiate Request Packet 300 according to an embodiment of the present disclosure is shown. The SMB Direct Negotiate Request Packet 300 can include, but is not limited to, for example, one or more of the following fields: MinVersion302, MaxVersion304, Reserved306, CreditsRequested308, PreferredSendSize310, MaxzReceiveSize312, and / or MaxSmb2MessageSize314. The SMB Direct Negotiate Request Packet 300 can also contain more or fewer fields than the fields shown in Figure 3A, as represented by the ellipse 316.
[0049] In one embodiment, the MinVersion field 302 can contain a value for the lowest SMB direct protocol version supported by the client / requester 102. The MaxVersion field 304 can store the highest SMB direct protocol version supported by the client / requester 102. This value may be greater than or equal to the value of MinVersion field 302. In embodiments, the client / requester supports all (included) protocol versions in the range between the value in the MinVersion field 302 and the value in the MaxVersion field 304. Reserved field 306 is simply a field reserved for future requests that are not yet known and is not used by the client according to one embodiment.
[0050] According to an embodiment, the CreditsRequested field 308 contains a value for the number of outgoing credits that the client / requester 102 is requesting from the server / receiver 106. In an embodiment, the value in the CreditsRequested field 308 is greater than zero (0) to ensure that SMB2 data is sent in subsequent messages. However, the value in the CreditsRequested field 308 may be any number and, according to embodiments, can be set based on average use. In an alternative embodiment, the value in the CreditsRequested field 308 can be at least based on the size of the SMB message sent. In yet other embodiments, other or additional factors can also be considered when requesting credit. Also, in embodiments, large messages can also be forwarded using RDMA.
[0051] The PreferredSendSize field 310 can include the size (perhaps in bytes) of the largest SMB direct data transfer packet 338 that the client / requester 102 wants to be able to send to the server / receiver 106. The MaxReceiveSize field 312 contains the size of the largest SMB direct data data transfer message that the client / requester 102 accepts from the server / receiver 106 (perhaps in bytes). That is, one SMB2 data transfer does not exceed the memory allocation. In embodiments, this value is greater than or equal to the threshold of 128. Threshold 128 is at least the size of the SMB direct header in data transfer packets and small SMB2 messages. The MaxSMB2MessageSize field 314 can contain, according to one embodiment, the size of the largest SMB2 protocol message (perhaps in bytes) that the client / requester accepts from the server / receiver 106. This value is predetermined and set by the client 102. In the embodiment, the MaxSmb2MessageSize field 314 must not be larger than the total size of the memory buffer 222a. In this way, SMB messages can be prevented from overflowing memory buffer 222a. However, according to one embodiment, the MaxSmb2MessageSize field 314 may be less than the amount of memory in the memory buffer 222a, as determined by the user.
Moving on to FIG. 3B, the SMB direct negotiate response packet 336 according to the embodiment is shown. The SMB direct negotiate response packet 336 can be sent by peer 106 to complete the negotiation for the RDMA connection. The SMB Direct Negotiate Response Packet 336 can include, but is not limited to, one or more of the following fields: For example, MinVersion302b, MaxVersion304, PreferredVersion318, Reserved306b, CreditsRequested308b, CreditsGranted320a, Status322, PreferredSendSize310b, MaxReceiveSize312b, and / or MaxSMB2MessageSize314b. The SMB direct negotiate response packet 336 can also contain more or fewer fields than the fields shown in Figure 3B, as represented by the ellipse 324.
[0053] According to embodiments, the MinVersion field 302b contains the lowest SMB direct protocol value supported by the server / receiver 106. The MaxVersion field 304b stores the highest SMB direct protocol version supported by the server / receiver 106. This value may be greater than or equal to the value of MinVersion field 302b. In an embodiment, the server / receiver 106 supports all protocol versions in the range (included) between the value in the MinVersion field 302b and the value in the MaxVersion field 304b. PreferredVersion field 318 stores the value of the common SMB direct protocol version. The value of PreferredVersion field 318 is, in the embodiment, between the range specified by MinVersion field 302a and MaxVersion field 302b of the client's SMB direct negotiate request packet 300. In other embodiments, the value of PreferredVersion field 318 is between the range specified by MinVersion field 302b and MaxVersion field 304b of the server's SMB direct negotiate response packet 336. According to embodiments, Reserved field 306b is simply a field reserved for future unknown requests and is not used by the server.
[0054] The CreditsRequested field 308b contains, according to the embodiment, a value for the number of outgoing credits that the server / receiver 106 is requesting the client / requester 102. In an embodiment, the value in the CreditsRequested field 308b is greater than zero (0) to ensure that SMB2 data is sent in subsequent messages. However, the value in the CreditsRequested field 308b may be any number and may be set based on average usage. In an alternative embodiment, the value in the CreditsRequested field 308b may be based on the size of the SMB message being sent, or on some other factor. According to embodiments, a large message may be sent when one credit is requested. The CreditsGranated field 320a contains the number of credits granted by the server / receiver 106 to the client / requester 102. In an embodiment, the value of the CreditsGranted field 320a is greater than zero (0) to cause the client / requester 102 to send the next SMB direct message.
[0055] In the embodiment, the status field 322 contains at least one flag or value. In one embodiment, status field 322 comprises one or two values, ie status successful or status unsupported. The success flag indicates that the server / receiver 106 has accepted the client / requester SMB direct negotiate request packet 300. The unsupported flag indicates that the server / receiver 106 rejected the client / requester 102 SMB direct negotiate request packet 300.
According to embodiments, PreferredSendSize310b can include the size (perhaps in bytes) of the largest SMB direct data transfer packet 338 that the server / receiver 106 wants to be able to send to the client / requester 102. PreferredSendSize310b, in the embodiment, is less than or equal to the MaxReceiveSize field 312a in the SMB direct negotiate request packet 300. That is, the packet size sent by the server / receiver 106 must not be larger than the memory buffer 222a allocation of the client / requester 102.
[0057] In an embodiment, the MaxReceiveSize field 312b contains the size of the largest SMB direct data transfer message (perhaps in bytes) that the server / receiver 106 accepts from the client / requester 102. This value should be equal to the block or predetermined allocation of memory buffer 222b set by the server / receiver 106. That is, in the embodiment, the SMB2 data transfer does not exceed this memory allocation. In embodiments, this value is greater than or equal to the threshold of 128. Threshold 128 is at least the size of the SMB direct header in the data transfer packet. The MaxSmb2MessageSize field 314b can contain the size of the largest SMB2 protocol message (perhaps in bytes) that the server / receiver 106 accepts from the client / requester 102. This value is predetermined and set by the server / receiver 106. In the embodiment, the value in the MaxSmb2MessageSize field 314b must not be greater than the total size of the memory buffer 222b. In this way, SMB messages never overflow the memory buffer 222b. However, the value of MaxSmb2MessageSize314b may be less than the amount of memory in the memory buffer 222b, as determined by the user, according to one embodiment.
[0058] The SMB direct data transfer packet 338 according to the embodiment disclosed herein is shown in FIG. 3C. The SMB direct data transfer packet 338 can be sent to transfer SMB2 data over the RDMA connection created during the negotiation. Either the client / requester 102 or the server / receiver 106 can send or receive SMB2 data. Therefore, the client / requester 102 and the server / receiver 106 are commonly referred to as the receiver and sender. SMB direct data transfer packet 338 can include, but is not limited to, one or more of the following fields: For example, CreditsRequested308c, CreditsGranted320b, Reserved306c, RemainingDataLength326, DataOffset328, DataLength330, and / or SMB2Data322. According to the embodiment, the SMB direct data transfer packet 338 can also include more or less fields than the fields shown in FIG. 3C, as represented by the ellipse 334.
[0059] In an embodiment, the CreditsRequested field 308c contains a value for the number of outgoing credits that the sender is requesting from the receiver. In an embodiment, the value in the CreditsRequested field 308c is greater than zero (0) to ensure that SMB2 data is sent in subsequent messages. However, the value in the CreditsRequested field 308c can be any number and can be set based on expected future use, ie, the number of additional packets to complete the transfer of SMB2 data. .. The CreditsGranated field 320b contains the number of credits granted by the sender to the receiver. In the embodiment, the value of the Credits Granted field 320b can be zero. This is because peers are not forced to accept the peer's request for credit. However, the peer can provide credit in the Credits Granted field 320b to allow the client / requester 102 to send the next SMB direct message. Reserved field 306c, according to the embodiments disclosed herein, is merely a field reserved for future unknown requests and is not used by the server.
[0060] In embodiments, the RemainingDataLength field 326 can include the number of bytes in a fragmented SMB2 message that the receiver has not yet received. That is, any non-zero value in this field indicates that another SMB direct data transfer packet 338 will be sent to the receiver along with the other data. If the SMB direct data transfer packet 338 conveys a complete SMB2 message, or if it conveys the last of two or more SMB direct data transfer packets 338 that convey fragmented SMB2 data, the value in the RemainingDataLength field 326 is: According to the embodiment, it will be zero (0). RDMA can send messages sequentially. Therefore, reassembling the message is simplified. This is because the series of messages is reassembled in a strict receiving order. That is, the embodiment eliminates the need to use, for example, message identifiers in headers, or other more complex reassembly techniques.
[0061] In one embodiment, the DataOffset field 328 contains, in bytes, the value for the offset from the beginning of the SMB direct data transfer packet 338 to the first 8-byte aligned byte of the encapsulated SMB2 protocol message. In the embodiment, if the value of the DataLength field 330 is zero, then the DataOffset field 328 is also set to zero. If the value of the DataLength field is non-zero, then the DataOffset field 328 is some value, preferably 24 bytes or more. This is, according to one embodiment, the size of the other field in the header. DataLength field 330 contains the size of the encapsulated SMB2 protocol message in SMB2 data field 332 in bytes. In the embodiment, if the SMB direct data transfer packet 338 does not encapsulate the SMB2 protocol message, the value of the DataLength field 330 is set to zero. The SMB2 data field contains any SMB2 protocol message, according to embodiments.
[0062] The SMB2 read / write request 340 according to the embodiments disclosed herein is shown in FIG. 3D. The SMB2 read / write request 340 can be encapsulated in SMB direct data transfer packet 338 to perform a direct RDMA read / write. Direct RDMA read / write only sends unencoded application data. According to an embodiment of the present disclosure, an SMB2 read / write request 340 can be an SMB2 read request if the server 106 attempts to perform an RDMA write to the client 102, with the server 106 coming from the client. Can be an SMB2 read request when attempting to perform an RDMA read of. In either case, the fields in SMB2 read / write request 340 are similar. It should be noted that the server 106 performs the RDMA transfer requested by the client 102. The SMB2 read / write request 340 can include, but is not limited to, one or more of the following fields: For example, Channel342, ChannelInfoOffset346, and Channelinfolength348. In one embodiment, these fields can include steering information to complete the RDMA data transfer. Reserved field 344, according to one embodiment, is a field reserved for future unknown demands and is not used. The SMB2 read / write request 340 can also contain more or fewer fields than the fields shown in Figure 3D, as represented by the ellipse 351. The illustrated fields are presented for illustrative purposes only and are not intended to be limiting.
[0063] In the embodiment, the Channel field 342 contains a value for the channel for which data is to be found. The RDMA connection can include various channels. Channel information, according to embodiments, can include information provided by the peer to the RDMA device to perform the transfer. For example, the channel information can include the length of one or more tokens, offsets, and memory segments, along with other RDMA specific information, as shown in Figure 3E.
[0064] In one embodiment, the ChannelInfoOffset field 346 and the ChannelInfoLength field 348 are pointers to offset, token, and length information in the data packet. These pointers provide a position where information can be located in SMB2 read or write requests. In embodiments, flag 350 includes any information that controls or alters the behavior of RDMA direct data transfer.
[0065] Moving on to FIG. 3E, the RDMA channel descriptor 352 according to the embodiments disclosed herein is shown. The RDMA channel descriptor 352 can be encapsulated within the SMB direct request, typically in channel field 342. The RDMA channel descriptor 352 can include, but is not limited to, one or more of the following fields: For example, offset 354, token 346, and length 358. In one embodiment, these fields are guidance information to complete the RDMA data transfer. The RDMA channel descriptor 352 can also contain more or fewer fields than the fields shown in Figure 3E, as represented by the ellipse 360. The illustrated fields are presented for illustrative purposes only. To complete the direct data transfer, the RDMA is brought to memory, along with the data to be read or written and the length of the data, based on the information in channel field 342. This information helps ensure proper transfer, as direct data transfer sends only application data.
[0066] According to one embodiment, the offset 354 is a value in bytes or bits from which the data begins. Offset 354 can be measured from the start address of the memory block. The starting address can be determined using the information provided in token 356. Length 358 is a value in bits or bytes, which is the length of the data segment. This guidance information guides, for example, direct data transfer.
[0067] The interaction of the various software function modules shown in FIGS. 2A and 2B is negotiated for RDMA connections in operation step 400 shown in FIGS. 4A-4C according to the embodiments disclosed herein. Will be further illustrated. FIG. 4A shows FIG. 400A of the transfer of packets 408 and 424 between the client 102 and the server 106 according to an embodiment of the present disclosure. FIG. 4B shows from the perspective of client 102, while FIG. 4C shows from the perspective of server 106, according to an embodiment of the present disclosure. It should be noted that process 400 describes when the client requests an RDMA connection and server 106 responds. However, according to the embodiment, the reverse is also possible, in which case the server 106 requests and the client responds. Further, the method can be performed, for example, between a plurality of clients and / or between a plurality of servers.
[0068] Moving on to FIG. 4B, an example operation step 400B is shown in which RDMA uses SMB2 to negotiate a connection according to one embodiment. Process 400B starts in START operation 402B, process 400B proceeds and the client pre-posts the reception (404). In an embodiment, the RDMA interface 216a automatically detects and utilizes RNIC108a / 108b to ensure that they are available and properly configured. Next, in one embodiment, the SMB direct NPI is exposed by the SMB direct module 214. The SMB direct client 214a can open a handle to the RDMA adapter to form an RDMA connection to the SMB2 client 208. In one embodiment, the SMB direct client 214a then requests an RDMA connection via the RDMA interface 216a. The SMB direct client 214a can also make allocations in the memory buffer 222a to determine what to request for the first outbound credit and what is allowed for the inbound credit. Allocations in the memory buffer 222a can, in embodiments, be of a predetermined size. According to embodiments, the total amount of memory in memory buffer 222a can also be determined.
[0069] Returning to FIG. 4B, the SMB direct client 214a can then construct the SMB direct negotiate request packet 300 for sending to the RDMA interface 216a. For example, the SMB direct negotiate request packet 300 is as described in FIG. 3A. Therefore, the SMB direct client 214a sets the fields in the SMB direct negotiate request packet 300. That is, SMB Direct Client 214a determines the minimum and highest version of SMB Direct that Client 102 can support. The SMB direct client 214a determines the number of credits requested. In embodiments, the number of credits requested can be based on future SMB messages known to be sent, expected future SMB traffic, past use of SMB, or by some other method. .. The transmission size is determined by internal functions and speed considerations. The maximum receive size can be fixed (peg) to the size of the memory buffer allocation. Finally, the maximum SMB message size is determined and set (generally, the maximum SMB message size is large enough for peers to send large SMB2 packets, but enough to use all memory. Not the size). This collected information is input to the SMB Direct Negotiate Request Packet 300 according to the embodiments of the present disclosure.
[0070] The SMB direct client 214a then sends an SMB direct negotiate request packet 300 (408B). In one embodiment, the SMB direct client 214a requests the RDMA interface 216a to send the SMB direct negotiate request packet 300 by sending operation. The RDMA interface 216a communicates with the RDMA interface 216b and sends the SMB direct negotiate request packet 300 to the server 106. Upon sending the SMB direct negotiate request packet 300, the SMB direct client 214a activates the negotiation request expiration timer 310. The negotiation request expiration timer can have a predetermined value and counts down from that value to zero. The SMB direct client 214a then waits for a response to the SMB direct negotiate request packet 300. If it determines that the negotiation request expiration timer has expired before receiving a response to the SMB direct negotiate request packet 300 (411), process 400B proceeds from NO to END operation 432B and the connection is dropped.
[0071] Moving on to FIG. 4C, in the start operation 402C, in the embodiment, the server 106 anticipates the SMB direct negotiate request packet 300 and preposts the receive 412. According to the embodiment, the SMB2 server 212b can listen to the activity and automatically discover that RDMA is available and bound to the port of RNIC108b. The SMB2 server 212b opens the listener endpoint and determines that RDMA is available in RNIC108b. The SMB2 server 212b then initiates communication with the SMB direct server 214b. The SMB direct server 214b can open an RDMA connection and receive the RDMA adapter handle. The SMB direct server 214b then requests an RDMA connection via the RDMA interface 216b. The SMB direct server 214b can then accept the RDMA connection, which informs the client 102 that the connection request was successful. This success instruction from RDMA interface 216a causes the client to send a negotiate request.
[0072] The SMB direct server 214b can also make allocations in the memory buffer 222b to determine what to request for the first outgoing credit and what is allowed for the incoming credit. it can. Allocations in the memory buffer 222b can, in embodiments, be of a predetermined size. According to embodiments, the total amount of memory in memory buffer 222b can also be determined.
Returning to Figure 4C, the SMB Direct Server 214b can then activate the negotiation request expiration timer 414. In the embodiment, the negotiation request expiration timer on the server 106 can have a predetermined value and counts down from that value to zero. The SMB direct server 214b then waits to receive the SMB direct negotiate request packet 300. While waiting, the SMB direct server 214b monitors the RDMA connection and determines if the negotiation request expiration timer has expired before receiving the SMB direct negotiate request packet 300 (416). When the negotiation request timer expires, process 400C proceeds from NO to END operation 432C and the connection is dropped.
[0074] However, if the SMB direct negotiate request packet 300 reaches RNIC108b and RDMA interface 216b before the negotiation request expiration timer expires, process 400C proceeds to YES and at 408C the RDMA interface 216b makes an SMB direct negotiate request. Receive packet 300, verify its validity, and put this packet into memory buffer 222b. The SMB direct server 214b is then informed of this data and determines whether the SMB direct negotiate request packet 300 is valid (418).
[0075] In the embodiment, in order to determine the validity of the SMB direct negotiate request packet 300, the SMB direct server 214b reads data from the SMB direct negotiate request packet 300 and determines the following. For example, whether the value of MaxVersion field 304 is less than or less than the value of MinVersion field 302, or whether values in that range are supported, whether the value of CreditsRequested field 308 is zero, and whether the value of MaxReceiveSize field 312 is predetermined. Determines if it is less than a threshold (eg 128 bytes) or if the value of the MaxSmb2MessageSize field 314 is less than a predetermined threshold. If any of the above is true, SMB direct negotiate request packet 300 is not valid, process 400C proceeds from NO to END operation 432C, and SMB direct server 214b disconnects. If all of the checks are not true, process 400C proceeds to YES to process the SMB direct response 420 and the SMB direct negotiate request packet 300 is sent to the SMB direct client 214a by the SMB direct server 214b.
[0076] When processing the SMB Direct Negotiate Request Packet 300, in an embodiment, the SMB Direct Server 214b sets the settingProtocolVersion to equal the highest protocol version shared by the client 102 and the server 106. The SMB Direct Server 214b also determines the number of credits to grant. The number of credits granted depends, in the embodiment, on the space available in the memory buffer 222b or other factors. After making these decisions, SMB Direct Server 214b generates SMB Direct Negotiate Response Packet 336 (422). In one embodiment, the SMB direct server 214b sets the fields in the SMB direct negotiate response packet 336. For example, the PreferredVersion field is set to the value in settingProtocolVersion. CreditsGranted320a is set to the number of credits determined by the SMB direct server 214b, according to one embodiment. Other fields are filled as well as the SMB Direct Negotiate Request Packet 300. The SMB direct negotiate response packet 336 is then sent to the client (424C). In an embodiment, the SMB direct server 214b sends to the RDMA interface 216b to send the SMB direct negotiate response packet 336.
[0077] Returning to FIG. 4B, the SMB direct client 214a determines, according to the embodiment, whether a response was received before the expiration of the negotiate expiration timer (411). If a response is received before the timer expires, process 400B proceeds to YES to determine the validity of SMB Direct Negotiate Response Packet 336. To determine the validity of the SMB Direct Negotiate Response Packet 336, the SMB Direct Client 214a determines that: For example, according to embodiments of the present disclosure, whether status field 322 is not STATUS_SUCCESS, PreferredVersion field 318 is the range specified by MinVersion field 302 and MaxVersion field 304 of the client's SMB direct negotiate request packet 300. Whether or not it contains a value in, whether or not the CreditsRequested field 308b is zero, whether or not the CreditsGranted field 320a is zero, and whether or not the PreferredSendSize field 310b is the value specified by the MaxReceiveSize field 312 of the client's SMB direct negotiate request packet 330. Determines if it is greater than or if the MaxReceiveSize field 312b is less than a given threshold (eg, 128 bytes). If any of the above is true, the SMB direct negotiate response packet 336 is not valid, process 400B proceeds from NO to END operation 432B, and SMB direct client 214a disconnects. If all of these checks are not true, process 400B proceeds to YES and processes SMB direct response 430. Here, the SMB direct negotiate response packet 336 is processed by the SMB direct client 214a according to the embodiments disclosed herein.
[0078] When processing the SMB Direct Negotiate Response Packet 336, the SMB Direct Client 21a completes the following: For example, in the embodiment, the PeerTargetSendCreditsCount setting for the connection in setting 226 is set equal to the CreditsRequested field 308b. Set the SendCreditsCount setting for the connection in the SMB direct connection properties to equal the CreditsGranted field 320a. Set the connection's MaxSendSize setting in setting 226 to the MaxReceiveSize field 312b. Set the Connection ReceiveSize setting in setting 226 to the PreferredSendSize field 310b. Set the connection's MaxOutboundFragmentedMessageSize setting in setting 226 to equal the MaxReceiveSize field 312b. Then, the idle connection timer of the connection is set to expire after a predetermined time amount (for example, several hours, several minutes, etc.), and this timer is activated. Once the above steps have been performed, the RDMA connection negotiation is complete and the client 102 and server 106 can begin exchanging SMB direct data transfer packets. Then, process 400B ends in END operation 432B.
[0079] The interaction of the various software functional modules shown in FIGS. 2A and 2B is SMB2 via an RDMA connection made in accordance with the embodiments disclosed herein in operation step 500 shown in FIGS. 5A-5D. The case of exchanging data will be further illustrated. According to embodiments of the present disclosure, in request and response communications, SMB2 data is created between client 102 and server 106 by encapsulating SMB2 data as the data payload of SMB direct data transfer packet 338. Sent over the SMB direct connection. Request and response communications can be used, for example, for control channel communications, file metadata exchange, or other processes. FIG. 5A shows a diagram of packets 516/518, 530, 532, and 546 between a client (s) 102 and a server (s) 106 according to an embodiment of the present disclosure. 5B and 5C are shown from the perspective of client 102, while FIG. 5D is shown from the perspective of server 106. It should be noted that process 500 describes when the client sends SMB3 data to server 106 independently or in response to a request from the server. However, the reverse is also true, in which case server 106 responds to the request and sends the data back to client 102. That is, the process described herein also applies, in embodiments, when the server sends a request to request credit. Further, this method can be performed, for example, between a plurality of clients or between a plurality of servers.
[0080] Moving to FIG. 5B, an operation step example 500B is shown in which data is exchanged by RDMA using SMB2 according to an embodiment disclosed herein. Start operation 502 is started, process 500B proceeds, and client 102 makes RDMA connection 504. Making an RDMA connection can be described with respect to FIGS. 4A-4C according to embodiments of the present disclosure. In one embodiment, the SMB direct client 214a sets various connection properties. This includes setting the MaxSendSIze setting to the MaxReceiveSize field 312b, the connection ReceiveSize setting to the PreferredSendSize field 310b, and the connection's MaxOutboundFragmentedMessageSize setting in setting 226 equal to the MaxReceiveSize field 312b. In this way, the client 102 determines the size of the packet that can be sent to the server 106 according to the embodiment.
[0081] Returning to FIG. 5B, the SMB direct client 214a can determine the number of SMB direct data transfer packets required to transfer the SMB2 message to the server 106 according to the embodiment. To make this decision, the SMB Direct Client 214a can first pull the MaxSendSize (indicated by R) set during the negotiation. The SMB direct client 214a can then determine the size of the SMB2 protocol message (s) sent. In the embodiment, this value is set as "S". The SMB direct client 214a is also consumed by padding (eg, DataOffset328, etc.) to ensure that the header of the SMB direct data transfer packet 338 and the payload of the SMB direct data transfer packet 338 start at the 8-byte alignment boundary. Determine the number of bytes (indicated by P in the embodiment). In an embodiment, the SMB direct client 214a then determines if S is less than or equal to (RP), and if so, according to the embodiment, by dividing S by (RP). , The number of packets can be determined.
[0082] Process 500B then proceeds to determine if fragmentation is used by the SMB direct client 214a (510). In one embodiment, fragmentation is used when an SMB2 protocol message "does not fit" in one SMB direct data transfer packet 338. In other words, is S greater than (RP)? If fragmentation is not used, process 500B proceeds to NO and sends one SMB direct data transfer packet 518B. On the other hand, if fragmentation is used, process 500B proceeds to YES and initializes transmit buffer byte 512.
[0083] Returning to step 518, according to the embodiment, the SMB2 protocol message can also be sent unfragmented. A portion of the memory buffer 222a used to send the packet is initialized. In an embodiment, the first P bytes are to set the header information for the SMB direct data transfer packet 338, and to add zero padding bytes to ensure that the SMB2 payload is aligned to 8 bytes. It is initialized. That is, according to the embodiment, in the buffer, the DataOffset field 328 is set to P, the DataLength field is set to S, and the RemainingDataLength field 326 is set to zero (0). Information about the credit, or its request, is stored in the CreditsRequested field 308c or Credits Granted field 320b. That is, the client 102 can request or grant more credit in the SMB direct data transfer packet 338. In the embodiment, the sender of the SMB direct data transfer packet must set the CreditsRequested field to at least 1. The next S byte is initialized in memory buffer 222a and SMB2 protocol messages are stored in SMB2 data field 332. After the SMB direct data transfer packet 338 is assembled, the SMB direct client 214a sends the packet to the RDMA interface 216a for sending to the server 106 over the RDMA connection (518B). Process 500B then goes through page connector B422 to END operation 548C, ending process 500B.
[0084] Returning to step 512, according to the embodiment, if S is greater than (RP), the SMB2 protocol message is split into two or more parts. These parts are then sequentially sent in a series of RDMA transmission operations. Each transmission operation conveys a portion (called a fragment) of an SMB2 protocol message. Set the size of the first payload to X to split the SMB2 protocol message into pieces. This is less than or equal to (RP). X represents, according to one embodiment, the number of bytes of the SMB2 protocol message sent by at least the first RDMA transmission operation. Similar to step 518B, some of the memory buffer 222a used to send the packet is initialized. In the embodiment, the first P bytes are initialized to set the header information and padding bytes for the SMB direct data transfer packet 338. Then the DataOffset field 328 is set to P, the DataLength field 330 is set to X, and the RemainingDataLength field 326 is set to (SX) (514). In an embodiment, the RemainingDataLength field 326 indicates the number of bytes remaining untransmitted in the SMB2 protocol message. In an embodiment, (SX) becomes zero (0) when the last fragment of the SMB2 protocol message is sent. In the last packet, the RemainingDataLength field 426 is set to zero to indicate that the SMB direct data transfer packet 338 conveys the last fragment of the SMB2 protocol message. Information about the credit, or its request, is stored in the CreditsRequested field 308c or Credits Granted field 320b. That is, in the embodiment, the client 102 requests or grants more credits in the SMB direct data transfer packet 338. Can be The next X bytes of memory buffer 222a are then initialized. Then, according to the embodiments disclosed herein, fragments of SMB2 protocol messages can be stored in SMB2 data field 332.
[0085] After assembling the SMB direct data transfer packet 338 with the first untransmitted X bytes of the SMB2 protocol message, the SMB direct data transfer packet 338 is sent to the server (516B). In an embodiment, the SMB direct client 214a sends an SMB direct data transfer packet 338 to the RDMA interface 216a for sending to the server 106 over an RDMA connection (516B). Process 500B then proceeds to arbitrary step 526 via page connector A520, where the SMB direct client 214a determines if another SMB direct data transfer packet 338 containing the other fragment must be sent. That is, in the embodiment, the SMB direct client 214a determines whether or not (SX) is 0 after sending the last packet. If (SX) is less than or equal to 0, it means that the last fragment was sent in the last SMB direct data transfer packet 338, process 500B proceeds from NO to END operation 548C, and process 500B terminates. However, if (SX) is greater than 0, process 500B proceeds to YES to determine if there are sufficient outgoing credits (528). In embodiments, step 526 may be optional (as shown). This is because the SMB direct data transfer packet 338 that transfers the fragments is pre-staged in memory buffer 222a. This is because they can be sent in sequence. If buffer 222a is empty, process 500B exits. In this case, it is not necessary to determine whether the next packet should be sent. According to the embodiment, when the SMB2 protocol message is fragmented, the RDMA transmission operation that conveys the fragment is transmitted sequentially and monotonously by another RDMA transmission operation that is not related to the fragmented SMB2 protocol message. It should be noted that it does not have to be interrupted. According to one embodiment, the RDMA transport sequence on the receiving side preserves the sequence processing of the fragments so that the receiving peer can reproduce the original message.
[0086] Returning to step 528, the SMB direct client 214a determines if there are sufficient sending credits to send the next fragment (528). In essence, the SMB direct client 214a determines if one outgoing credit remains. If there are outgoing credits, process 500B goes from YES to step 512 via page connector C524. In an embodiment, RDMA transport has a receive buffer preposted by the receiver before the sender can send the packet. This rule requires coordination between the sender and the receiver to ensure that the sender does not attempt to send the packet before the receiver preposts the reception. SMB Direct uses the transmission Credits system to achieve the desired adjustments, according to embodiments of the present disclosure.
[0087] In one embodiment, the outgoing credits granted by the server 106 or the client 102 represent one receipt preposted by the server 106 or the client. Peers are eligible to perform one RDMA transmit operation with one transmit credit. Client 102 also sends by setting the CreditsRequested field 308c or between another outgoing credit request (according to one embodiment, a set of SMB direct data forwarding packets 338 that forward fragmented SMB2 data). In the form of (not), it provides server 106 with information about the number of credits sent that the client needs to efficiently support its workload. Outgoing credits are awarded in Credits Granted field 320b of SMB direct data transfer packet 338. That is, the SMB direct client 214a can request more outbound credits by setting the value in the CreditsRequest field 308c to a higher number (perhaps covering all future fragmented packets). The SMB direct client 214a may then wait to receive the outgoing credit by waiting for the SMB direct data transfer packet 338 sent from the server 106 (530C). According to one embodiment, this packet 338 has no data, but contains outgoing credits in the Credits Granted field 320b of the SMB direct data transfer packet 338. The SMB direct client 214a can determine in query 542C whether or not credit has been received by setting the SendCreditsGrantedTimer in timer 224a. If the Send Credits Granted Timer expires before the credit is granted (547), process 500B will take page connector C524 from YES. Go to step 523 through. According to one embodiment, if the timer is not set and / or the timer has not expired (547), process 500B proceeds to NO and does not perform a transmit operation, i.e. does not terminate and instead Continue to wait to receive outgoing credits (530C).
[0088] Moving on to the start operation 502D of Figure 5D and process 500D, server 106 receives SMB direct data transfer packet 338 (516D). In one embodiment, the RDMA interface 216b receives the SMB direct data transfer packet 338 and sends it to the SMB direct server 214b. First, the SMB direct server 214b reads the header information. The SMB direct server 214b goes to query 536 to determine if the RemainingDataLength field 326 is something other than zero. If the RemainingDataLength field 326 is zero, the SMB direct data transfer packet 338 is the only packet and process 500D proceeds to NO to process the SMB packet data 554. Here, in one embodiment, the SMB direct server 214b processes the packet data and sends an SMB2 protocol message to the SMB2 server 212b. If RemainingDataLength field 326 is something other than zero, the process 500D proceeds to YES viewed, assigns the assembly buffer again (538).
[0089] In step 538, the SMB direct server 214b, in one embodiment, allocates a portion of the memory buffer 222b to reassemble the SMB2 protocol message in the buffer 222b. That is, the SMB direct server 214b can allocate enough blocks in the memory buffer 222b to accept the data in the current SMB direct data transfer packet 338 and the upcoming SMB direct data transfer packet 338. It is based on the value of DataLength field 330 and the value of RemainingDataLength field 326. According to embodiments of the present disclosure, the SMB direct server 214b can then read the SMB2 data from the SMB2 data field 332 and copy this data into the allocated buffer 540.
[0090] Next, in one embodiment, the SMB direct server 214b waits for and receives the next SMB direct data transfer packet 338 (546D). In one embodiment, the data in SMB2 data field 332 is also copied to the next part of memory buffer 222b. In an alternative embodiment, the SMB direct server 214b can optionally determine if this next SMB direct data transfer packet 338 is the last packet containing fragmented data 542. To make this decision, SMB Direct Server 214b checks if the RemainingDataLength field 326 is zero. If RemainingDataLength feel 326 is zero, SMB direct server 214b understands that no more packets will be received and process 500D proceeds to YES to process SMB2 protocol message 544 from memory buffer 222b. Process 500D then terminates in the END operation (548D). On the other hand, if the RemainingDataLength field 326 is something other than zero, according to one embodiment, the SMB direct server 214b understands that other packets (s) will be received and the process 500D will Proceed to NO and receive the next SMB direct data transfer packet 338 (546D).
For the interaction of the various software function modules shown in FIGS. 2A and 2B, RDMA direct data transfer shall be performed in operation step 600 shown in FIG. 6 according to one embodiment disclosed herein. Will be further illustrated. FIG. 6 shows a diagram of data transfer between client 102 and server 106 according to an embodiment of the present disclosure.
[0092] As shown in Figure 6, process 600 is started in start operation 602 and the SMB2 client 208 makes an SMB2 read / write request 340 (604). Application 220 can request that data be transferred to or read from server 106. This data transfer can be instructed by the SMB2 client 208 to employ RDMA direct data transfer so that the SMB2 server 106 uniquely executes the actual RDMA request, according to embodiments of the present disclosure. .. To initiate this transfer, the SMB2 client 208 registers a target memory buffer to supply or receive unencoded application data, and then generates an SMB read / write command. This command is sent and guided by RDMA. This command may be included in the SMB2 read / write request 340 and sent to the SMB direct client 214a. The SMB direct client 214a encapsulates this SMB2 read / write request 340 into an SMB direct data transfer packet (606). In an embodiment, the SMB2 read / write request 340 is stored in the SMB2 data field 332 of the SMB direct data transfer packet. SMB direct data transfer packets can then be sent to server 106 via RDMA interface 216 (608). In one embodiment, the server 106 receives the SMB direct data transfer packet and reads the SMB2 read / write request 340 from the SMB2 data field 332. The SMB direct server 214b can then read data from the SMB2 read / write request 340. This data includes channel 342, ChannelInfoOffset346, and ChannelInfoLength348. This information is SMB Da The elect server 214b can be directed to the guidance information 610 in the RDMA channel descriptor 352. From this guidance information, the SMB direct server 214b can initiate a direct data transfer by sending unencoded application data to or withdrawing from this memory location in the client 102's buffer. Process 600 then terminates in END action 614.
[0093] FIGS. 4A-4C, 5A-5D, and 6 respectively re-negotiate communication using the file access protocol by RDMA according to the embodiments disclosed herein. An example of operating characteristics when exchanging data using the file access protocol by RDMA is shown. In embodiments, the illustrated operation steps can be combined and / or rearranged with other steps. Further, for example, fewer steps or more steps can be used.
[0094] Overall, credit is an advantage of this embodiment. The outgoing credit represents a buffer preposted to receive incoming data at the peer. Therefore, the sending credit represents a limited receiver resource (memory, memory area, etc.) passed to the peer so that it can be used by the peer to send the data. Due to the nature of RDMA, once posted reception is irrevocable in embodiments. For resources related to inbound to be released, this resource is used to serve the incoming send. This can cause problems in embodiments if the receiving peer starts running low on resource and wants to reclaim some of the resources already dedicated to outstanding receive only. There is sex. Since reception cannot be canceled according to the embodiments disclosed herein, it is necessary to rely on peer cooperation to regain these resources.
[0095] The solution to this problem in SMB Direct is, according to one embodiment, called sending credit abolition. The SMB direct data transfer packet sent to the peer has a Credits Granted field that specifies the amount of additional outgoing credit granted to the peer. In an alternative embodiment, setting the CreditRevocation flag (not shown) in the SMB direct data transfer packet 338 changes the meaning of the Credits Granted field 320b to the number of outgoing credits that the peer can hold. For example, if the CreditRevocation flag is set and the Credits Granted field 320b has a value of 10, 10 is the number of credits the recipient can hold. If the receiver currently holds more than 10 send credits, the receiver will send a series of RDMA transmissions in order to run out of the canceled send credits so that the receiver will have 10 or less send credits. Perform the action. The transmit operation performed to run out of canceled transmit credits can include sending an empty SMB direct data transfer packet 338 (packet with no data payload). Upon receiving an incoming empty SMB direct data transfer packet 338, according to one embodiment, the peer who canceled the outgoing credit releases these resources and reuses them as appropriate. Can be done.
[0096] Outgoing credits allow two peers to synchronize their sending and receiving operations, but in embodiments, these peers seek to avoid deadlocks in sending credits. Imagine a scenario where peer Y has one sending credit from peer X and peer X has one sending credit from peer Y. In this scenario, each peer is entitled to make one transmission to that peer at any given time. Both X and Y performed send operations simultaneously using one of their sending credits in the process, but SMB direct data transfer packet 338 sent by both peers does not grant the peer any additional sending credits. Imagine. The resulting condition is called a sending credit deadlock. Both X and Y have used up their last sending credit. Each peer requires additional outgoing credits in order to be able to send additional packets to the peers. However, the outgoing credit is granted by the SMB direct data transfer packet 338, and none of the peers can perform any further sending operations. As a result, neither peer can send any additional packets and there is no mechanism by which they can obtain other outgoing credits. Therefore, a deadlock has occurred.
[0097] The solution to this problem is, according to the embodiments of the present disclosure, merely a rule. When the SMB direct peer uses its last outgoing credit, according to one embodiment, the SMB direct data transfer packet 338 being sent grants the peer at least one additional outgoing credit. If both peers follow this rule, no deadlock will occur. This is because each peer can always respond to the other with an additional sending credit.
Finally, FIG. 7 shows an example 700 of a computing system that can implement the embodiments disclosed herein. A computer system 700, such as a client / application server 102 or a file server 106, has at least one processor 702 to exchange message data as shown herein and is described herein. It is illustrated according to the embodiments disclosed in the document. System 700 has memory 704. Memory 704 includes, for example, system memory, volatile memory, and non-volatile memory. In its most basic configuration, the computational system 700 is shown in FIG. 7 by the dashed line 706. In addition, the system 700 can also include additional storage (removable and / or non-removable). Additional storage includes, but is not limited to, magnetic or optical discs or tapes. Such additional storage is illustrated in FIG. 7 by removable storage 708 and non-removable storage 710.
[0099] As used herein, the term computer-readable medium can include computer storage media. Computer storage media can include volatile and non-volatile, removable and non-removable media, any method for storing information such as computer-readable instructions, data structures, program modules, or other data. Or it is realized by technology. System memory 704, removable storage 708, and non-removable storage 710 are all examples of computer storage media (ie, memory storage). Computer storage media are not limited to RAM, ROM, electronically erasable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other. It may include optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store information and can be accessed by the computing system 700. it can. Any such computer storage medium may be part of system 700. The illustration in FIG. 7 is not intended to limit the scope of this disclosure at all.
[00100] Also, as used herein, the term computer readable medium may also include communication media. Communication media can be embodied by computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transport mechanisms, including any information distribution medium. The term "modulated data signal" can describe a signal whose one or more characteristics have been set or changed in such a way as to encode information within this signal. As an example, and without limitation, communication media may include wired media such as wired networks or direct wired connections and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media. it can.
[00101] The system 700 may also include a communication connection (s) 716 that allows devices to communicate with each other. In addition, the system 700 can also have input devices (s) such as keyboards, mice, pens, voice input devices, touch input devices and the like. Output devices (s) 712 such as displays, speakers, printers, etc. can also be included. All of these devices are well known in the art and need not be described endlessly here. The device described above is an example, and other devices can be used.
[00102] The embodiments of the present disclosure have been described above with reference to the above figures, but these embodiments are easily recalled by those skilled in the art and are defined in the appended claims. It should be acknowledged that multiple changes contained within the scope and gist of the disclosure may be made. In fact, although embodiments have been described for the purposes of the present disclosure, various changes and changes that fall within the scope of the present disclosure may be made.
[00103] Similarly, the present disclosure uses specific language for structural features, methodological behaviors, and computer-readable media containing such behaviors, but the subject matter set forth in the appended claims is described herein. It goes without saying that the specific structure, operation, features, or medium described in the above section is not necessarily limited. Specific names or naming conventions, such as names for APIs, routines, etc., have been used to describe aspects of the embodiment, but a number of changes within the gist and scope of this disclosure. Can also be done for such names and / or naming conventions. The specific structures, features, behaviors, names, naming conventions, and / or media described above have been disclosed as examples of forms that realize the scope of claims. Aspects of the embodiment anticipate a plurality of client / application servers, a plurality of file servers, a plurality of networks, a plurality of connected peers, and the like. Also, in other embodiments, one client computer with one server and one network were used. Those skilled in the art will also appreciate other embodiments or improvements that fall within the scope and gist of this disclosure. Therefore, these specific structures, actions, or media shall be disclosed as examples of embodiments that realize the present disclosure. This disclosure shall be defined by the appended claims.
17 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17
Every citation, both ways
| Document | Relation | Office |
|---|---|---|
| US20100161855A1 | Cites | United States of America |
| JP2006333434A | Cites | Japan |
19 members in 6 offices
Priority claims9
| Document | Office | Kind | Date |
|---|---|---|---|
| 13172757 | United States of America | – | |
| 201113172757 | United States of America | A | |
| 201113172757 | United States of America | A | |
| 2012041049 | United States of America | W | |
| 2012041049 | United States of America | W | |
| 13172757 | – | – | – |
| US201113172757 | – | – | – |
| US2012041049 | – | – | – |
| WO2012US41049 | – | – | – |
Members19
| Document | Office | Kind | |
|---|---|---|---|
| US2013007180A1 | United States of America | A1 | |
| WO2013002980A2 | World Intellectual Property Organization (WIPO) | A2 | |
| WO2013002980A3 | World Intellectual Property Organization (WIPO) | A3 | |
| WO2013002980A3 | World Intellectual Property Organization (WIPO) | A3 | |
| CN103636181A | China | A | |
| KR20140034872A | Republic of Korea | A | |
| KR20140034872A | Republic of Korea | A | |
| EP2737685A2 | European Patent Office (EPO) | A2 | |
| JP2014528107A | Japan | A | |
| EP2737685A4 | European Patent Office (EPO) | A4 | |
| US9331955B2 | United States of America | B2 | |
| US2016226951A1 | United States of America | A1 | |
| CN103636181B | China | B | |
| JP6047158B2This record | Japan | B2 | |
| CN106411767A | China | A | |
| US10284626B2 | United States of America | B2 | |
| CN106411767B | China | B | |
| KR102047142B1 | Republic of Korea | B1 | |
| KR102047142B1 | Republic of Korea | B1 |
17 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Notification of acceptance of power of attorneyJAPANESE INTERMEDIATE CODE: R3D02RD02 | RD02 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD | |
| Request for written amendment filedJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Report on retrievalJAPANESE INTERMEDIATE CODE: A971007A977 | A977 | |
| Notification of reasons for refusalJAPANESE INTERMEDIATE CODE: A131A131 | A131 | |
| Request for written amendment filedJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Written request for application examinationJAPANESE INTERMEDIATE CODE: A621A621 | A621 | |
| Notification of change in applicantJAPANESE INTERMEDIATE CODE: A711A711 | A711 |
Numbers
- Publication
- 6047158
- Publication, DOCDB
- 6047158
- Publication, EPODOC
- JP6047158B
- Application
- 2014518588
- Application, DOCDB
- 2014518588
- Application, EPODOC
- JP20140518588
Titles2
- Japanese
- リモート・ダイレクト・メモリー・アクセスによる任意サイズの移送動作
- English
- Arbitrary size transport operation with remote direct memory access
Classification
- CPC, 11
- H04L47/39
- H04L67/1097
- H04L67/06
- H04L47/722
- H04L2012/5603
- H04L49/9057
- H04L1/1835
- G06F16/183
- G06F3/061
- G06F3/0656
- G06F3/067
- IPC, 6
- G06F13 00
- H04L47 36
- H04L47 43
- H04L47 722
- H04L12 805
- G06F13 28
