Systems and methods for i/o command scheduling based on multiple resource parameters

FIELD

0001The present disclosure is related to storage devices, and more particularly to systems and methods for I/O command scheduling based on multiple resource parameters.

BACKGROUND

0002A host computing device may provide commands (e.g., read and write commands) for processing by a storage device. The host computing device may have certain quality of service (QoS) requirements for the processing of the commands. In certain situations, however, there may not be enough hardware resources to process the commands in compliance with the QoS requirements, for example, due to traffic collisions and bandwidth collisions, thus resulting in jitter.

0003Accordingly, there is a need for a system and method for scheduling commands for processing by a storage device that considers availability of the hardware resources of the storage device for addressing QoS requirements.

SUMMARY

0004According to an embodiment, a method for scheduling input/output (I/O) commands is described. The method may include: receiving, by an I/O scheduler, an I/O command from an application; generating, by the I/O controller, an I/O resource requirement based on the I/O command; determining, by a traffic controller, that an amount of available resources satisfies a criteria based on the I/O resource requirement; and sending, by the traffic controller, the I/O command to a queue in response to the criteria being satisfied.

0005The resources associated with storage device may include at least one of a number of available hardware tokens, an amount of available buffer, or an amount of available direct memory access (DMA) descriptor, and wherein information on the amount of available resources is obtained via a device hint and adjusted by an oversubscription factor.

0006The determination of the criteria being satisfied comprises: determining, by the traffic controller, that an amount of scheduled I/O commands is less than a reserved amount of resources; determining, by the traffic controller, that the amount of scheduled I/O commands is less than a predetermined traffic threshold value; and determining, by the traffic controller, that an amount of required resources based on the I/O resource requirement is less than an amount of available resources.

0007The method may further include: allocating required resources to the I/O command based on the amount of required resources determined from the I/O resource requirement; and deallocating the resources from the I/O command in response to determining that the I/O command is complete.

0008The method may further include sending, by the traffic controller, the I/O command to the I/O scheduler and incrementing a traffic collision count of an I/O utilization counter, in response to the amount of scheduled I/O command being greater than the predetermined traffic threshold value.

0009The method may further include sending, by the traffic controller, the I/O command to the I/O schedule and incrementing a resource collision count of an I/O utilization counter, in response to the amount of required resources based on the I/O resource requirement being less than the amount of available resources.

0010The method may further include increasing the predetermined traffic threshold value in response to a traffic collision.

0011The method may further include resetting the traffic threshold value to the predetermined traffic threshold value in response to a pause in receiving an I/O command from the I/O scheduler for a predetermined duration of time.

0012The I/O resource requirement may include a buffer and a DMA descriptor.

0013According to another embodiment, a system for scheduling I/O commands for processing by a storage device is described. The system may include: a processor; and <ul id="ul0001" list-style="none" compact="compact"><li>a memory configured to store instructions that, when executed, cause the processor to: receive, by an I/O scheduler, an I/O command from an application;</li><li>generate, by the I/O controller, an I/O resource requirement based on the I/O command; determine, by a traffic controller, that an amount of available resources satisfies a criteria based on the I/O resource requirement; and send, by the traffic controller, the I/O command to a queue in response to the criteria being satisfied.</li></ul>

0014The resources associated with storage device may include at least one of a number of available hardware tokens, an amount of available buffer, or an amount of available direct memory access (DMA) descriptor, and wherein information on the amount of available resources is obtained via a device hint and adjusted by an oversubscription factor.

0015The determination of the criteria being satisfied for the system may include: determining, by the traffic controller, that an amount of scheduled I/O commands is less than a reserved amount of resources; determining, by the traffic controller, that the amount of scheduled I/O commands is less than a predetermined traffic threshold value; and determining, by the traffic controller, that an amount of required resources based on the I/O resource requirement is less than an amount of available resources.

0016The instructions may further cause the processor to: allocate required resources to the I/O command based on the amount of required resources determined from the I/O resource requirement; and deallocate the resources from the I/O command in response to determining that the I/O command is complete.

0017The instructions may further cause the processor to send, by the traffic controller, the I/O command to the I/O scheduler and increment a traffic collision count of an I/O utilization counter, in response to the amount of scheduled I/O command being greater than the predetermined traffic threshold value.

0018The instructions may further cause the processor to send, by the traffic controller, the I/O command to the I/O schedule and increment a resource collision count of an I/O utilization counter, in response to the amount of required resources based on the I/O resource requirement being less than the amount of available resources.

0019The instructions may further cause the processor to increase the predetermined traffic threshold value in response to a traffic collision.

0020The instructions may further cause the processor to reset the traffic threshold value to the predetermined traffic threshold value in response to a pause in receiving an I/O command from the I/O scheduler for a predetermined duration of time.

0021The I/O resource requirement for the system may include a buffer and a DMA descriptor.

0022According to another embodiment, a system is described. The system may include: a storage device include storage device resources for processing I/O commands to access a non-volatile memory of the storage device; a host device associated with the storage device and configured to send the I/O commands to the storage device, wherein the host device comprises an I/O controller configured to schedule the I/O commands, wherein the I/O controller may be configured to: generate an I/O resource requirement based on the I/O command received from an I/O scheduler; determine, by a traffic controller of the I/O controller, that an amount of available resources satisfies a criteria based on the I/O resource requirement; and send, by the traffic controller, the I/O command to a queue in response to the criteria being satisfied.

0023The I/O controller of the system may be further configured to: determine, by the traffic controller, that an amount of scheduled I/O commands is less than a reserved amount of resources; determine, by the traffic controller, that the amount of scheduled I/O commands is less than a predetermined traffic threshold value; determine, by the traffic controller, that an amount of required resources based on the I/O resource requirement is less than an amount of available resources; allocate required resources to the I/O command based on the amount of required resources determined from the I/O resource requirement; process the I/O command by the storage device; determine that the processing of the I/O command is complete; and deallocate the resources from the I/O command in response to the determination of the I/O command being complete.

BRIEF DESCRIPTION OF THE DRAWINGS

0024<ul id="ul0002" list-style="none" compact="compact"><li><figref idref="f0001">FIG. 1</figref> is a conceptual block diagram of a system for scheduling input/output (I/O) commands, according to some embodiments.</li><li><figref idref="f0002">FIG. 2</figref> is a block diagram of a system for resource-based scheduling of I/O command, according to some embodiments.</li><li><figref idref="f0003">FIG. 3</figref> is a block diagram of various abstraction layers of the system of <figref idref="f0002">FIG. 2</figref>, according to some embodiments.</li><li><figref idref="f0004 f0005">FIGS. 4A-4B</figref> are block diagrams of a system for scheduling I/O commands, according to some embodiments.</li><li><figref idref="f0006">FIG. 5</figref> is a block diagram of a system for scheduling I/O commands, according to some embodiments.</li><li><figref idref="f0007">FIG. 6</figref> is a block diagram of an example converter for converting an I/O command, according to some embodiments.</li><li><figref idref="f0007">FIG. 7</figref> is a flow chart illustrating a traffic flow process, according to some embodiments.</li><li><figref idref="f0008">FIG. 8</figref> is a flow chart illustrating a process for adjusting a traffic threshold value, according to some embodiments.</li><li><figref idref="f0009">FIG. 9</figref> is a flow chart illustrating a process for scheduling of I/O commands based on multiple resource parameters, according to some embodiments.</li></ul>

0025Embodiments of the present disclosure and their advantages are best understood by referring to the detailed description that follows. Unless otherwise noted, like reference numerals denote like elements throughout the attached drawings and the written description, and thus, descriptions thereof will not be repeated. In the drawings, the relative sizes of elements, layers, and regions may be exaggerated for clarity.

DETAILED DESCRIPTION

0026Hereinafter, example embodiments will be described in more detail with reference to the accompanying drawings. The present invention, however, may be embodied in various different forms, and should not be construed as being limited to only the illustrated embodiments herein. Rather, these embodiments are provided as examples so that this disclosure will be thorough and complete, and will fully convey the aspects and features of the present invention to those skilled in the art. Accordingly, processes, elements, and techniques that are not necessary to those having ordinary skill in the art for a complete understanding of the aspects and features of the present invention may not be described.

0027Generally speaking, a storage device may have a limited number of hardware resources for processing input/output (I/O) commands from a host computing device. The hardware resources may be, for example, hardware queues and associated tokens that may be used for scheduling the I/O commands for processing. The size of the hardware queues and associated number of tokens may be implementation specific, and may depend on the storage device's performance and QoS requirements.

0028A device driver interacting with the storage device may allocate software resources to the I/O commands that are to be processed. The software resources may be, for example, software queues and associated tokens. The depth of the software queues and associated tokens may be arbitrary, and may exceed the depth of the hardware queues and its associated tokens. This may sometimes result in the device driver scheduling more I/O commands than the hardware resources available in the storage device. In such a situation, an I/O command that is scheduled by the device driver, but that cannot consume a hardware resource, may experience delays in processing, resulting in high jitter.

0029<figref idref="f0001">FIG. 1</figref> is a conceptual block diagram of a system for scheduling I/O commands according to some embodiments. The system includes a device driver 100 configured to submit I/O commands for processing by a storage device 102. The device driver 100 may include various software queues 104-108 for storing I/O commands submitted by one or more applications of a host processing device. For example, queue 104 (Q1) may be a deep write queue configured to store write I/O commands, while queue 106 (Q2) may be a shallow read queue configured to store read I/O commands. The I/O commands may consume software resources (e.g. software queues 104-108 and tokens 110), to deliver the I/O commands to the storage device 102 for processing.

0030Similar to the device driver 100, the storage device 102 may include one or more hardware queues 112-114 configured to store the I/O commands submitted by the device driver 100. For example, hardware queue 112 may be configured to store I/O commands fetched from Q1 104 and Q2 106. The I/O commands in the hardware queue 112 may consume hardware resources (e.g. hardware tokens) 116. The I/O commands with assigned hardware resources 116 may be scheduled for processing by the storage device.

0031The various software queues 104-108 configured in the device driver may be for addressing QoS provisions for the processing of the I/O commands. For example, by separating write and read I/O commands into separate queues 104, 106, and by employing a scheduling algorithm such as round robin, the read I/O commands in Q2 may generally be serviced in a predictable time window. In some situations, however, although there are I/O commands in Q2 that have been queued up with the expectation of being processed, resource conflicts within the storage device may prevent the servicing of the I/O commands in Q2 as expected.

0032To illustrate this point, an example may be considered where Q1 104 has 1000 write I/O commands queued up, and Q2 106 has 10 read I/O commands queued up. It is assumed for purposes of this example that the 1000 write I/O commands consumes 1000 software tokens 110, and may be submitted to the storage device for processing. According to this example, however, the storage device 102 only has 768 hardware tokens 116 to be assigned. Thus, the storage device 102 fetches up to 768 write I/O commands from Q1 104 for processing, causing all 768 hardware tokens 116 to be consumed. Given that the write I/O commands have consumed all the hardware tokens 116, with more write I/O commands with software tokens 110 assigned that are left to be processed, the read I/O commands in Q2 may be unable to be processed during their expected time window. Thus, the QoS requirements for Q2 may not be able to be satisfied.

0033Embodiments of the present disclosure are directed to a system and method for resource-based scheduling of I/O commands that consider availability of multiple resource parameters of the storage device for submitting the I/O commands for processing. According to one embodiment, hardware resources of a storage device are monitored for matching/synchronizing with the software resources. Once synchronized, the software resources may be assigned to I/O commands as desired. The assigning of the software resources to the I/O commands may be based on, for example, a determined QoS as set forth in a service level agreement (SLA). Although QoS is used as an example of a criteria that may be used to determine allocation of the software resources, a person of skill in the art should recognize that other criteria may also be considered, such as, for example, user preferences, and/or the like.

0034According to one embodiment, the device driver includes a software token manager configured to identify hardware resource information, and update the software resource information accordingly. In one embodiment, the software token manager updates a number of available software resources to match a number of available hardware resources. A certain portion of the available software resources may be reserved for one or more queues of the device driver based on, for example, QoS expectations. The remaining software resources may be part of a shared pool. In one embodiment, the software token manager assigns software tokens/resources from the shared pool, to I/O commands in the software queues, for submitting the I/O commands to the storage device for processing. According to one embodiment, if there are no software resources in the shared pool to be assigned to an I/O command in a queue, the software token manager may access the software tokens reserved for the queue, and assign one of the reserved tokens to the command. The reserved tokens may allow I/O commands to be processed in a timely manner for satisfying QoS requirements.

0035<figref idref="f0002">FIG. 2</figref> is a block diagram of a system for resource-based scheduling of I/O commands according to one embodiment. The system may include a host computing device 200 coupled to a data storage device 202 over a storage interface bus 204. The storage device 202 may be a non-volatile storage device such as, for example, a solid state drive (SSD), an Ethernet SSD (eSSD), Universal Serial Bus (USB) drive, Security Digial (SD) Card, embedded Multi-Media Controller (eMMC), Universal Flash Storage (UFS), and/or the like. The storage interface bus 106 may be, for example, a Peripheral Component Interconnect Express (PCIe) bus, Ethernet, and CXL (Compute Express Link). In one embodiment, the host device 200 transfer and receive data to and from the data storage device 202 over the storage interface bus 206, using a storage interface protocol. The storage interface protocol may be, for example, a non-volatile memory express (NVMe) protocol or any other like protocol that uses queues for storing I/O commands to be processed.

0036In one embodiment, the host 200 includes one or more applications 206 running in an application layer of the host 200. The one or more applications 206 may be software applications that are stored in host memory space for execution by a processor. In one embodiment, the one or more applications 206 may send I/O commands to the storage device 202 for processing. For example, the one or more applications may issue read I/O commands for reading data from the storage device, write I/O commands for writing data into the storage device, and/or other I/O requests.

0037In one embodiment, the host 200 includes a device driver 208 configured to interface with the storage device 202. In one embodiment, the device driver is implemented as software instructions that are stored in the host memory, and which are executed by the processor. The device driver 208 may include one or more queues 212a, 212b (hereinafter referred to as software queues 212). The software queues 212 may include, for example, one or more submission queues and completion queues. The submission queues may be configured to store I/O commands/requests submitted by the various applications 206. The completion queues may be configured to store completion messages for the I/O commands/requests processed by the by the storage device 202.

0038The one or more software queues 212 (e.g. submission queues) may be dedicated to store certain types of I/O commands from the host 200. For example, one queue may be dedicated to store read I/O commands from the applications, while another queue may be dedicated to store write I/O commands. In one embodiment, certain QoS requirements may be imposed on the software queues 212 based on one or more SLAs. For example, a certain number of the I/O commands in the software queues 212 may be expected to be processed in a given time period. Different QoS requirements may be associated with the different software queues 212.

0039The device driver 208 may further include a token manager 210 configured to manage software resources that may influence the scheduling of I/O commands that are to be processed. The software resources may be, for example, software tokens and/or queues 212. In one embodiment, the token manager 210 is configured to determine availability of software tokens based on information from the storage device, and assign available tokens to I/O commands stored in the software queues 212. The token manager 210 may be configured to maintain availability information on a per controller and/or token type basis. The assigned tokens may be from a shared pool when tokens are available in the shared pool, or from a pool that is reserved for a queue, for I/O commands submitted to the queue.

0040In one embodiment, the storage device 202 includes a host interface layer (HIL) 214 for interfacing between the host 200 and a device control subsystem 216. The HIL 214 may include, without limitation, one or more controllers 218 and one or more queues 220a-220c (hereinafter referred to as hardware queues 220). The depth of the hardware queues 220 may depend, for example, on a processing power of the storage device. In one embodiment, different queues may be maintained for different types of processing by the storage device. For example, one hardware queue may store I/O commands for processing by a hardware acceleration engine, and another queue may store I/O commands for processing via firmware.

0041The controllers 218 may be implemented via one or more processors such as, for example, a field programmable gate array (FPGA), embedded FPGA (eFPGA), an application specific integrated circuit (ASIC), co-processor, complex programmable logic device (CPLD), and/or any suitable digital and/or analog circuitry. One of the one or more controllers 218 may be associated with one or more hardware queues 220. The one of the controllers 218 may be configured to manage and assign hardware resources to I/O commands submitted for processing in the software queues 212. The hardware resources may be, for example, hardware tokens and/or queues 220.

0042In one embodiment, the one or more controllers 218 are configured to fetch I/O requests/commands from one or more of the software queues 212, and store the fetched requests into the one or more of hardware queues 220 corresponding to the one or more controllers 218. In some embodiments, the requests may be fetched and submitted by the device driver 208.

0043The I/O commands stored in the hardware queues 220 may be assigned hardware tokens for processing. In one embodiment, different types of hardware tokens may be maintained and assigned depending on the type and number of hardware queues 220. For example, a first type of hardware token may be assigned to a first hardware queue 220a dedicated for processing by a hardware acceleration engine, while a second type of hardware token may be assigned to a second hardware queue 220b dedicated for processing via firmware.

0044The number of available hardware tokens to be assigned to the I/O commands in the different hardware queues 220 may depend on the number of I/O commands already in the queues. In one embodiment, the tokens are assigned by the controller 218 based on hardware queue type. In one embodiment, hardware tokens are automatically assigned upon storing of I/O commands in the one or more hardware queues 220. Commands that have assigned hardware tokens may be scheduled for processing by the device control subsystem 216.

0045In one embodiment, the device control subsystem 216 interacts with the controllers 218 for executing I/O commands requested by the applications 206. The subsystem 216 may include, without limitation, one or more processors 222 and one or more media interface(s) 224. The one or more processors 222 may be configured to execute computer-readable instructions for processing I/O commands to and from the controllers 218, and for managing operations of the storage device 202. The computer-readable instructions executed by the one or more processors 222 may be, for example, firmware code.

0046In one example, the one or more processors 222 may be configured to interact with the controllers 218 for receiving write or read I/O commands to or from NVM media 226. The one or more processors 222 may interact with the NVM media 226 over the media interface 224 for effectuating the write or read actions. The NVM media 226 may include one or more types of non-volatile memory such as, for example, flash memory.

0047In one embodiment, the storage device 202 further includes an internal memory 228 for short-term storage or temporary memory during operation of the storage device 202. The internal memory 228 may include a DRAM (dynamic random access memory), SRAM (static random access memory), and/or DTCM (Data Tightly Coupled Memory). The internal memory 228 may be used to store, for example, the hardware queues 220.

0048<figref idref="f0003">FIG. 3</figref> is a block diagram of various abstraction layers of the system of <figref idref="f0002">FIG. 2</figref> according to one embodiment. The abstraction layers of the host 200 may include an application layer 300 and a device driver layer 302. The application layer 300 may be configured to generate I/O commands based on I/O requests issued by the applications 206. The I/O commands generated by the application layer 300 may confirm to the storage interface protocol of the storage interface bus 204 used for communicating with the storage device 202.

0049In one embodiment, the device driver layer 302 includes the software queues 212 and token manager 210. The I/O commands generated by the application layer 300 are stored in the queues 212 in the device driver 208 based on, for example, the I/O command type. The token manager 210 may determine availability of software tokens based on information from the storage device, and assign available tokens to I/O commands stored in the software queues 212. In the event that all tokens are assigned, service may be rejected. In one embodiment, an I/O command may be allowed to be queued in the software queues 212, but if no available tokens exist, the I/O command may not be submitted to the hardware queues 220 for processing.

0050In one embodiment, the storage device 202 includes a storage layer 304 that may include various sublayers. The various sublayers may include the host interface layer 214 as well as other layers for interfacing with the NVM media 226 such as, for example, a flash translation layer 306 and flash interface layer 308. The NVM media 226 may be included in the storage layer 304 as flash media 310.

0051<figref idref="f0004">FIG. 4A</figref> is a conceptual block diagram of a system for scheduling I/O commands according to another embodiment. The system 400 includes a storage device 402 such as, for example a solid-state drive (SSD) that uses non-volatile memory express (NVMe) protocol and a host device 404 for controlling the operations of the storage device. The storage device may include hardware tokens 406, hardware direct memory access (DMA) descriptor 408, and hardware buffer 410, and the host device may include queues 412 Q1-Qn (e.g., NVMe queues) and block multiple queues and I/O scheduler 414. Accordingly, a software application may send an I/O command (e.g., a read or write command) to the block multiple queue 414 to be processed by the storage device 402, and the block multiple queue 414 pushes the I/O command to the queue 412. The storage device 402 may then fetch the I/O command from the queue 412 for processing the I/O command. If there are multiple I/O commands in the queue 412, then the storage device 402 may fetch the I/O commands, for example, in a round robin manner from each queue 412. Because I/O commands are usually read or write I/O commands, data is transferred between the host device 404 and the storage device 402 (e.g., to/from a NAND flash memory of the storage device 402). In such case, when the I/O command is fetched by the storage device, a hardware token 406 is assigned to each I/O command, and a determination is made as to the amount of hardware resources needed to process each I/O command. For example, hardware resources may include hardware buffers and DMA descriptors. Accordingly, the I/O command may be processed to transfer the data and once the transfer is complete, the storage device informs the queue 412 that the I/O command is finished. Then, the queue 412 informs the block multiple queue 414 that the processing of the I/O command is finished and the block multiple queue 414 informs the software application that the I/O command is complete.

0052In more detail, the I/O commands generated from the software application to the block multiple queue 414 may be in different sizes and different I/O rates. In the conceptual example of <figref idref="f0004">FIG. 4A</figref>, there are different I/O streams and each I/O stream has a different I/O rate. That is, the I/O stream represented by ovals have a 32 KB I/O rate, the rectangles have a 64 KB I/O rate, and the triangles have a 4 KB I/O rate, and each of these I/O streams are assigned to a different queue 412. For example, the 32 KB I/O stream is assigned to Q1, the 64 KB I/O stream is assigned to Qn-1, and the 4 KB I/O stream is assigned to Qn. Each queue 412 Q1 to Qn may be able to queue multiple I/O commands as indicated by each of the ovals, rectangles, or triangles in <figref idref="f0004">FIG. 4A</figref>. As the queues 412 fill up with I/O commands, the storage device 402 fetches from the queue 412 in a round robin manner, but because the I/O rates for Q1 to Qn-1 are faster than the I/O rate for Qn, most of the I/O commands that are scheduled to be fetched by the storage device are taken by the Q1 to Qn-1 queues. For example, the I/O command for the 32 KB I/O stream occupies all of the available resources (e.g., hardware buffer and DMA descriptor), and therefore the I/O commands for the 64 KB I/O stream has to wait until all of the I/O commands for the 32 KB I/O stream are finished so that the hardware resources may be released. Consequently, the slower queues such as Qn has to wait until all of the faster I/O commands are completed before they can be scheduled to be fetched. Accordingly, the I/O commands for the slower I/O streams are delayed, resulting in I/O traffic collisions and unfair scheduling of the I/O commands.

0053Additionally, some scheduling techniques as described above may result in uncontrolled I/O bandwidth. As the data moves between the host device and the storage device, the I/O commands that require larger size data transfers consume the storage device resources such as the hardware buffer and DMA descriptor. Consequently, the queues for the other I/O commands may be delayed and negatively affect I/O responsiveness even for the smaller I/O commands. Thus, techniques to overcome uncontrolled I/O bandwidth and uncontrolled I/O traffic collisions are desired.

0054<figref idref="f0005">FIG. 4B</figref> is another conceptual block diagram of a system for scheduling I/O commands according to some embodiments. In some systems, only a portion of an amount of available resources (e.g., hardware tokens) may be pooled and shared by all of the queues and the remaining amount of the resources are reserved for designated queues so that the reserved resource are not used by the faster rate I/O stream queues 412.

0055For example, 4 tokens for I/O commands may be reserved in each queue 412 Q1 to Qn. Thus, if there are three queues (i.e., Q1, Qn-1, Qn), then a total of 12 (i.e., 4 tokens x 3 queues) tokens are reserved. Accordingly, if there are 768 hardware tokens, then only 12 out of the 768 hardware tokens are reserved and the remaining 756 hardware tokens are available to be shared by all of the queues 412 Q1 to Qn. Accordingly, even if the I/O commands that have the faster I/O stream rates take up all of the shared hardware tokens, the hardware tokens that are reserved by the slower I/O streams will not be taken. While, this technique improves the fairness of scheduling of the I/O commands, the shared portion of the hardware tokens are still uncontrolled and may result in uncontrolled I/O traffic.

0056Thus, according to another embodiment, the I/O command may be scheduled based on availability of resources such as, for example, hardware token, DMA descriptor, and buffer. In some embodiments, the availability of other resources may be considered depending on the type of storage system. Accordingly, the I/O command may be converted from an I/O request to an amount of required resources for the I/O command before making a scheduling decision (e.g., whether to schedule the I/O command or reschedule for a later time). In some embodiments, the scheduler may check the available resources with the amount of required resources to determine whether there are sufficient resources to process the I/O command, and then decide to either schedule the I/O command or not (e.g., reschedule for later).

0057For example, queues 412 Q1 to Qn-1 in <figref idref="f0004">FIG. 4A</figref> are filled with I/O requests, which are beyond the I/O processing capability of the storage device 402. Accordingly, it may cause delays in scheduling the I/O commands for the slower rate I/O streams, such as Qn. Thus, according to some embodiments of the present disclosure as illustrated in <figref idref="f0005">FIG. 4B</figref>, a resource-based I/O controller 416 may determine the amount resources available from a device hint 418 (e.g., HW-HINT), and then scheduling the I/O command based on the available resources. In some embodiments, the availability and utilization of such resources may be tracked by decreasing the available amount of resources when then I/O command is scheduled, and increasing the amount of available resources when the processing of the I/O command is finished. Accordingly, because queues 412 Q1 to Qn-1 in <figref idref="f0005">FIG. 4B</figref> are filled based on available resources, the I/O controller is able prevent or reduce over-scheduling of the faster rate I/O streams on queues 412 Q1 to Qn-1. Accordingly, queuing delays by the I/O commands from the slower rate I/O streams may be reduced or avoided.

0058<figref idref="f0006">FIG. 5</figref> is a block diagram of a system for resource-based scheduling of IO commands according to an embodiment of the present disclosure. The system 500 may include a storage device 504 such as, for example an SSD that uses non-volatile memory express (NVMe) protocol, serial advance technology attachment (SATA), serial attached small computer system interface (SAS), and/or any other suitable protocol and a host device 502 that includes a kernel for controlling the operations of the storage device. In some embodiments, the host device 502 is coupled to the storage device 504 by a storage interface that uses, for example NVMe or other suitable protocol. The storage device may include hardware tokens 510, hardware DMA 524, hardware buffer 520, and a NAND flash 522 for storing data. The host device 502 may include queues 506 (e.g., NVMe queues) and block multiple queues 508. In some embodiments, the queues 506 may be configured to support at most 1024 queue entries (e.g., maximum queue entries supported (MQES)). In other embodiments, the queues 506 may have a smaller or larger capacity, for example, less than or greater than 1024 MQES. Yet in some embodiments, the queues 506 may be configured by a Linux driver based on a smallest value (e.g., minimum value) of maximum submission/completion queue count ("NSQ") or a number of central processing unit (CPU) count of the host device 504 ("#CPU"). Thus, as previously described with reference to <figref idref="f0004 f0005">FIGS. 4A-4B</figref>, the I/O commands that are in the queues 506 may be fetched by the storage device 504, for examples, in a round robin manner for processing by the storage device 504. Once the I/O commands are fetched, each I/O command is assigned a hardware token 510 (i.e., bound to a hardware token) and then sent to a resource allocator 530 where a DMA descriptor and a buffer are is assigned to the I/O command, and the I/O command is processed (e.g., DMA operation happens between host memory and NAND flash). See, for example, in <figref idref="f0006">FIG. 5</figref> where the I/O command is fetched from the queue 506 of the host device 502 and a hardware token 510 is assigned. Then the I/O command with the token already assigned, is sent to a resource allocator 530. In some embodiments, the available DMA descriptor 224 resources are provided to the resource allocator 530 shown by the shaded boxes and the available buffer 520 are provided to the resource allocator 530 shown by the blank boxes. The resource allocator 530 may then assign the DMA descriptors and the buffer to the I/O commands.

0059Turning back to the host device 502, in some embodiments, the host device 502 also includes an I/O controller 526 for controlling I/O commands from the block multiple queue 508, according to various embodiments of the present disclosure. In some embodiments, the I/O controller 526 includes a converter 528 for converting I/O commands that are received as an I/O request format from the block multiple queue 508 to an I/O resource requirement format, and a traffic controller 512 for determining whether to send the I/O command to queues 506 or back to the block multiple queue 508. For example, <figref idref="f0007">FIG. 6</figref> illustrates a block diagram of the example converter 528, where the I/O command is received in the I/O request format, which, in this example is [8K, 88K], which corresponds to a starting address of 8K, and a data size of 88K for this command. In other words, for example, if the command was for a write operation, then the size of the data that will be written to the storage device is 88K and that data will be stored at a location on the storage device at address 8K. The converter 528 converts this information into a resource requirement format, that includes, for example, 11 DMA descriptors, or some other buffer size. Accordingly, the I/O command in the form of an I/O resource requirement is generated from the I/O request based on the I/O command.

0060In some embodiments, the I/O controller 526 further includes a traffic controller 512 configured to analyze the I/O command received in the I/O resource requirement format, a software token manager 514 configured to determine an availability of the hardware token 510, and a DMA descriptor (e.g., "DMA D") and buffer checker 516 configured to determine an availability of the DMA descriptor 524 and buffer 520. In some embodiments, the availability of the resources is determined by a device hint (e.g., "HW-HINT") of the host device 502. In other words, the device hint determines and monitors the status and/or availability of the resources in the storage device and provides this information to the host device (e.g., the I/O controller 526). More in particular, the status and/or availability of the hardware tokens 510 is mirrored by the software token manager 514 so that the I/O controller 526 can make decisions based on resource availability. In addition, the device hint also provides the availability of DMA descriptor 524 and buffer 520 at the storage device 504 to the DMA descriptor and buffer checker 516 so that the I/O controller 526 can use this information to determine whether sufficient resources (e.g., DMA descriptor and/or buffer) are available to process the I/O command. While herein the present disclosure, resources refers to hardware token 510, DMA descriptor 524, and buffer 520 by way of example, resources may also refer to other parameters depending on the type of storage device 504 or system 500.

0061In some embodiments, an oversubscription factor may be utilized to adjust the amount of resources that the device hint has determined to be available. In other words, because the volume of resources indirectly represents a processing power or processing ability of a storage device, scheduling I/O commands based on actual resources may introduce less I/O command submission than the actual capability of the storage device. For example, a 128KB read operation may be split into multiple data segments inside the storage device, and each of the data segments may be assigned a DMA descriptor and a buffer, and may be processed with DMA operation. In some embodiments, even if most of DMA operations are finished, and the DMA descriptors and the buffer are returned to the storage device, the host device cannot detect that the DMA descriptor and the buffer are available until the I/O command is complete and this indication is provided to the I/O controller 526.

0062Therefore, the oversubscription factor accounts for this difference by adjusting the available resources. More particularly, the resource-based I/O controller 526 adjusts the amount of resources count of the HW-HINT by considering the differences between resource release time of the host device and the storage device.

0063Accordingly, the I/O controller 526 may take the I/O command and determine whether certain criteria are satisfied to process this I/O command, and if so, then send the I/O command to the queue 506. For example, the I/O controller 526 may determine that a criteria is satisfied if the storage device 504 has sufficient resources (e.g., DMA descriptors and buffer) as dictated by the I/O resource requirement. In other words, for example, if the resource requirement indicates that a particular I/O command requires 11 DMA descriptors and 88K of buffer, and 20 DMA descriptors and 100K of buffer are available, then the criteria may be satisfied. On the other hand, if there were only 10 DMA descriptors or 50K of buffer, then the resource criteria may not be satisfied. Accordingly, if the criteria are not satisfied, then the I/O command is sent back to the block multiple queue 508 and rescheduled for resubmission to the I/O controller 526 at a later time. At that point, the I/O controller 526 will determine again, whether the criteria are satisfied to process this I/O command the then either send the I/O command to the queue 506 or back to the block multiple queue 508.

0064<figref idref="f0007">FIG. 7</figref> is a flow chart illustrating the steps taken by the traffic controller 512 to determine whether to send the I/O command to the queue 506 or to send the I/O command back to the block multiple queue 508 to be rescheduled. In some embodiments, the I/O command is sent from the block multiple queue 508 to the I/O controller 526, and the traffic controller 512 determines whether the I/O command satisfies certain criteria to send the I/O command to the queue 506. More particularly, when the I/O command is received by the traffic controller 512, the I/O command is converted from the I/O request format to the I/O resource requirement format by the converter 528 (702). Thus, the I/O command is now in the format that facilitates determining resource requirements for processing the I/O command, such as, for example, the size and address of the command. In some embodiments, such converted format may be in the form of an I/O descriptor ("IOD") structure.

0065In some embodiments, the host device includes a memory 518. Memory 518 may be, for example, an internal memory used for short-term storage or temporary memory during operation of the system 500. The memory 518 may include a DRAM (dynamic random access memory), SRAM (static random access memory), and/or DTCM (Data Tightly Coupled Memory). The memory 518 may be used to store, for example, a data structure that includes an I/O statistic and resource utilization counter managed by a device driver to determine the availability of resources for each queue 506. For example, the information may include, a reserved value for tokens, threshold value for tokens, traffic collision count, resource collision count, and on-the-fly count. In some embodiments, the reserved value for tokens correspond to a number of tokens that are reserved for specific queues, as discussed above, for example, where 4 tokens were reserved for each of the queues Q1 to Qn. The threshold value corresponds to the dynamic traffic threshold window value, which will be described in more detail later. The traffic collision count corresponds to the number of times an I/O command collided or had to wait due to all of the token being used. The resource collision count corresponds to the number of times a restricted resource not assignable. Herein the present disclosure, "on-the-fly" refers to the number of I/O requests or I/O request counts that are already scheduled in the queue and/or are being processed. Thus, the I/O command, which is now in the form of I/O resource requirements, is evaluated to determine whether the amount of reserved resources (e.g., reserved hardware token value) is greater than an amount of resources that are on-the-fly (e.g., on-the-fly count) (704). If the amount of reserved resources is greater than the amount of resources that are on-the-fly, then an I/O statistics and resource utilization counter is updated (710) and the I/O command is sent to the queue 506 to be scheduled (712). If the amount of reserved resources is not greater than the amount of resources that are on-the-fly, then a determination is made whether a traffic threshold is greater than an amount of resources that are on-the-fly and whether there are sufficient resources (706). If the traffic threshold is not greater than the amount of resources that are on-the-fly, then the I/O command is sent back to the block multiple queue 508 where it is rescheduled for a later time, and the traffic collision count is incremented. On the other hand, if the traffic threshold is greater than the amount of resources that are on-the-fly and there are sufficient resources, then a serialized hardware resources check is performed (708).

0066In some embodiments, the serialized hardware resources check includes determining whether an amount of required resources to process this I/O command is less than an amount of currently available resources. In other words, a decision is made to determine whether there are sufficient resources to process this I/O command. If there aren't enough available resources, then the I/O command is sent back to the block multiple queue 508 where it is rescheduled for a later time and the resource collision count is incremented. If the required amount of resources is less than the amount of currently available resources, then there are enough resources to process this I/O command and therefore the I/O stat and resource utilization counter is updated (710) and the I/O command is sent to the queue 506 to be scheduled (712).

0067In some embodiments, the traffic threshold is a dynamic threshold that is adjustable based on current traffic status information pertaining to traffic collision, resource collision, and current on-the-fly request count. <figref idref="f0008">FIG. 8</figref> is a flow chart that illustrates a process for adjusting the traffic threshold value. For example, the traffic threshold window may be determined to be too small if there is a traffic collision but there is no resource collision, and the current threshold is less than some predetermined threshold limit. In this case, the traffic threshold window may be increased (802). On the other hand, the traffic threshold window may be determined to be big enough if there is no traffic collision and there is no resource collision, and the amount of on-the-fly resources is less than the traffic threshold. In this case, the traffic threshold window may be decreased (804). If there are no I/O commands for some predetermined duration of time, for example, if there is a pause or a momentary stop in receiving I/O commands from the I/O scheduler 508, for 10 seconds, 12 second, etc., then the traffic threshold window may be reset to a default value, for example, 4 or 8 (806). It should be noted that the example duration and default values are provided merely by way of example and is not intended to be limiting. Accordingly, the traffic threshold value may be dynamically adjusted in real-time.

0068<figref idref="f0009">FIG. 9</figref> is a flow chart illustrating a process for scheduling of I/O commands based on multiple resource parameters, according to some embodiments. In some embodiments, a system includes a storage device and a host device, and the host device may be configured to obtain information pertaining to an amount of resources associated with the storage device during an initialization by the device driver. In some embodiments, this happens one time during the initialization. The host device includes an I/O scheduler configured to receive an I/O command from an application (e.g., software application) that includes an I/O request (902). The host device further includes an I/O controller and an I/O resource requirement may be generated based on the I/O command so that the resource requirements for the I/O command may be determined (904).

0069In some embodiments, the I/O controller includes a traffic controller. Accordingly, the traffic controller may determine, that the information received regarding the resources associated with the storage device satisfies a criteria based on the I/O resource requirement of the I/O command (906). In response to the criteria being satisfied, the traffic controller may send the I/O command to a queue, so that the I/O command may be scheduled for processing (908). Next, the required resources are allocated to the I/O command based on the amount of required resources that were determined from the I/O resource requirement (9910). Once the resources are allocated, the I/O command may be processed or executed by the storage device (e.g., DMA operation occurring between host memory and NAND flash), and a determination is made that the processing of the I/O command is complete. Once the processing of the I/O command is complete, the previously allocated resources may be released by deallocating the resources from the I/O command (912). Accordingly, I/O commands may be scheduled on storage devices based on multiple parameter resources according to various embodiments of the present disclosure. Accordingly, delays resulting from uncontrolled I/O traffic and uncontrolled I/O bandwidth collisions may be reduced or avoided. However, the described methods and operations are provided in by way of example only so that a person having ordinary skill in the art would understand, and may involve various additional steps that are not explicitly provided herein. In some embodiments, the temporal order of the operations may be varied.

0070It will be understood that, although the terms "first," "second," "third," etc., may be used herein to describe various elements, components, regions, layers and/or sections, these elements, components, regions, layers and/or sections should not be limited by these terms. These terms are used to distinguish one element, component, region, layer or section from another element, component, region, layer or section. Thus, a first element, component, region, layer or section described below could be termed a second element, component, region, layer or section, without departing from the spirit and scope of the present invention.

0071It will be understood that when an element or layer is referred to as being "on," "connected to," or "coupled to" another element or layer, it can be directly on, connected to, or coupled to the other element or layer, or one or more intervening elements or layers may be present. In addition, it will also be understood that when an element or layer is referred to as being "between" two elements or layers, it can be the only element or layer between the two elements or layers, or one or more intervening elements or layers may also be present.

0072The terminology used herein is for the purpose of describing particular embodiments and is not intended to be limiting of the present invention. As used herein, the singular forms "a" and "an" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises," "comprising," "includes," and "including," when used in this specification, specify the presence of the stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items. Expressions such as "at least one of," when preceding a list of elements, modify the entire list of elements and do not modify the individual elements of the list.

0073As used herein, the terms "substantially," "about," and similar terms are used as terms of approximation and not as terms of degree, and are intended to account for the inherent deviations in measured or calculated values that would be recognized by those of ordinary skill in the art. Further, the use of "may" when describing embodiments of the present invention refers to "one or more embodiments of the present invention." As used herein, the terms "use," "using," and "used" may be considered synonymous with the terms "utilize," "utilizing," and "utilized," respectively.

0074The electronic or electric devices and/or any other relevant devices or components according to embodiments of the present invention described herein may be implemented utilizing any suitable hardware, firmware (e.g. an application-specific integrated circuit), software, or a combination of software, firmware, and/or hardware. For example, the various components of these devices may be formed on one integrated circuit (IC) chip or on separate IC chips. Further, the various components of these devices may be implemented on a flexible printed circuit film, a tape carrier package (TCP), a printed circuit board (PCB), or formed on one substrate. Further, the various components of these devices may be a process or thread, running on one or more processors, in one or more computing devices, executing computer program instructions and interacting with other system components for performing the various functionalities described herein. The computer program instructions are stored in a memory which may be implemented in a computing device using a standard memory device, such as, for example, a random access memory (RAM). The computer program instructions may also be stored in other non-transitory computer readable media such as, for example, a CD-ROM, flash drive, or the like. Also, a person of skill in the art should recognize that the functionality of various computing devices may be combined or integrated into a single computing device, or the functionality of a particular computing device may be distributed across one or more other computing devices without departing from the spirit and scope of the example embodiments of the present invention.

0075It will be understood that any of the components or any combination of the components described throughout the present disclosure may be used to performed one or more of the operations described in the flow charts. Moreover, the described operations are merely example operations and may involve various additional steps that are not explicitly described herein. Additionally, the temporal order of the operations may be varies.

0076Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and/or the present specification, and should not be interpreted in an idealized or overly formal sense, unless expressly so defined herein.

0077Embodiments described herein are examples only. One skilled in the art may recognize various alternative embodiments from those specifically disclosed. Those alternative embodiments are also intended to be within the scope of this disclosure. As such, the embodiments are limited only by the following claims and their equivalents.