EP1544737A1

Thread execution scheduler for multi-processing system and method

Abstract

A processing system includes a plurality of processors capable of executing a plurality of threads and supporting at least one of hardware context switching and software context switching. The processing system also includes at least one hardware scheduler capable of scheduling execution of the plurality of threads by the plurality of processors. The at least one hardware scheduler is capable of scheduling execution of the threads by performing instruction-by-instruction scheduling of the threads.

EP1544737A1, drawing sheet 1 — Sheet 1 of 12

Term

Term ended

Projected expiry passed 14 December 2024, 1.8 years ago.

26 claims: 14 independent, 12 dependent

1
A processing system, comprising:a plurality of processors capable of executing a plurality of threads and supporting at least one of hardware context switching and software context switching;and at least one hardware scheduler capable of scheduling execution of the plurality of threads by the plurality of processors by performing instruction-by-instruction scheduling of the threads.
4
The processing system of any preceding Claim, further comprising a hardware concurrency engine capable of managing a plurality of concurrency primitives that coordinate execution of the threads by the processors.
5
The processing system of any preceding Claim, wherein:the at least one hardware scheduler comprises a hardware scheduler disposed in one of the processors;the processing system further comprises at least one memory, at least one programmable hardware processing engine, at least one interface to an external device, and a network coupling the processors, the memory, the hardware processing engine, and the interface;and the processors, the memory, the hardware processing engine, the interface, and the network comprise a system on a chip.
6
The processing system of any preceding Claim, wherein the at least one scheduler is capable of scheduling execution of the plurality of threads by at least one of:initiating a context switch every clock cycle;and initiating a context switch when an instruction from one of the threads is determined to have an associated latency.
7
The processing system of any preceding Claim, wherein at least one of the processors comprises a dedicated register set for each thread executed by the processor, the dedicated register set for a particular thread storing a priority associated with the particular thread.
8
The processing system of any preceding Claim, wherein the plurality of processors comprises homogenous processors in a symmetric multi-processing arrangement.
9
A processor, comprising:a plurality of pipeline stages collectively capable of executing a plurality of instructions from a plurality of threads;a memory capable of storing the instructions;and a hardware scheduler capable of scheduling execution of the instructions from the plurality of threads by the pipeline stages.
12
An apparatus, comprising:at least one of: an input device and an interface to the input device, the input device capable of providing input data;at least one of: an output device and an interface to the output device, the output device capable of receiving output data;and a processing system capable of receiving the input data and generating the output data, the processing system comprising: a plurality of processors capable of executing a plurality of threads;and at least one hardware scheduler capable of scheduling execution of the plurality of threads by the plurality of processors.
15
The apparatus of any of Claims 12 to 14, wherein:the apparatus is coupled to a network;and the processing system is capable of performing at least one of packet forwarding and traffic management.
20
A method, comprising:executing a plurality of threads using at least one processor;and scheduling execution of the plurality of threads by the at least one processor using at least one hardware scheduler.
23
The method of any of Claims 20 to 22, wherein scheduling execution of the plurality of threads comprises determining a dynamic priority for each thread using a formula of:P t = P t 0 + K · W td D , where P t represents the dynamic priority for a particular thread, P t0 represents a static component of the dynamic priority, K represents a constant, W td represents an amount of work to be done by the thread, and D represents an amount of time from a deadline for the thread.
24
The method of any of Claims 20 to 23, further comprising managing a plurality of concurrency primitives that coordinate execution of the threads by the processors using a hardware concurrency engine; and wherein scheduling execution of the plurality of threads comprises determining a dynamic priority for each thread using a formula of:P t = P t 0 + K · W td D + Y t , where P t represents the dynamic priority for a particular thread, P t0 represents a static component of the dynamic priority, K represents a constant, W td represents an amount of work to be done by the thread, D represents an amount of time from a deadline for the thread, and Y t represents a value from the concurrency engine.
25
The method of any of Claims 20 to 24, further comprising storing a priority for a particular thread in a dedicated register set associated with the particular thread;wherein scheduling execution of the plurality of threads comprises using the priority stored in the dedicated register set.
26
The method of any of Claims 20 to 25, wherein scheduling execution of the plurality of threads comprises at least one of:initiating a context switch every clock cycle;and initiating a context switch when an instruction from one of the threads is determined to have an associated latency.

Independent claims14

119 paragraphs in 6 sections, as filed

This patent application claims priority under 35 U.S.C. § 119(e) to the following U.S. Provisional Patent Applications: <ul id="ul0001" list-style="none" compact="compact"><li>Serial No. 60/531,502 filed on December 19, 2003; and</li><li>Serial No. 60/531,538 filed on December 19, 2003;</li></ul> which are hereby incorporated by reference.

This patent application is related to U.S. Patent Application No. 10/967,059 entitled "ACCELERATOR FOR MULTIPROCESSING SYSTEM AND METHOD" filed on October 15, 2004, [Attorney Docket No. 03-OT-125 (STMI01-03125)], which is hereby incorporated by reference.

TECHNICAL FIELD

This disclosure is generally directed to processing systems and more specifically to a thread execution scheduler for multi-processing system and method.

BACKGROUND

Multi-processor computing systems are becoming more and more popular for use in applications that require large amounts of processing power. One type of multi-processor computing system is a symmetric multi-processing or "SMP" architecture. In an SMP architecture, multiple processors are used to perform tasks or processes in parallel. This allows, for example, the processors to execute different portions or "threads" of an application at the same time. Any available processor may be assigned to execute one or multiple threads. Also, additional processors may be added to improve the performance of the computer system and to execute additional threads.

In order to execute multiple threads at the same time, the execution of the threads typically needs to be scheduled. Within a single processor, scheduling typically involves selecting which thread to execute at any given time. An instruction from the selected thread is then issued for execution by the processor. Any remaining threads are "stalled," meaning that instructions for those threads are not issued to the processor for execution. A problem with conventional scheduling mechanisms is that the mechanisms are typically slow, which often reduces the performance of the multi-processor systems.

SUMMARY

This disclosure provides a thread execution scheduler for multi-processing system and method.

In a first embodiment, a processing system includes a plurality of processors capable of executing a plurality of threads and supporting at least one of hardware context switching and software context switching. The processing system also includes at least one hardware scheduler capable of scheduling execution of the plurality of threads by the plurality of processors. The at least one hardware scheduler is capable of scheduling execution of the threads by performing instruction-by-instruction scheduling of the threads.

In particular embodiments, the at least one scheduler is capable of determining a dynamic priority for each thread. Also, in particular embodiments, the dynamic priority is based on an amount of work to be done by the thread and an amount of time from a deadline for the thread. The amount of work to be done by the thread represents a difference between an estimated total number of instructions for the thread to do the work and a number of instructions actually executed by the thread. The amount of time from the deadline for the thread represents a difference between a value of a system clock when the thread must be completed and a current value of the system clock.

In a second embodiment, a method includes executing a plurality of threads using at least one processor. The method also includes scheduling execution of the plurality of threads by the at least one processor using at least one hardware scheduler.

In a third embodiment, a processor includes a plurality of pipeline stages collectively capable of executing a plurality of instructions from a plurality of threads. The processor also includes a memory capable of storing the instructions. In addition, the processor includes a hardware scheduler capable of scheduling execution of the instructions from the plurality of threads by the pipeline stages.

In a fourth embodiment, an apparatus includes at least one of an input device and an interface to the input device. The input device is capable of providing input data. The apparatus also includes at least one of an output device and an interface to the output device. The output device is capable of receiving output data. In addition, the apparatus includes a processing system capable of receiving the input data and generating the output data. The processing system includes a plurality of processors capable of executing a plurality of threads and at least one hardware scheduler capable of scheduling execution of the plurality of threads by the plurality of processors.

Other technical features may be readily apparent to one skilled in the art from the following figures, descriptions, and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of this disclosure and its features, reference is now made to the following description, taken in conjunction with the accompanying drawings, in which:

FIGURE 1 illustrates an example processing system according to one embodiment of this disclosure;

FIGURE 2 illustrates additional details of an example processing system according to one embodiment of this disclosure;

FIGURE 3 illustrates an example concurrency engine for accelerating multi-processing in a processing system according to one embodiment of this disclosure;

FIGURES 4A and 4B illustrate example scheduling in a multi-threaded processor according to one embodiment of this disclosure;

FIGURE 5 illustrates an example method for accelerating multi-processing according to one embodiment of this disclosure;

FIGURE 6 illustrates an example method for thread scheduling in a multi-processing system according to one embodiment of this disclosure; and

FIGURE 7 illustrates an example apparatus using a processing system that accelerates multi-processing according to one embodiment of this disclosure.

DETAILED DESCRIPTION

FIGURE 1 illustrates an example processing system 100 according to one embodiment of this disclosure. The embodiment of the processing system 100 shown in FIGURE 1 is for illustration only. Other embodiments of the processing system 100 may be used without departing from the scope of this disclosure.

In this example embodiment, the processing system 100 includes a processor core 101 having multiple processors 102a-102n. The processors 102a-102n represent any suitable processing devices, such as reduced instruction set computer ("RISC") processors. As a particular example, the processors 102a-102n may represent multi-threaded processors having separate register banks for different threads. As another particular example, the processors 102a-102n may represent homogenous processors configured in a symmetric multi-processing ("SMP") arrangement.

In this example, the processors 102a-102n include customized instruction sets 104a-104n. Each of the customized instruction sets 104a-104n allows one of the processors 102a-102n to operate according to a particular set of instructions. For example, the customized instruction sets 104a-104n could be configured to allow the processors 102a-102n to operate in a particular environment. Each of the customized instruction sets 104a-104n could be implemented in any suitable manner, such as by using an application specific integrated circuit ("ASIC") implemented in one or more of the last metal layers of the processors 102a-102n. At least some of the customized instruction sets 104a-104n could represent the same instruction set, or all of the customized instruction sets 104a-104n could represent different instruction sets. In this document, the term "each" refers to every of at least a subset of the identified items.

The processors 102a-102n also include one or more embedded memories, such as random access memories ("RAM") 106a-106n. The random access memories 106a-106n are used to store any suitable information used by the processors 102a-102n, such as data and instructions executed by the processors 102a-102n. Each of the random access memories 106a-106n could represent any suitable amount of memory. Also, other types of embedded memories, such as read only memories ("ROM"), could be used in the processors 102a-102n.

In the illustrated embodiment, the processors 102a-102n are coupled to a network 108. In this document, the term "couple" and its derivatives refer to any direct or indirect communication between two or more elements, whether or not those elements are in physical contact with one another. The network 108 facilitates communication of information between various components of the processing system 100. For example, the network 108 allows the processors 102a-102n to retrieve instructions to be executed by the processors 102a-102n and data to be used during execution of the instructions. In some embodiments, the processing system 100 represents a "system on a chip" ("SoC"), and the network 108 represents a "network on a chip" ("NoC").

In this example, the network 108 is coupled to one or more memories 110, one or more hardware processing engines ("PE") 112, and one or more additional hardware processing engines 114. The memory 110 represents one or more memories capable of storing information used by the processing system 100. The memory 110 could represent a single type of memory or multiple types of memory, such as random access memory, read only memory, flash memory, programmable read only memory, and/or other or additional types of memory.

The hardware processing engines 112, 114 represent programmable hardware that may be customized according to particular needs. For example, the hardware processing engines 112, 114 may be programmed for use in particular applications or particular environments. The hardware processing engines 112, 114 could be programmed to perform any of a wide variety of functions. The hardware processing engines 112, 114 represent any suitable structures capable of providing customized or programmable functionality. As particular examples, the hardware processing engine 112 could be implemented using ASICs or Sea of Gates ("SoG") structures, and the hardware processing engine 114 could be implemented using field programmable gate arrays ("FPGA"). While FIGURE 1 illustrates the use of both hardware processing engines 112 and 114, any number of different types of hardware processing engines (including a single type) could be used in the processing system 100.

In the illustrated example, the network 108 is also coupled to multiple interfaces 116-124. The interfaces 116-124 allow the processing system 100 to communicate with various external devices, such as external memories or peripheral devices. As shown in FIGURE 1, a micro-processing unit ("MPU") interface 116 facilitates communication with an external micro-processing unit. A System Packet Interface Level 4, Phase 2 ("SPI4.2") interface 118 allows the processing system 100 to communicate using multiple communication protocols and variable data rates through a common system-level interface standard. A hyper transport interface 120 facilitates high bandwidth, chip-to-chip communications. A memory interface 122 allows communication with an external memory, such as a quad data rate ("QDR") memory. A general-purpose interface 124 allows other communications to and from the processing system 100. These represent some of the interfaces that may be used in the processing system 100. Other or additional interfaces, such as an interface to a co-processor or an application-specific interface, could also be used in the processing system 100.

The various components in the processing system 100 may communicate over the network 108 using any suitable protocol or protocols. In some embodiments, the components in the processing system 100 communicate with the network 108 using a standardized protocol. As a particular example, hardware wrappers may be used to convert between an internal data format used by the hardware processing engines 112, 114 or other components and a packet-oriented format used by the network 108.

In one aspect of operation, the processors 102a-102n in the processor core 101 are coordinated using various concurrency primitives. The concurrency primitives include structures such as monitors, conditions, semaphores, priority queues, and counters. The processor core 101 implements an accelerator in hardware that accelerates the use of the concurrency primitives. This allows the processor core 101 to more effectively perform coordination. This may be particularly useful in certain embodiments, such as when the processors 102a-102n represent homogenous processors arranged in an SMP configuration. Through the use of the hardware accelerator in the processor core 101, the performance of the processor core 101 may increase.

In another aspect of operation, at least some of the processors 102a-102n in the processor core 101 are capable of executing multiple threads, such as when at least one of the processors 102a-102n is capable of hardware multithreading. The threads represent portions of one or more applications, where the portions are capable of being executed in parallel. The processor core 101 implements at least one scheduler in hardware that schedules the execution of the threads by the processors 102a-102n. This may allow the processor core 101 to more efficiently schedule execution of the threads, which may also help to increase the performance of the processor core 101.

While this document may describe a processing system 100 that implements both acceleration of concurrency primitives and scheduling of thread execution, only one of these features could be implemented in a processing system. For example, a processing system could implement only the acceleration of concurrency primitives without implementing the thread scheduling mechanism described in more detail below. Similarly, a processing system could implement only the scheduling of thread execution without implementing the concurrency primitive acceleration mechanism described in more detail below.

Although FIGURE 1 illustrates one example of a processing system 100, various changes may be made to FIGURE 1. For example, the processing system 100 could include any number and type of processors and any number and type of hardware processing engines. The processing system 100 could also include any number and type of interfaces to external devices. In addition, while the processing system 100 has been described as possibly representing a "system on a chip," the processing system 100 could be implemented using any suitable number of chips or other structures.

FIGURE 2 illustrates additional details of an example processing system 100 according to one embodiment of this disclosure. The embodiment of the processing system 100 shown in FIGURE 2 is for illustration only. Other embodiments of the processing system 100 may be used without departing from the scope of this disclosure.

As shown in FIGURE 2, the processing system 100 includes the processors 102a-102n, which are coupled to network interfaces 202a-202n. The network interfaces 202a-202n facilitate communication between the processors 102a-102n and a network, such as the network 108. The network interfaces 202a-202n may include any suitable interface for communicating over a network.

The processors 102a-102n are also coupled to other local devices 204a-204n. The other local devices 204a-204n represent any suitable devices that may be integrated into the processing system 100 and used by the processors 102a-102n. For example, the other local devices 204a-204n could represent co-processors or hardware accelerators. The other local devices 204a-204n could also represent memories, such as internal caches, or any other or additional devices.

In the illustrated example, the processors 102a-102n communicate with a concurrency engine 206 through the network interfaces 202a-202n and the network 108. The concurrency engine 206 represents a hardware device used to accelerate the use of concurrency primitives, which coordinate the operation of the processors 102a-102n. For example, the concurrency engine 206 may perform concurrency operations that implement and manage the use of semaphores, priority queues, counters, timers, monitors, and associated condition variables. The concurrency engine 206 may also support the scheduling of the execution of various threads by the processors 102a-102n. The concurrency engine 206 could represent any device capable of supporting the use of concurrency primitives. As a particular example, the processing system 100 could represent a system on a chip, and the concurrency engine 206 could represent a device in a region of a 32-bit or 64-bit address space.

In some embodiments, the concurrency engine 206 implements a number of concurrency "objects." These objects support the various coordination functions performed by the concurrency engine 206. Various techniques may be common or generic in handling the concurrency objects, such as methods for creating an object, deleting an object, and calling an object method.

The various functions performed by the concurrency engine 206 may be implemented in any suitable manner. For example, in a 32-bit address space with a 32-bit word size, the concurrency engine 206 could be mapped into a 24-bit or 16 megabyte region. Reading from or writing to locations in this space performs concurrency operations. Also, each of the concurrency objects may have two associated 32-bit values (denoted <i>val1</i> and <i>val2</i>), which are used to implement the functionality of the concurrency operation associated with an object. In other embodiments, the functions performed by the concurrency engine 206 could be invoked using special concurrency control instructions defined in the instruction set architecture of the processors 102a-102n.

In particular embodiments, the address of a location in the address space of the concurrency engine 206 encodes an object index, an object type, and an operation type for a particular concurrency operation. For example, bits 2:4 in the address could encode an operation type, bits 5:7 in the address could encode an object type, and bits 8:23 in the address could encode an object index. The operation type identifies the concurrency operation desired, the object type identifies the type of concurrency object (counter, timer, semaphore, etc.) associated with the desired operation, and the object index identifies a particular concurrency object.

Different types of objects are encoded using bits 5:7 of the address. One example of the encodings for the object types in bits 5:7 is shown in Table 1. <tables id="tabl0001" num="0001"><table frame="all"><title>Table 1</title><tgroup cols="2" colsep="1" rowsep="1"><colspec colnum="1" colname="col1" colwidth="78.75mm" /><colspec colnum="2" colname="col2" colwidth="78.75mm" /><thead valign="top"><row><entry namest="col1" nameend="col1" align="center"><u>Example Object Type Encoding</u></entry><entry namest="col2" nameend="col2" align="center"><u>Object Type</u></entry></row></thead><tbody valign="top"><row><entry namest="col1" nameend="col1" align="center">0</entry><entry namest="col2" nameend="col2" align="center">Counting semaphore</entry></row><row><entry namest="col1" nameend="col1" align="center">1</entry><entry namest="col2" nameend="col2" align="center">Monitor</entry></row><row><entry namest="col1" nameend="col1" align="center">2</entry><entry namest="col2" nameend="col2" align="center">Normal condition</entry></row><row><entry namest="col1" nameend="col1" align="center">3</entry><entry namest="col2" nameend="col2" align="center">Deferred condition</entry></row><row><entry namest="col1" nameend="col1" align="center">4</entry><entry namest="col2" nameend="col2" align="center">Timeout condition</entry></row><row><entry namest="col1" nameend="col1" align="center">5</entry><entry namest="col2" nameend="col2" align="center">Priority Queue</entry></row><row><entry namest="col1" nameend="col1" align="center">6</entry><entry namest="col2" nameend="col2" align="center">Timer</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="center">7</entry><entry namest="col2" nameend="col2" align="center">Counter</entry></row></tbody></tgroup></table></tables> The operation type is encoded using bits 2:4 of an address. The operation type for a particular object depends on the object type. In particular embodiments, the values <i>val1</i> and <i>val2</i> for a specific object are accessed using an operation type of six or seven, respectively. It should be understood that other encodings and/or mechanisms for accessing the concurrency engine 206 may be used.

A generic scheme for creating and deleting objects could be implemented as follows. One of the processors 102a-102n reads from an address with an object index of zero (address bits 8:23 are zero), bits 5:7 indicating the type of concurrency object requested (semaphore, counter, etc.), and an operation type of zero (bits 2:4 are zero). The concurrency engine 206 then returns a value with bits 8:23 encoding either the object index of the newly created object or a value of zero if the object could not be created for some reason. To free an object, one of the processors 102a-102n could write a 32-bit value with bits 8:23 indicating the object index, bits 5:7 encoding the object type, and bits 2:4 encoding a value of zero.

In some embodiments, high-use concurrency operations are implemented by a single read or write operation to a location in the concurrency engine address space. However, operations that involve multiple transactions could also be supported. Operations performed with a single read or write may be performed atomically. As a result, these operations are performed for a thread on one of the processors 102a-102n without concern that operations from other threads could corrupt object data structures in mid--operation. Operations that involve multiple transactions (such as multiple reads and/or writes) may be performed by locking a concurrency object, performing the operations, and unlocking the object. In particular embodiments, an object may be locked by reading with an operation type of zero and unlocked by writing with an operation type of zero. This encoding does not collide with the object creation and deletion encoding described above because the object identifier is non-zero for a lock/unlock operation. If multiple threads attempt to lock the same object, the concurrency engine 206 may stall one, some, or all of these threads so that only one thread at a time can execute while the object is locked.

At least some of the concurrency objects have an associated wait queue that identifies any threads waiting for some kind of state change in an object. For example, multiple threads may be attempting to lock and use an object. One thread is allowed to execute while the others are placed in the wait queue. Once the first thread unlocks the object, another thread from the wait queue is allowed to use the object.

If there are multiple threads in a wait queue, the concurrency engine 206 often needs to determine which one of the waiting threads should be woken up and allowed to resume execution. Several techniques may be used by the concurrency engine 206 to select one of the multiple threads. For example, the concurrency engine 206 could use round robin ordering, a first-in first-out (FIFO) ordering where the thread that has been waiting the longest is serviced first, an ordering based on static priority, and an ordering based on dynamically-calculated priority. In the static and dynamic ordering schemes, FIFO ordering may be used if multiple threads have the same priority.

The following describes several examples of the various functions supported by the concurrency engine 206. This represents one specific implementation of the concurrency engine 206. Any number of modifications may be made to the concurrency engine 206 described below according to particular needs.

As described above, the concurrency engine 206 may support the scheduling of the execution of various threads by the processors 102a-102n. The concurrency engine 206 facilitates the scheduling of thread execution by suspending and resuming execution of the threads. The concurrency engine 206 executes various functions to support the suspension and resumption of the threads. For example, the concurrency engine 206 supports the stalling of read or write acknowledgements, which are generated and provided to the processors 102a-102n after successful read or write operations. If the network 108 supports split-transactions, a read or a write operation may require a separate response before one of the processors 102a-102n continues execution. The concurrency engine 206 uses this mechanism to suspend execution by one of the processors 102a-102n by delaying the response to a read or write operation requested by that processor until the executing thread is allowed to continue.

The concurrency engine 206 also supports interrupts and/or exceptions. One of the processors 102a-102n may request permission for an operation by reading or writing to the address space of the concurrency engine 206. If the operation is allowed, the concurrency engine 206 allows the transaction to complete normally. If the operation should be delayed (such as to enforce SMP execution semantics), the concurrency engine 206 issues an interrupt or exception, which causes that processor to save its state and switch to another thread.

The concurrency engine 206 could further have direct hardware access to logic in the processors 102a-102n. The concurrency engine 206 may use this access to perform any of a wide variety of functions. For example, the concurrency engine 208 could enable or disable processor execution or increase power or clock speed for higher priority processes. The concurrency engine 208 could also gate or alter a clock signal of at least one of the processors 102a-102n. The concurrency engine 208 could further provide control signals to a thread scheduler (described in more detail below) in at least one of the processors 102a-102n. In addition, the concurrency engine 208 could provide control signals to load/store logic in at least one of the processors 102a-102n, where the load/store logic performs the read and write operations.

While this represents several possible ways that the concurrency engine 206 facilitates the scheduling of thread execution, the concurrency engine 206 could use any other or additional approaches or a combination of approaches. For example, the concurrency engine 206 may initially stall a thread executed on one of the processors 102a-102n by delaying a read or write acknowledgement. If the concurrency engine 206 later determines the processor could be doing other useful work, the concurrency engine 206 could provide an interrupt or exception.

As described above, the concurrency engine 206 may also implement and manage the use of objects providing semaphores, priority queues, counters, timers, monitors, and associated condition variables.

The concurrency engine 206 supports the use of counter objects to implement counters in the processing system 100. A counter could represent a 64-bit value. The 32 lower-order bits of the counter value are stored in <i>val1,</i> and the 32 higher-order bits are stored in <i>val2.</i> Increments and decrements are performed on the 64-bit value. Read and write operations involving the lower-order bits of the counter value may be performed using an operation type of six (encoded in bits 2:4), and the higher-order bits of the counter value may be directly accessed using an operation type of seven. Table 2 gives additional example operation types for a counter object and their associated semantics, assuming a 32 bit architecture. <tables id="tabl0002" num="0002"><table frame="all"><title>Table 2</title><tgroup cols="3" colsep="1" rowsep="1"><colspec colnum="1" colname="col1" colwidth="52.50mm" /><colspec colnum="2" colname="col2" colwidth="52.50mm" /><colspec colnum="3" colname="col3" colwidth="52.50mm" /><thead valign="top"><row><entry namest="col1" nameend="col1" align="center"><u>Operation Type (encoding of bits 2:4)</u></entry><entry namest="col2" nameend="col2" align="center"><u>Associated Action on Read</u></entry><entry namest="col3" nameend="col3" align="center"><u>Associated Action on Write of Value N</u></entry></row></thead><tbody valign="top"><row><entry namest="col1" nameend="col1" align="center">1</entry><entry namest="col2" nameend="col2" align="center">Return the current 32 low-order bits.</entry><entry namest="col3" nameend="col3" align="center">Set the 32 low-order bits of the counter to N, and clear the 32 high-order bits.</entry></row><row><entry namest="col1" nameend="col1" align="center">2</entry><entry namest="col2" nameend="col2" align="center">Increment the current value, and return the 32 low-order bits of the new result.</entry><entry namest="col3" nameend="col3" align="center">Add N to the current value.</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="center">3</entry><entry namest="col2" nameend="col2" align="center">Decrement the current value, and return the 32 low-order bits of the new result.</entry><entry namest="col3" nameend="col3" align="center">Subtract N from the current value.</entry></row></tbody></tgroup></table></tables>

The concurrency engine 206 also supports the use of counting semaphore objects. The value of a semaphore is held in <i>val1,</i> which is interpreted as a 32-bit unsigned integer. In some embodiments, the concurrency engine 206 may use <i>val2</i> to encode operational variants, such as an ordering scheme for the object's wait queue. In particular embodiments, the concurrency engine 206 places threads waiting to acquire a semaphore in a FIFO wait queue, a wait queue ordered based on request priority or an application-specific ordering, or other type of wait queue. The thread at the head of the wait queue is allowed to resume execution when the value <i>val1</i> is greater than or equal to the number of units the thread is requesting. Table 3 gives example operation types for a counting semaphore object and their associated semantics. <tables id="tabl0003" num="0003"><table frame="all"><title>Table 3</title><tgroup cols="3" colsep="1" rowsep="1"><colspec colnum="1" colname="col1" colwidth="52.50mm" /><colspec colnum="2" colname="col2" colwidth="52.50mm" /><colspec colnum="3" colname="col3" colwidth="52.50mm" /><thead valign="top"><row><entry namest="col1" nameend="col1" align="center"><u>Operation Type (encoding of bits 2:4)</u></entry><entry namest="col2" nameend="col2" align="center"><u>Associated Action on Read</u></entry><entry namest="col3" nameend="col3" align="center"><u>Associated Action on Write of Value N</u></entry></row></thead><tbody valign="top"><row><entry namest="col1" nameend="col1" align="center">1</entry><entry namest="col2" nameend="col2" align="center">Acquire one unit of the semaphore. Return the number of units remaining. Stall the caller if the semaphore is currently zero.</entry><entry namest="col3" nameend="col3" align="center">Acquire N units of the semaphore. Stall the caller if the current value of the semaphore is less than N.</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="center">2</entry><entry namest="col2" nameend="col2" align="center">Release one unit of the semaphore. Return the 32 low-order bits of the new result. Wake up any threads that have stalled due to an acquire operation.</entry><entry namest="col3" nameend="col3" align="center">Release N units of semaphore. Wake up any threads that have stalled due to an acquire operation.</entry></row></tbody></tgroup></table></tables> The term "caller" in Table 3 and in the following description refers to the thread that is performing the read or write access to the concurrency engine 206.

The concurrency engine 206 further supports the use of monitor objects. A thread identifier identifying an owner of a monitor object is held in <i>val1.</i> In some embodiments, a value of -1 for <i>val1</i> indicates that the monitor object is free. Also, the concurrency engine 206 may use <i>val2</i> to encode operational variants, such as an ordering scheme for the wait queue. If the monitor object is occupied (<i>val1</i> not equal to -1), a thread trying to enter the monitor is placed in the wait queue. When the monitor object becomes free, the concurrency engine 206 selects the next thread in the queue to enter the monitor object based on the thread ordering scheme for the queue. Table 4 gives example operation types for a monitor object and their associated semantics. <tables id="tabl0004" num="0004"><table frame="all"><title>Table 4</title><tgroup cols="3" colsep="1" rowsep="1"><colspec colnum="1" colname="col1" colwidth="52.50mm" /><colspec colnum="2" colname="col2" colwidth="52.50mm" /><colspec colnum="3" colname="col3" colwidth="52.50mm" /><thead valign="top"><row><entry namest="col1" nameend="col1" align="center"><u>Operation Type (encoding of bits 2:4)</u></entry><entry namest="col2" nameend="col2" align="center"><u>Associated Action on Read</u></entry><entry namest="col3" nameend="col3" align="center"><u>Associated Action on Write of Value N</u></entry></row></thead><tbody valign="top"><row><entry namest="col1" nameend="col1" align="center">1</entry><entry namest="col2" nameend="col2" align="center">Enter the monitor. This will stall the caller until the monitor is free, and then enter the monitor.</entry><entry namest="col3" nameend="col3" align="center">Exit the monitor.</entry></row><row><entry namest="col1" nameend="col1" align="center">2</entry><entry namest="col2" nameend="col2" align="center">Try to enter the monitor. This will return 1 if the entry was successful and 0 if the entry failed. This call is non-blocking.</entry><entry namest="col3" nameend="col3" align="center">Exit the monitor.</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="center">3</entry><entry namest="col2" nameend="col2" align="center">Check the ownership of the monitor. This will return 1 if the caller is already in the monitor and 0 otherwise.</entry><entry namest="col3" nameend="col3" align="center">Exit the monitor.</entry></row></tbody></tgroup></table></tables>

Some embodiments the concurrency engine 206 may be capable of dynamically increasing the priority P<sub>o</sub> of the thread currently owning the monitor to P<sub>emax</sub>, where P<sub>emax</sub> is the maximum priority of threads on the monitor entry list. The priority of the owner thread would be returned to the original value once it releases the monitor. This mechanism is a hardware solution to the priority inversion problem associated with monitors.

The concurrency engine 206 also supports the use of condition objects. Conditions are associated with a monitor object. The index of a monitor object is held in <i>val1.</i> Conditions also have an associated wait queue. In some embodiments, the concurrency engine 206 may use <i>val2</i> to encode operational variants, such as an ordering scheme for the wait queue. When a thread suspends execution by waiting on a condition, the associated monitor data structures are updated to indicate the caller thread is no longer using the monitor object. If other threads are waiting to enter the associated monitor, one is selected using the thread-ordering scheme in effect for that monitor, and execution is resumed for that thread. The monitor data structures are updated to indicate this thread is now the owner of the monitor. The concurrency engine 206 may choose to deliver a memory access exception to the calling thread if it tries to perform a wait or signal operation on a condition and that thread is not the current owner of the associated monitor. Table 5 gives example operation types for a condition object and their associated semantics. <tables id="tabl0005" num="0005"><table frame="all"><title>Table 5</title><tgroup cols="3" colsep="1" rowsep="1"><colspec colnum="1" colname="col1" colwidth="52.50mm" /><colspec colnum="2" colname="col2" colwidth="52.50mm" /><colspec colnum="3" colname="col3" colwidth="52.50mm" /><thead valign="top"><row><entry namest="col1" nameend="col1" align="center"><u>Operation Type (encoding of bits 2:4)</u></entry><entry namest="col2" nameend="col2" align="center"><u>Associated Action on Read</u></entry><entry namest="col3" nameend="col3" align="center"><u>Associated Action on Write of Value N</u></entry></row></thead><tbody valign="top"><row><entry namest="col1" nameend="col1" align="center">1</entry><entry namest="col2" nameend="col2" align="center">Wait on the condition. Suspend the caller. Put the caller on the wait queue associated with the condition. If one or more threads are on the monitor wait queue, select one using the ordering mechanism in effect for the monitor. Resume execution of the selected thread, and update concurrency engine state to reflect the new monitor owner.</entry><entry namest="col3" nameend="col3" align="center">Signal the condition. If the wait queue associated with the condition is empty, this has no effect. Otherwise, the caller is put in the associated monitor's wait queue and suspended. A thread is taken off the condition wait queue and marked as owning the monitor, and execution resumes for this thread.</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="center">2</entry><entry namest="col2" nameend="col2" align="center">Same as read with OP equal to 1.</entry><entry namest="col3" nameend="col3" align="center">Signal N operation. Same as write with OP of 1, with the following exceptions. If N is 0, this is a no-operation. If N is 1, this is the same as write operation with OP of 1. If N > 0, up to N-1 threads are taken from the wait queue and placed in the monitor wait queue. If N is -1, all remaining threads on the condition wait queue are placed in the monitor wait queue.</entry></row></tbody></tgroup></table></tables>

The concurrency engine 206 further supports the use of deferred condition objects. Deferred conditions are associated with a monitor object. The index of the monitor object is held in <i>val1.</i> Deferred conditions also have an associated wait queue. In some embodiments, the concurrency engine 206 may use <i>val2</i> to encode operational variants, such as an ordering scheme for the wait queue. A deferred condition is similar to a normal condition except that the signal operation does not immediately remove the caller from the monitor object. Instead, the deferred condition puts any signaled threads in the monitor wait queue. The concurrency engine 206 may deliver a memory access exception to a calling thread if it tries to perform a wait or signal operation on a condition and it is not the current owner of the associated monitor object. Table 6 gives example operation types for a deferred condition object and their associated semantics. <tables id="tabl0006" num="0006"><table frame="all"><title>Table 6</title><tgroup cols="3" colsep="1" rowsep="1"><colspec colnum="1" colname="col1" colwidth="52.50mm" /><colspec colnum="2" colname="col2" colwidth="52.50mm" /><colspec colnum="3" colname="col3" colwidth="52.50mm" /><thead valign="top"><row><entry namest="col1" nameend="col1" align="center"><u>Operation Type (encoding of bits 2:4)</u></entry><entry namest="col2" nameend="col2" align="center"><u>Associated Action on Read</u></entry><entry namest="col3" nameend="col3" align="center"><u>Associated Action on Write of Value N</u></entry></row></thead><tbody valign="top"><row><entry namest="col1" nameend="col1" align="center">1</entry><entry namest="col2" nameend="col2" align="center">Wait on the condition. Suspend the caller. Put the caller in the wait queue associated with the condition. Resume execution of any thread trying to enter the associated monitor.</entry><entry namest="col3" nameend="col3" align="center">Signal the condition. If the wait queue associated with the condition is empty, this has no effect. Otherwise, a thread is taken off the condition wait queue and placed in the monitor wait queue.</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="center">2</entry><entry namest="col2" nameend="col2" align="center">Same as read with OP equal to 1.</entry><entry namest="col3" nameend="col3" align="center">Signal N operation. Same as write with OP of 1, with the following exceptions. If N is 0, this is a no-operation. If N > 0, up to N threads are taken from the wait queue and placed in the monitor wait queue. If N is -1, all remaining threads on the condition wait queue are placed in the monitor wait queue.</entry></row></tbody></tgroup></table></tables>

Beyond that, the concurrency engine 206 supports the use of timeout condition objects. Timeout conditions are associated with a monitor object. The index of the monitor object is held in <i>val1.</i> Timeout conditions also have an associated wait queue. In some embodiments, the concurrency engine 206 may use <i>val2</i> to encode operational variants, such as an ordering scheme for the wait queue. When a thread suspends execution by waiting on a condition, the associated monitor data structures are updated to indicate the caller thread is no longer using the monitor object. If other threads are waiting to enter the associated monitor, one is selected using the thread-ordering scheme in effect for that monitor object, and execution is resumed for the selected thread. The monitor data structures are updated to indicate the selected thread is now the owner of the monitor object. A timeout condition is the same as a deferred condition except that a numerical timeout value of N is given when a thread waits on a condition. If the thread is not signaled at the end of N time units, the concurrency engine 206 automatically allows the thread to use the monitor object if the monitor object is empty. Otherwise, the thread is placed in the monitor wait queue. The concurrency engine 206 may choose to deliver a memory access exception to a calling thread if it tries to perform a wait or signal operation on a condition and it is not the current owner of the associated monitor. Table 7 gives example operation types for a timeout condition object and their associated semantics. <tables id="tabl0007" num="0007"><table frame="all"><title>Table 7</title><tgroup cols="3" colsep="1" rowsep="1"><colspec colnum="1" colname="col1" colwidth="52.50mm" /><colspec colnum="2" colname="col2" colwidth="52.50mm" /><colspec colnum="3" colname="col3" colwidth="52.50mm" /><thead valign="top"><row><entry namest="col1" nameend="col1" align="center"><u>Operation Type (encoding of bits 2:4)</u></entry><entry namest="col2" nameend="col2" align="center"><u>Associated Action on Read</u></entry><entry namest="col3" nameend="col3" align="center"><u>Associated Action on Write of Value N</u></entry></row></thead><tbody valign="top"><row><entry namest="col1" nameend="col1" align="center">1</entry><entry namest="col2" nameend="col2" align="center">No operation.</entry><entry namest="col3" nameend="col3" align="center">Wait on the condition. Suspend the caller. Put the caller in the wait queue associated with the condition. Associate a timeout value N with the caller. Resume execution of any thread trying to enter the associated monitor.</entry></row><row><entry namest="col1" nameend="col1" align="center">2</entry><entry namest="col2" nameend="col2" align="center">No operation.</entry><entry namest="col3" nameend="col3" align="center">Signal the condition. If the wait queue associated with the condition is empty, this has no effect. Otherwise, a thread is taken off the condition wait queue and placed in the monitor wait queue.</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="center">3</entry><entry namest="col2" nameend="col2" align="center">No operation.</entry><entry namest="col3" nameend="col3" align="center">Signal N operation. Same as write with OP of 1, with the following exceptions. If N is 0, this is a no-operation. If N > 1, up to N threads are taken from the wait queue and placed in the monitor wait queue. If N is -1, all remaining threads on the condition wait queue are placed in the monitor wait queue.</entry></row></tbody></tgroup></table></tables>

In addition, the concurrency engine 206 supports the use of priority queue objects. The number of items in the queue is held in <i>val1,</i> which may be interpreted as a 32-bit unsigned integer. A wait queue is associated with each priority queue object and is used for threads waiting to obtain values from the queue. In some embodiments, the concurrency engine 206 may use <i>val2</i> to encode operational variants, such as an ordering scheme for the wait queue.

While the encoding of entries in a priority queue may be implementation specific, the following serves as an example. Entries in the priority queue could represent 32-bit values, where the eight bits in locations 0:7 may encode a priority value, the eleven bits in locations 8:18 may encode a numerical value N, and the thirteen bits in locations 19:31 may indicate a work identifier. A priority queue containing 32-bit values may be ordered based on the encoded priority value with higher priority values at the head of the queue. When a thread performs a read operation to obtain the highest priority entry in the queue, the 32-bit value at the head of the queue is returned to the caller, and the N field is decremented. When N reaches zero, the entry value is removed or popped from the head of the queue. The work identifier is available to an application for encoding work descriptors or some other application-specific purposes.

When threads are on the wait queue associated with the priority queue and the priority queue is non-empty, a decision is made as to what thread to wake up to perform the work. In some embodiments, the concurrency engine 206 uses a FIFO queue or other type of queue. In other embodiments, the concurrency engine 206 selects a thread running on a processor that is the least lightly loaded or that has a performance suitable for the work priority. The concurrency engine 206 could also use an application-specific ordering scheme. Table 8 gives example operation types for a priority queue object and their associated semantics. <tables id="tabl0008" num="0008"><table frame="all"><title>Table 8</title><tgroup cols="3" colsep="1" rowsep="1"><colspec colnum="1" colname="col1" colwidth="52.50mm" /><colspec colnum="2" colname="col2" colwidth="52.50mm" /><colspec colnum="3" colname="col3" colwidth="52.50mm" /><thead valign="top"><row><entry namest="col1" nameend="col1" align="center"><u>Operation Type (encoding of bits 2:4)</u></entry><entry namest="col2" nameend="col2" align="center"><u>Associated Action on Read</u></entry><entry namest="col3" nameend="col3" align="center"><u>Associated Action on Write of Value N</u></entry></row></thead><tbody valign="top"><row rowsep="1"><entry namest="col1" nameend="col1" align="center">1</entry><entry namest="col2" nameend="col2" align="center">Queue read. Acquire a 32-bit value from the priority queue. Encoding of value obtained is as described above. Stall the caller if no items are currently in the queue.</entry><entry namest="col3" nameend="col3" align="center">Queue push. Insert a 32-bit value in the priority queue. Encode as described above. Wake up any threads stalled in a queue pop operation.</entry></row></tbody></tgroup></table></tables>

Furthermore, the concurrency engine 206 could also maintain thread descriptors, also referred to as "fork descriptors." Thread descriptors could include, but are not limited to, a stack pointer, a stack size, a pointer to a function that the thread is to execute initially, arguments to the function, the number N of threads that have been requested to execute the function, an index I (0 <= I < N) that corresponds to the position of the caller in the initial fork, the priority of initial execution, the status of children threads, return values of children threads, the number of children threads, and thread descriptor identifiers or process identifiers (PID) of the caller's parent.

Various ways of representing the thread descriptors in the concurrency engine 206 are possible. In some embodiments, a special address range for the thread descriptor for the caller thread (the "self" thread descriptor) and another address range for the thread descriptors for the caller's children (the child thread descriptor) are used. The various entries in the thread descriptors could then be placed at locations within these address ranges. An additional address range could map the descriptors for each thread or process identifier.

In addition, accesses to various locations in the thread descriptor address ranges of the concurrency engine 206 could have additional mechanisms to enforce the semantics of thread trees. For example, writing a control value to a location in a child thread descriptor could start, or fork, children threads. The children threads could be inserted in a parent/child thread tree, or they could be executed as orphans with no parent in the thread tree. The children threads may have a self thread descriptor that is a copy of the parent's child thread descriptor with a few updated fields (such as the index I, where 0 <= I < N, indicating the number of the thread in the original parent fork request). Writing a value V to a location in the self descriptor could indicate to the parent thread that the self thread has finished execution and return the status value V. Reading a location could have the effect of waiting for either one, some, or all children to terminate execution.

The concurrency engine 206 could support any of a wide variety of additional semantics in the processing system 100. For example, the concurrency engine 206 could enforce semantics when managing the concurrency primitives. The semantics could be associated with mutual exclusion, waiting and signaling conditions, timeout conditions, deferred conditions, priority inheritance in monitors to avoid priority inversion, priority queues, and operation ordering.

Using the various objects described above, the concurrency engine 206 allows the processors 102a-102n to execute multi-threaded applications more efficiently. In particular, the objects implement various primitives and allow the concurrency engine 206 to act as a hardware operating system. In some embodiments, the processors 102a-102n are arranged in an SMP configuration, and the concurrency engine 206 helps to greatly increase the SMP performance of the processing system 100.

The concurrency engine 206 represents any suitable hardware device(s) to implement at least some of the features described above. For example, the concurrency engine 206 could be implemented using a Sea of Gates structures, an ASIC structure, an FPGA structure, or other suitable hardware. One embodiment of the concurrency engine 206 is shown in FIGURE 3, which is described below.

Although FIGURE 2 illustrates additional details of one example of a processing system 100, various changes may be made to FIGURE 2. For example, the processing system 100 could include any number of processors 102a-102n. Also, the concurrency engine 206 could be coupled directly to the processors 102a-102n or implemented in one of the processors 102a-102n.

FIGURE 3 illustrates an example concurrency engine 206 for accelerating multi-processing in a processing system according to one embodiment of this disclosure. The embodiment of the concurrency engine 206 shown in FIGURE 3 is for illustration only. Other embodiments of the concurrency engine 206 may be used without departing from the scope of this disclosure. Also, for ease of explanation, the concurrency engine 206 is described as operating in the processor core 101 of FIGURE 2. The concurrency engine 206 may be used in any other suitable environment.

In the illustrated example, the concurrency engine 206 includes a message passing engine 302. The message passing engine 302 represents an interface to the network 108. For example, the message passing engine 302 allows the concurrency engine 206 to receive requests from the processors 102a-102n and to provide responses to the processors 102a-102n. The message passing engine 302 represents any suitable interface to the network 108. As an example, the message passing engine 302 may represent a read/write interface operating in slave mode.

A command decoder 304 and a response formatter 306 are coupled to the message passing engine 302. The command decoder 304 decodes requests received by the concurrency engine 206 so that requested operations may be performed by the concurrency engine 206. For example, the command decoder 304 may decode the object index, the object type, and the operation type for a requested concurrency operation. The command decoder 304 then provides this information for execution by the concurrency engine 206. The response formatter 306 receives the results from the execution of the operations and formats the results for communication to one or more of the processors 102a-102n. For example, the response formatter 306 may receive results from the execution of a concurrency operation, and the response formatter 306 may encode the results in any suitable format for communication over the network 108.

A request queue 308 and a response queue 310 are coupled to the command decoder 304 and the response formatter 306, respectively. The request queue 308 temporarily stores decoded requests until the requested operations may be performed. The response queue 310 temporarily stores the results of the executed operations until the results may be retrieved and encoded by the response formatter 306.

A command processor 312 is coupled to the request queue 308 and the response queue 310. The command processor 312 is capable of performing requested concurrency operations and providing the results from the execution. For example, the command processor 312 could update the values contained in timer objects, semaphore objects, queue objects, counter objects, monitor objects, and condition objects. The command processor 312 could also retrieve and provide the values contained in these objects. In addition, the command processor 312 could schedule the execution of the threads by the various processors 102a-102n in the processing system 100. As particular examples, the command processor 312 could execute the various functions described above with respect to FIGURE 2.

A memory 314 is coupled to the command processor 312. The memory 314 stores various information used by the command processor 312 to perform the concurrency operations. In this example, the memory 316 is subdivided into three different sections 316-320. A process information section 316 stores the thread descriptors for the various threads being executed by the processors 102a-102n as described above.

An object information section 318 stores information about the various concurrency objects used to implement the concurrency operations. For example, the concurrency objects could include the timer objects, semaphore objects, queue objects, counter objects, monitor objects, and condition objects described above. Each object may have a unique object identifier ("OID"), and the object information section 318 may contain an entry for each of these object identifiers. For each object identifier, the associated entry could include information such as the object type, one or more status values, and a pointer. The object type identifies whether the associated object is a counter, semaphore, timer, or other type of object. The status value(s) represent the values associated with the object, such as the <i>val1</i> and <i>val2</i> values described above. The pointer identifies the next object identifier entry in the object information section 318.

A queue information section 320 represents the various wait queues described above. These wait queues are used to queue requests for each of the concurrency objects. For example, each concurrency object may have an associated wait queue identifying the various threads that wish to access and use that concurrency object, and the queue information section 320 may contain an entry for the each object's wait queue. For each entry, the queue information section 320 includes queue information, a head pointer, and a tail pointer. The queue information identifies the type of queue (FIFO, etc.) and the size (number of entries) currently in the wait queue. The head and tail pointers point to the beginning and end of a linked list that identified the threads waiting on a particular concurrency object.

In some embodiments, all of the various components shown in FIGURE 3 may be implemented in hardware. For example, the components could be implemented using Sea of Gates structures. Also, the memory 314 may represent any suitable amount of memory. The amount of memory 314 could, for example, be based on the expected number of process identifiers, the expected number of object identifiers, and the size of the fork descriptors in a particular processing system 100.

Although FIGURE 3 illustrates one example of a concurrency engine 206 for accelerating multi-processing in a processing system, various changes may be made to FIGURE 3. For example, while FIGURE 3 illustrates the memory 314 as a single memory structure, the memory 314 could also represent any number of separate memories. Also, the functional division of FIGURE 3 is for illustration only. Various components in FIGURE 3 may be combined or omitted and additional components may be added according to particular needs.

FIGURES 4A and 4B illustrate example scheduling in a multi-threaded processor according to one embodiment of this disclosure. In particular, FIGURE 4A illustrates a multi-threaded processor 400 that interleaves threads on a cycle-by-cycle basis, and FIGURE 4B illustrates the multi-threaded processor 400 interleaving threads in blocks. The multi-threaded processor 400 could, for example, be used as at least one of the processors 102a-102n in the processing system 100 of FIGURE 1. The multi-threaded processor shown in FIGURES 4A and 4B is for illustration only. Other embodiments of the multi-threaded processor may be used without departing from the scope of this disclosure.

As shown in FIGURE 4A, the multi-threaded processor 400 includes multiple threads 402a-402d. The threads 402a-402d represent portions of one or more applications that are capable of being executed at the same time by the multi-threaded processor 400. The states for threads 402a-402d shown in FIGURE 4A could be stored in any suitable medium, such as a random access memory 404. In this example, the multi-threaded processor 400 includes four threads, although any suitable number of threads may be used.

In this embodiment, the multi-threaded processor 400 represents a pipelined processor that includes a pipeline having multiple stages 406a-406n. Collectively, the stages 406a-406n of the processor 400 are capable of performing the required functions to execute instructions that form the threads 402a-402d. Individually, each of the stages 406a-406n performs one of the functions needed to execute the instructions. For example, the stages 406a-406n may include an address generation stage, an instruction fetch stage, a decode stage, a read stage, an execution stage, and a write or write-back stage.

In addition, the processor 400 may have an instruction cache ("I-CACHE") 408, a data cache ("D-CACHE") 410, and a register set ("REG SET") 412. The instruction cache 408 stores program instructions for execution by the stages 406a-406n of the processor 400. The data cache 410 stores data operands used during execution of the instruction by the stages 406a-406n of the processor 400. The register set 412 stores data operands and other data used during execution of the instruction by the stages 406a-406n of the processor 400. In some embodiments, each of the threads 402a-402d in the processor 400 has a dedicated register set 412 and a dedicated instruction pointer into the instruction cache 408.

The address generation stage generates the address of the next instruction to be fetched from the instruction cache 408. The instruction fetch stage fetches an instruction for execution from the instruction cache 408 and stores the fetched instruction in an instruction buffer. The decode stage takes the instruction from the instruction buffer and decodes the instruction into a set of signals that can be directly used for executing subsequent pipeline stages. The read stage fetches required operands from the data cache 410 or from registers in the register set 412. The execution stage performs the actual program operation (such as add, multiply, or divide) using the fetched operands and generates a result. The write stage then writes the result generated by the execution stage back into the data cache 410 or the register set 412.

Assuming that each of the stages 406a-406n completes its operation in one clock cycle, an N-stage processor pipeline takes N clock cycles to process one instruction. As previously described, once the pipeline is full, an instruction can theoretically be completed every clock cycle.

In the illustrated example, each of the stages 406a-406n is shown as containing one of the instructions from one of the threads 402a-402d. This technique is used to identify the thread currently being executed by each of the processor stages 406a-406n. For example, the first stage 406a is currently executing an instruction from the first thread 402a, and the second stage 406b is currently executing an instruction from the fourth thread 402d.

The processor 400 or the concurrency engine 206 may be responsible for scheduling the execution of the threads 402a-402d by the stages 406a-406n of the processor. In other words, the processor 400 or the concurrency engine 206 determines which thread provides an instruction to the first stage 406a of the processor. If the scheduling of the threads 402a-402d is to be controlled by the processor 400, the processor 400 may include a scheduler 414. The scheduler 414 is responsible for scheduling the execution of instructions from the various threads 402a-402d. The scheduler 414 represents any suitable structure for scheduling the execution of instructions from the threads 402a-402d. For example, the scheduler 414 could represent hardware that identifies a priority for each thread and selects the next thread to be executed based on their priorities.

In FIGURE 4A, the threads 402a-402d are interleaved on a cycle-by-cycle basis. This means that the processor 400 or the concurrency engine 206 selects a thread for execution during each clock cycle. This means that a different thread could be scheduled for execution each clock cycle. However, the same thread could be scheduled for more than one consecutive clock cycle, depending on the circumstances (such as the priority values of the threads).

Each time a stage in the processor 400 switches between two of the threads 402a-402d, a hardware or software context switch occurs. A context switch may cause a processor stage to write data for one thread to one register set 412 and read data for the next thread from another register set 412.

FIGURE 4B illustrates a different interleaving technique for scheduling execution of the threads 402a-402d. In this example, instructions from only one of the threads 402a-402d are provided to the first stage 406a until an event that may cause latency occurs. For example, an instruction that requires a read from or write to external memory typically has an associated delay or latency. When an event causing latency is detected, a context switch is triggered, and instructions from another thread are provided to the first stage 406a. In this example, the third thread 402c was being executed by the processor 400 until an operation with latency was detected. At that point, a context switch occurred, and instructions from the first thread 402a began execution. While FIGURE 4B illustrates a gap between execution of the third and first threads, the processor 400 could detect operations involving latency and perform context switches without requiring a gap in the execution of instructions.

In both examples shown in FIGURES 4A and 4B, the scheduler 414 in the processor 400 or the concurrency engine 206 needs to select a thread for execution (either each clock cycle or when latency is detected). The scheduler 414 or the concurrency engine 206 could use any suitable technique to select a thread for execution. For example, a round robin scheduling approach or a FIFO scheduling approach could be used.

The scheduler 414 or the concurrency engine 206 could also use a static priority scheme where each thread has an associated static priority. For example, each of the threads 402a-402d could have a register in its associated register set 412 that encodes the thread's execution priority. The scheduler 414 or the concurrency engine 206 could use these values to schedule execution of the threads. In particular embodiments, the thread with the highest priority is scheduled first. If multiple threads have the same priority, the threads with the highest priority are scheduled in a round-robin fashion. Threads with lower priorities are not executed if there are currently threads with higher priorities awaiting execution.

In addition, the scheduler 414 or the concurrency engine 206 could use a dynamic priority scheme where each thread has an associated dynamic priority. For example, if the network 108 is becoming congested, the scheduler 414 or the concurrency engine 206 could reduce the priority of threads that are generating large amounts of network traffic. The dynamic priority for each thread could be stored in that thread's register set 412. In some embodiments, the dynamic priority could change on a cycle-by-cycle basis, although other embodiments could be used.

As a particular example, a dynamically-calculated priority could be defined using the formula:<maths id="math0001" num="(1)"><math display="block"><mrow><msub><mrow><mtext mathvariant="italic">P</mtext></mrow><mrow><mtext mathvariant="italic">t</mtext></mrow></msub><mtext> = </mtext><msub><mrow><mtext mathvariant="italic">P</mtext></mrow><mrow><mtext mathvariant="italic">t</mtext></mrow></msub><msub><mrow><mtext></mtext></mrow><mrow><mtext>0</mtext></mrow></msub><mtext> + </mtext><mtext mathvariant="italic">K</mtext><mtext> · </mtext><mfrac><mrow><msub><mrow><mtext mathvariant="italic">W</mtext></mrow><mrow><mtext mathvariant="italic">td</mtext></mrow></msub></mrow><mrow><mtext mathvariant="italic">D</mtext></mrow></mfrac><mtext> ,</mtext></mrow></math><img file="EP1544737A1_D0001.tif" /></maths> where P<sub>t</sub> represents the dynamic priority for a particular thread, P<sub>t0</sub> represents a static component of the priority, K represents a constant, W<sub>td</sub> represents a dynamic measure of the work to be done for that thread, and D represents a measure of the amount of time from a deadline for the thread. The "dynamic measure of the work to be done" could represent an estimate of how many machine instructions are needed to complete the thread. For example, the estimated number of instructions for a thread could be stored in one register of the thread's register set 412, and the value in another register could be incremented each time an instruction from the thread is issued into the first stage 406a of the processor 400. The value of W<sub>td</sub> could be calculated as a difference between these values. The "measure of the amount of time from a deadline" may represent the number of clock cycles before the thread's deadline. For example, the value of a system clock when the thread must be completed could be stored in one register, and the value of D could be calculated as a difference between this value and the current value of the system clock.

In some embodiments, both the scheduler 414 and the concurrency engine 206 are responsible for scheduling execution of the threads 402a-402d. For example, the concurrency engine 206 may use information (such as a state machine) to improve or optimize one or more operating metrics of the processors 102a-102n. As particular examples, the concurrency engine 206 could improve or optimize power usage, real time response, or quality of service metrics of the processors 102a-102n. The concurrency engine 206 uses various inputs to perform this optimization. Example inputs include information identifying the number of threads active for each processor, a deadline for each thread, the estimated number of the instructions needed to complete each thread, the instructions executed by each thread, the priority (such as a statically allocated or dynamic priority) for each thread, a power consumption for each processor, and an amount of utilization of the network 108 for each processor. As a particular example, the scheduler 414 could use the following equation to calculate the dynamic priority for each thread:<maths id="math0002" num="(2)"><math display="block"><mrow><msub><mrow><mtext mathvariant="italic">P</mtext></mrow><mrow><mtext mathvariant="italic">t</mtext></mrow></msub><mtext> = </mtext><msub><mrow><mtext mathvariant="italic">P</mtext></mrow><mrow><mtext mathvariant="italic">t</mtext></mrow></msub><msub><mrow><mtext></mtext></mrow><mrow><mtext>0</mtext></mrow></msub><mtext> + </mtext><mtext mathvariant="italic">K</mtext><mtext> · </mtext><mfrac><mrow><msub><mrow><mtext mathvariant="italic">W</mtext></mrow><mrow><mtext mathvariant="italic">td</mtext></mrow></msub><mtext></mtext></mrow><mrow><mtext mathvariant="italic">D</mtext></mrow></mfrac><mtext> + </mtext><msub><mrow><mtext mathvariant="italic">Y</mtext></mrow><mrow><mtext mathvariant="italic">t</mtext></mrow></msub><mtext>,</mtext></mrow></math><img file="EP1544737A1_D0002.tif" /></maths> where Y<sub>t</sub> is an additional term injected or controlled by the concurrency engine 206. The value of Y<sub>t</sub> could vary from thread to thread or could be held constant, depending on the circumstances. By using the Y<sub>t</sub> term in Equation (2), the concurrency engine 206 may adjust the priority of one or more of the threads 402a-402d in processors 102a-102n as needed. For example, the Y<sub>t</sub> term could be changed by the concurrency engine 206 to properly enforce concurrency semantics, such as temporary elevation of priority when ownership of a monitor is obtained in order to avoid priority inversion as described above. The concurrency engine 206 could also establish value(s) for Y<sub>t</sub> to increase or optimize system performance, such as to optimize power consumption or quality of service.

This represents one way that the concurrency engine 206 and the scheduler 414 could participate in jointly scheduling execution of the threads 402a-402d. Other techniques could also be used. For example, the scheduler 414 could use Equation (1) above to schedule execution of the threads, and the concurrency engine 206 could simply disable execution of particular threads or all threads on one or more processors as needed.

A dynamic priority mechanism may allow the processing system 100 to adapt at a very fine grain to unforeseen jitter and delays in executing the threads 402a-402d. Also, it may allow the threads 402a-402d to meet real-time deadlines. In addition, this may allow system designers to reduce system headroom to allow for jitter and unexpected overheads, which may help to reduce device costs and power consumption while increasing performance.

While Equations (1) and (2) have been described above as being used to schedule execution of threads 402a-402d in a processor 400, the concurrency engine 206 could use the same equation(s) to manage the wait queues of the concurrency objects. For example, if multiple threads are stalled and listed in a semaphore object's wait queue, the concurrency engine 206 could use Equation (1) above to select which thread resumes execution first.

Although FIGURES 4A and 4B illustrate two examples of scheduling in a multi-threaded processor 400, various changes may be made to FIGURES 4A and 4B. For example, the processor 400 could include any number of threads and any number of pipeline stages. In particular, the processor 400 could have only one thread, representing a conventional single-threaded processor. Also, the scheduling techniques could be used in any other suitable processor. In addition, the processor 400 could use any other scheduling technique.

FIGURE 5 illustrates an example method 500 for accelerating multi-processing according to one embodiment of this disclosure. For ease of explanation, the method 500 is described with respect to the processor core 101 of FIGURE 2 operating in the processing system 100 of FIGURE 1. The method 500 could be used with any other suitable processor core and in any other suitable system.

One or more threads of an application are executed on a first processor at step 502. This may include, for example, providing portions of a software program to one of the processors 102a-102n for execution. The software program could represent any suitable multi-threaded program implementing any suitable functionality.

One or more threads of the application are executed on a second processor at step 504. This may include, for example, providing different portions of the software program to another of the processors 102a-102n for execution. The threads executed by the second processor potentially could execute in parallel with the threads executed by the first processor.

The execution of the threads by the first and second processors is controlled using a concurrency engine 206 at step 506. This may include, for example, the concurrency engine 206 stalling one of the threads being executed while another of the threads is being executed. In some embodiments, the concurrency engine 206 uses structures such as monitors, conditions, semaphores, queues, and counters to control the execution of the threads. The concurrency engine 206 also controls the suspension and resumption of the executing threads, such as by delaying read or write acknowledgements, using interrupts or exceptions, or directly accessing the logic of the processors 102a-102n such as by using the processor schedulers 414 to enable or disable processor execution or to increase or decrease power or clock speed.

Although FIGURE 5 illustrates one example of a method 500 for accelerating multi-processing, various changes may be made to FIGURE 5. For example, the various steps shown in FIGURE 5 could be performed in parallel in the processing system 100.

FIGURE 6 illustrates an example method 600 for thread scheduling in a multi-processing system according to one embodiment of this disclosure. For ease of explanation, the method 600 is described with respect to the multi-threaded processor 400 of FIGURE 4 operating in the processing system 100 of FIGURE 1. The method 600 could be used with any other suitable processor and in any other suitable system.

The multi-threaded processor 400 identifies a priority for multiple threads at step 602. This may include, for example, the scheduler 414 accessing a register for each thread and identifying a static priority contained in each register. This may also include the scheduler 414 calculating a dynamic priority for each of the threads. The dynamic priority calculated by the scheduler 414 for each thread may or may not be based on input from the concurrency engine 206. The input from the concurrency engine 206 may cause one or more threads to have a higher priority or lower priority.

The multi-threaded processor 400 identifies the thread or threads with the highest priority at step 604. This may include, for example, the scheduler 414 comparing the numerical values of the priorities and selecting the thread(s) with the highest numerical priority value.

If there are multiple threads with the highest priority, the multi-threaded processor 400 selects one of the threads at step 606. This may include, for example, the scheduler 414 using a round robin technique to select one of multiple threads having the same priority.

The multi-threaded processor 400 executes the selected thread at step 608. This may include, for example, issuing an instruction from the selected thread to the first stage 406a of the processor 400.

Although FIGURE 6 illustrates one example of a method 600 for thread scheduling in a multi-processing system, various changes may be made to FIGURE 6. For example, the scheduler 414 could perform scheduling using a round robin technique or other technique that does not involve using priorities associated with the threads.

FIGURE 7 illustrates an example apparatus 700 using a processing system that accelerates multi-processing according to one embodiment of this disclosure. The apparatus 700 shown in FIGURE 7 represents one operational environment in which the processing system 100 may be used. The processing system 100 may be used in any other suitable environment without departing from the scope of this disclosure.

As shown in FIGURE 7, the processing system 100 is coupled to one or more storage devices 702, one or more input devices and/or input device interfaces 704, and one or more output devices and/or output device interfaces 706. The storage devices 702 represent any suitable mechanism or mechanisms for storing and facilitating retrieval of information. For example, the storage devices 702 could represent read only memories, random access memories, hard disk drives, CD drives, DVD drives, and any other or additional storage devices.

The input devices and/or interfaces 704 represent any suitable input device(s) and/or interface(s) to input devices. For example, the input devices/interfaces 704 could represent a keyboard, mouse, light pen, and/or other input device or interface to such a device.

The output devices and/or interfaces 706 represent any suitable output device(s) and/or interface(s) to output devices. For example, the output devices/interfaces 706 could represent a monitor, printer, scanner, and/or other output device or interface to such a device.

In this example, the apparatus 700 may be capable of transmitting and/or receiving wireless signals. To support this functionality, the apparatus 700 may optionally include an antenna 708 and wireless interface circuitry 710. The antenna 708 represents any suitable structure capable of transmitting and/or receiving wireless signals, such as a radio frequency ("RF") antenna. The wireless interface circuitry 710 represents any suitable circuitry capable of facilitating communication using wireless signals. For example, the wireless interface circuitry 710 could modulate data for transmission over RF signals and demodulate received RF signals to extract data for processing. The wireless interface circuitry 710 could, for example, represent an RF transceiver.

The apparatus 700 of FIGURE 7 could represent any suitable device or system that is capable of using the processing system 100. For example, the apparatus 700 could represent a mobile telephone. In this example, the antenna 708 and wireless interface 710 are used to facilitate communication between the mobile telephone and a base station or other source of wireless signals. The input devices/interfaces 704 could represent a keypad on the mobile telephone and a microphone capable of captured audible sounds. The output devices/interfaces 706 could represent a display on the mobile telephone and a speaker capable of producing audible sounds. In addition, the storage devices 702 could represent memory and/or a smart card capable of storing information such as telephone numbers, a call log, and any other or additional information.

As another example, the apparatus 700 could represent a computing device, such as a networking device. In this example, the input devices/interfaces 704 and the output devices/interfaces 706 could represent ports that couple the apparatus 700 to one or more networks. The antenna 708 and wireless interface 710 could be omitted in this example. The apparatus 700 could implement any of a wide variety of functions, such as packet forwarding and traffic management in a network.

Although FIGURE 7 illustrates one example of an apparatus 700 using a processing system that accelerates multi-processing, various changes may be made to FIGURE 7. For example, the processing system 100 could be used in any other suitable device or system.

It may be advantageous to set forth definitions of certain words and phrases used in this patent document. The terms "include" and "comprise," as well as derivatives thereof, mean inclusion without limitation. The term "or" is inclusive, meaning and/or. The phrases "associated with" and "associated therewith," as well as derivatives thereof, may mean to include, be included within, interconnect with, contain, be contained within, connect to or with, couple to or with, be communicable with, cooperate with, interleave, juxtapose, be proximate to, be bound to or with, have, have a property of, or the like. The term "controller" means any device, system, or part thereof that controls at least one operation. A controller may be implemented in hardware, firmware, or software, or a combination of at least two of the same. It should be noted that the functionality associated with any particular controller may be centralized or distributed, whether locally or remotely.

While this disclosure has described certain embodiments and generally associated methods, alterations and permutations of these embodiments and methods will be apparent to those skilled in the art. Accordingly, the above description of example embodiments does not define or constrain this disclosure. Other changes, substitutions, and alterations are also possible without departing from the spirit and scope of this disclosure, as defined by the following claims.

Contents6

Every citation, both waysCites: the store holds 6 of 7

Citations
Document	Relation	Office	Category	Cited during	Relevant claims
GB2517266B	Cited by	United Kingdom	–	Search report	–
EP2431875A1	Cited by	European Patent Office (EPO)	–	Search report	–
EP3093686A1	Cited by	European Patent Office (EPO)	–	Search report	–
US9535700B2	Cited by	United States of America	–	Applicant	–
FR2960989A1	Cited by	France	–	Search report	–
EP2431875A1	Cited by	European Patent Office (EPO)	–	Search report	–
EP2551776A4	Cited by	European Patent Office (EPO)	–	Search report	–
US9141429B2	Cited by	United States of America	–	Applicant	–
GB2517266A	Cited by	United Kingdom	–	Search report	–
US10634783B2	Cited by	United States of America	–	Applicant	–
EP0683451A2	Cites	European Patent Office (EPO)	Y	Search report	3,19,22
US2001056456A1	Cites	United States of America	A	Search report	10,16,18,23,24
US2001056456A1	Cites	United States of America	A	Search report	10,16,18,23,24
US4796178A	Cites	United States of America	XY	Search report	1,2,4-18,20,21,23-26
US5202987A	Cites	United States of America	A	Search report	1-26
US5202987A	Cites	United States of America	A	Search report	1-26
KREUZINGER J ET AL: "Real-time event-handling and scheduling on a multithreaded Java microcontroller", MICROPROCESSORS AND MICROSYSTEMS, IPC BUSINESS PRESS LTD. LONDON, GB, vol. 27, no. 1, 1 February 2003 (2003-02-01), pages 19 - 31, XP004404896, ISSN: 0141-9331	Non-patent	–	–	Search report	–

10 members in 3 offices

This recordMember, by publication datePriority claim

Priority claims15

Priority claims
Document	Office	Kind	Date
531502P	United States of America	–	2003-12-19
53150203	United States of America	P	2003-12-19
53150203	United States of America	P	2003-12-19
531538P	United States of America	–	2003-12-19
53153803	United States of America	P	2003-12-19
53153803	United States of America	P	2003-12-19
967013	United States of America	–	2004-10-15
96701304	United States of America	A	2004-10-15
96701304	United States of America	A	2004-10-15
531502P	–	–	–
531538P	–	–	–
967013	–	–	–
US20030531502P	–	–	–
US20030531538P	–	–	–
US20040967013	–	–	–

Members10

Family members, oldest first
Document	Office	Kind
EP1544737A1This record	European Patent Office (EPO)	A1	2005-06-22
EP1544738A1	European Patent Office (EPO)	A1	2005-06-22
US2005149936A1	United States of America	A1	2005-07-07
US2005149937A1	United States of America	A1	2005-07-07
EP1544738B1	European Patent Office (EPO)	B1	2008-07-16
DE602004015052D1	Germany	D1	2008-08-28
EP1544737B1	European Patent Office (EPO)	B1	2010-05-19
DE602004027207D1	Germany	D1	2010-07-01
US7802255B2	United States of America	B2	2010-09-21
US8566828B2	United States of America	B2	2013-10-22

28 legal events, as 4 offices reported them to INPADOC

Over the term

Point at a mark for the event

Events

Legal events, newest first
	Event	Code	Office
2025-01-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2025-01-17	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2025-01-08	Patent expired after termination of 20 yearsExpiredPE20	PE20	GB
2024-01-31	Annual fee paid to national office [announced via postgrant information from national office to epo]GrantedPGFP	PGFP	EP
2024-01-18	Annual fee paid to national office [announced via postgrant information from national office to epo]GrantedPGFP	PGFP	EP
2017-11-21	Fee paymentPLFP	PLFP	FR
2016-11-21	Fee paymentPLFP	PLFP	FR
2015-11-23	Fee paymentPLFP	PLFP	FR
2011-11-30	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2011-11-24	Application deemed withdrawn, or ip right lapsed, due to non-payment of renewal feeWithdrawnR119	R119	DE
2011-06-09	No opposition filed against granted patent, or epo opposition proceedings concluded without decisionGrantedR097	R097	DE
2011-04-27	No opposition filedOpposition26N	26N	EP
2011-03-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2011-03-25	No opposition filed within time limitOppositionORIGINAL CODE: 0009261PLBE	PLBE	EP
2011-03-25	Information on the status of an ep patent application or granted ep patentGrantedSTATUS: NO OPPOSITION FILED WITHIN TIME LIMITSTAA	STAA	EP
2010-07-01	Corresponds to:REF	REF	EP
2010-05-19	Designated contracting statesAK	AK	EP
2010-05-19	European patent grantedGrantedFG4D	FG4D	GB
2010-04-16	(expected) grantORIGINAL CODE: 0009210GRAA	GRAA	EP
2010-04-09	Grant fee paidORIGINAL CODE: EPIDOSNIGR3GRAS	GRAS	EP
2009-12-23	Despatch of communication of intention to grant a patentORIGINAL CODE: EPIDOSNIGR1GRAP	GRAP	EP
2007-07-11	First examination report despatched17Q	17Q	EP
2006-08-30	First examination report despatched17Q	17Q	EP
2006-03-15	Designation fees paidAKX	AKX	EP
2006-02-08	Request for examination filed17P	17P	EP
2005-06-22	Designated contracting statesAK	AK	EP
2005-06-22	Request for extension of the european patentAX	AX	EP
2005-05-06	Public reference made under article 153(3) epc to a published international application that has entered the european phaseORIGINAL CODE: 0009012PUAI	PUAI	EP

Thread execution scheduler for multi-processing system and method

Abstract