EP2247040A2

Buffered crossbar switch system

Abstract

Described embodiments provide for transfer of data between data modules. At least two crossbar switches (103,104) are employed, where input nodes and output nodes of each crossbar switch are coupled to corresponding data modules. The ith crossbar switch has an Ni-input by Mi-output switch fabric, wherein Ni and Mi are positive integers greater than one. Each crossbar switch includes an input buffer at each input node, a crosspoint buffer at each crosspoint of the switch fabric, and an output buffer at each output node. The input buffer has an arbiter that reads data packets from the input buffer according to a first scheduling algorithm. An arbiter reads data packets from a crosspoint buffer queue according to a second scheduling algorithm. The output node receives segments of data packets provided from one or more corresponding crosspoint buffers.

EP2247040A2, drawing sheet 1 — Sheet 1 of 5

Term

3.1 yearsto projected expiry

Projected expiry 6 November 2029, counted from filing; an application has no term until it is granted.

15 claims: 1 independent, 14 dependent

1
Apparatus for transferring data between data modules, each of said data modules being at least one of a data processing module and a data storage module, the apparatus comprising:at least one crossbar switch, wherein input nodes and output nodes of each of the crossbar switches are coupled to corresponding ones of the data modules, the ith crossbar switch having an Ni-input node by Mi-output node switch fabric, wherein Ni and Mi are each positive integers greater than one and wherein each crossbar switch comprises: an input buffer at each input node having an input arbiter configured to read data packets from each input buffer in accordance with a first scheduling algorithm;a crosspoint buffer at each crosspoint of the switch fabric, the crosspoint buffer providing a queue having an output arbiter configured to read data packets from the crosspoint buffer queue in accordance with a second scheduling algorithm;and an output node configured to receive segments of data packets provided from one or more corresponding crosspoint buffers.

45 paragraphs in 4 sections, as filed

BACKGROUND OF THE INVENTION

0001Field of the Invention

0002The present invention relates to a buffered crossbar switch for inter-connection between multiple modules in a communication system.

0003Description of the Related Art

0004A crossbar switch (also known as a "crosspoint switch" or a "matrix switch") inter-connects a plurality of input ports and output ports to each other. A crossbar switch having P inputs and Q outputs has a switch fabric matrix with P x Q crosspoints where connections between input ports and output ports are made. Thus, packets arriving at one of the input ports might be routed to one or more specified output ports. For example, a packet might be routed to just a single specified output port (unicast), routed to all output ports (broadcast), or routed to multiple specified output ports (multicast).

0005Bufferless crossbar switches are frequently used in multi-processor computer systems to link processors with other resources, such as memory or other subsystems, or with internet switches or other high-performance communication networks. However, bufferless crossbar switches generally require switch fabric speedup, meaning that the internal data rate of the switch must be higher than the data rate of the link. These crossbar switches typically require high-complexity centrally-controlled scheduling algorithms to achieve 100% throughput, such as the Parallel Iterative Matching (PIM) and iSLIP algorithms.

0006In a buffered crossbar switch, buffers are included in the crossbar switch to temporarily store packets that cannot be routed to a specified output port instantly (i.e., blocked packets). For example, buffers might be provided for each input port, for each output port, for each crosspoint connection of the crossbar switch, or some combination thereof. Centrally-controlled scheduling algorithms might be employed to allow the input and output data streams to efficiently access the switch fabric.

SUMMARY OF THE INVENTION

0007In an exemplary embodiment, the present invention provides for transfer of data between data modules. Each data module is at least one of a data processing module and a data storage module. At least two coupled crossbar switches are employed and input nodes and output nodes of each of the crossbar switches are coupled to corresponding data modules. The ith crossbar switch has an Ni-input node by Mi-output node switch fabric, wherein Ni and Mi are each positive integers greater than one. Each crossbar switch includes an input buffer at each input node, a crosspoint buffer at each crosspoint of the switch fabric, and an output buffer at each output node. The input buffer has an input arbiter that reads data packets from the input buffer in accordance with a first scheduling algorithm. The crosspoint buffer provides a queue having an output arbiter that reads data packets from the crosspoint buffer queue in accordance with a second scheduling algorithm. The output node receives segments of data packets provided from one or more corresponding crosspoint buffers.

BRIEF DESCRIPTION OF THE DRAWINGS

0008Other aspects, features, and advantages of the present invention will become more fully apparent from the following detailed description, the appended claims, and the accompanying drawings in which like reference numerals identify similar or identical elements.

0009<figref idref="f0001">FIG. 1</figref> shows a block diagram of a buffered crossbar switch system operating in accordance with exemplary embodiments of the present invention;

0010<figref idref="f0002">FIG. 2</figref> shows a block diagram of a buffered crossbar switch in accordance with an embodiment of the present invention;

0011<figref idref="f0003">FIG. 3</figref> shows additional detail of the buffered crossbar switch of <figref idref="f0002">FIG. 2</figref>;

0012<figref idref="f0004">FIG. 4</figref> shows another buffered crossbar switch system operating in accordance with exemplary embodiments of the present invention; and,

0013<figref idref="f0005">FIG. 5</figref> shows another buffered crossbar switch system operating in accordance with exemplary embodiments of the present invention.

DETAILED DESCRIPTION

0014In accordance with embodiments of the present invention, a buffered crossbar switch system is provided for a high-throughput interconnection between multiple data modules in a communication system. For example, through distributed buffering and scheduling algorithms in the crossbar switch, embodiments of the present invention might be used to create concurrent virtual pipelines to interconnect multi-core processors with memory subsystems. Embodiments of the present invention might alternatively be used as, for example, client-server switches, internet routers, shared memory systems, or network switches. Embodiments of the present invention might achieve 100% throughput without speedup.

0015<figref idref="f0001">FIG. 1</figref> shows a block diagram of exemplary buffered crossbar switch system 100. As shown, buffered crossbar switch system 100 comprises processing unit 102, buffered crossbar switches 103 and 104, and memory unit 105. Buffered crossbar switches 103 and 104 are each in electrical communication with processing unit 102 and memory unit 105. Thus, as shown, buffered crossbar switches 103 and 104 might be configured to implement a high-throughput interconnection between multiple data modules within processing unit 102 and memory unit 105. In some embodiments, buffered crossbar switches 103 and 104 might be on one or more chips in communication with processing unit 102 and memory unit 105 via optical fiber.

0016In some embodiments, processing unit 102 might be a multi-core processor. For example, processing unit 102 might include N processors, where N is an integer greater than or equal to 1. As shown in <figref idref="f0001">FIG. 1</figref>, processing unit 102 includes processor 120(1), processor 120(2), and so on, through the Nth processor 120(N). Processors might be, for example, implemented as general-purpose processors, such as PowerPC or ARM processors, or the processors might be, for example, implemented as accelerators for specialized functions such as digital signal processing or security protocol processing. Processing unit 102 also includes arbiter 122(1), arbiter 122(2), and so on, through the Nth arbiter 122(N). As shown in <figref idref="f0001">FIG. 1</figref>, processor 120(1) is in electrical communication with arbiter 122(1), processor 120(2) is in electrical communication with arbiter 122(2) and so on, through processor 120(N) that is in electrical communication with arbiter 122(N). Each of arbiters 122(1) through 122(N) has at least one input port and one output port for communication with processors 120(1) through 120(N), respectively.

0017In some embodiments, memory unit 105 might employ memory controllers for coordination of reading and writing operations with memory. Memory might include, for example, at least one RAM buffer. Therefore, as shown in the exemplary embodiment, memory unit 105 might include M memory controllers for coordination of reading information from, and writing information to, one or more RAM buffers, where M is an integer greater than or equal to 1. Values for N and M might be equal, but are not necessarily equal. Memory is not limited to only a RAM buffer, and might be implemented with one or more other types of memory, such as flash memory. Memory unit 105 includes memory controllers 152(1) through memory controller 152(M). Memory unit 105 also includes arbiters 150(1) through arbiter 150(M). As shown in <figref idref="f0001">FIG. 1</figref>, memory controller 152(1) is in electrical communication with arbiter 150(1), memory controller 152(2) is in electrical communication with arbiter 150(2), and so on, through memory controller 152(M) that is in electrical communication with arbiter 150(M). Each of arbiters 150(1) through 150(M) has at least one input port and one output port for communication with memory controllers 152(1) through 152(M), respectively. Embodiments of the present invention might alternatively include a centralized arbiter for output arbitration. Other embodiments of the present invention might include a combination of localized and centralized arbiters.

0018Buffered crossbar switches 103 and 104 include a switch fabric configured to allow any input to the switch to be transferred to any one or more outputs of the switch. In exemplary embodiments, buffered crossbar switch 103 has N input ports and M output ports, and buffered crossbar switch 104 has M input ports and N output ports, where N is the number of processors and M is the number of memory controllers. As shown in <figref idref="f0001">FIG. 1</figref>, buffered crossbar switch 103 has N input ports, shown as 132(1) through 132(N), and M output ports, shown as 138(1) through 138(M). Buffered crossbar switch 104 has M input ports, shown as 148(1) through 148(M), and N output ports, shown as 142(1) through 142(N). While two buffered crossbar switches are shown in <figref idref="f0001">FIG. 1</figref>, the present invention is not so limited, and, thus, buffered crossbar switch system 100 might include up to B buffered crossbar switches, where B is an integer greater than or equal to 1.

0019For example, an exemplary embodiment of the present invention might include 5 buffered crossbar switches: one switch (not shown) to send addresses from processing unit 102 to memory unit 105, one switch (shown as buffered crossbar switch 103) to send data from processing unit 102 to memory unit 105, one switch (not shown) to send addresses from memory unit 105 to processing unit 102, one switch (shown as buffered crossbar switch 104) to send data from memory unit 105 to processing unit 102, and one switch (not shown) to send responses from the memory to the processors.

0020As shown in <figref idref="f0001">FIG. 1</figref>, buffered crossbar switch 103 is configured to switch data sent from processing unit 102 to memory unit 105, and buffered crossbar switch 104 is configured to switch data sent from memory unit 105 to processing unit 102. Thus, buffered crossbar switches 103 and 104 might be used to set which ones of N processors of processing unit 102 are in communication with which ones of the M memory controllers of memory unit 105. For example, data provided from processor 120(1) of processing unit 102 might be provided to input port 132(1) of buffered crossbar switch 103. Arbiter 122(1) controls what data processor 120(1) provides to input port 132(1) of buffered crossbar switch 103. As indicated by dashed arrow 134, buffered crossbar switch 103 might be configured to provide this data to output port 138(2) of buffered crossbar switch 103. Arbiter 150(2) controls which crosspoint buffer is transferred to output port 138(2) of buffered crossbar switch 103. Thus, buffered crossbar switch 103 provides this data to memory controller 152(2) of memory unit 105. Analogously, dashed arrows 135 and 136 indicate other exemplary data path settings of buffered crossbar switch 103. Buffered crossbar switch 103 might also be set to provide the data from one input port to multiple output ports. Similarly, data from memory controller 152(1) of memory unit 105 might be provided to buffered crossbar switch 104 at input port 148(2). Arbiter 150(1) controls what data memory controller 152(1) provides to input port 148(2) of buffered crossbar switch 104. As indicated by dashed arrow 145, buffered crossbar switch 104 might be set to provide this data to one output, in this instance, output port 142(2), which provides the data to processor 120(2) of processing unit 102. Arbiter 122(2) controls which crosspoint buffer is transferred to output port 142(2) of buffered crossbar switch 104. Analogously, dashed arrows 144 and 146 indicate other exemplary data path settings of buffered crossbar switch 104.

0021<figref idref="f0002">FIG. 2</figref> shows a block diagram of buffered crossbar switch system 200 in accordance with an embodiment of the present invention. As shown, buffered crossbar switch system 200 includes buffered crossbar switch 226 and virtual output queues (VOQs) 202, 204 and 206. Buffered crossbar switch 226 has 3 input ports, shown as input ports 220, 222 and 224, and 3 output ports, shown as output ports 246, 248 and 250. Thus, as shown in <figref idref="f0002">FIG. 2</figref>, N = M = 3. Input port 220 is in electrical communication with VOQ 202, input port 222 is in electrical communication with VOQ 204 and input port 224 is in electrical communication with VOQ 206. Output port 246 is in electrical communication with output queue 1 252, output port 248 is in electrical communication with output queue 2 254 and output port 250 is in electrical communication with output queue M 256. Some embodiments of the present invention might not include output queues 252, 254 and 256. Buffered crossbar switch 226 includes a FIFO buffer at each crosspoint of the 3 x 3 switch matrix. Thus, in exemplary embodiments, an N x M switch will have N x M buffers. These FIFO buffers are shown in <figref idref="f0002">FIG. 2</figref> as crosspoint queues 228, 230, 232, 234, 236, 238, 240, 242 and 244. Crosspoint queues temporarily store data before forwarding the data to the respective output port. Crosspoint queues allow the switch fabric to receive data nearly simultaneously at multiple input ports without blocking.

0022VOQs 202, 204 and 206 might each include one or more virtual first-in, first-out (FIFO) buffers. For example, VOQ 202 might include three FIFOs, shown in <figref idref="f0002">FIG. 2</figref> as FIFOs 202a, 202b and 202c; however, the present invention is not so limited and more or less FIFOs might be employed. Each VOQ buffer is in electrical communication with an arbiter that schedules access by the VOQs to the switch fabric of buffered crossbar switch 226. As shown in <figref idref="f0002">FIG. 2</figref>, VOQ 202 is in electrical communication with arbiter 214, VOQ 204 is in electrical communication with arbiter 216 and VOQ 206 is in electrical communication with arbiter 218. Although shown in <figref idref="f0002">FIG. 2</figref> as VOQs, embodiments of the present invention might employ other types of input buffering.

0023Packets sent to VOQs 202, 204 and 206 are sorted in FIFOs according to the destination addresses of the packets. Packets generally are of a variable length, thus, embodiments of the present invention segment packets into "cells" before they are provided to the switch fabric. A cell is a fixed number of bits such that packets of varying sizes might be segmented into a number of cells of fixed size, plus padding, if necessary. Crosspoint queues temporarily store data cells before cells are transferred to corresponding output queue(s). Each crosspoint queue is in electrical communication with an arbiter that schedules access by the crosspoint queues to the output ports of buffered crossbar switch 226. As shown, crosspoint queues 228, 230 and 232 are in electrical communication with arbiter 225, crosspoint queues 234, 236 and 238 are in electrical communication with arbiter 227 and crosspoint queues 240, 242 and 244 are in electrical communication with arbiter 229. Referring back to <figref idref="f0001">FIG. 1</figref>, arbiters 214, 216, and 218 correspond to arbiters 122(1) through 122(N) and arbiters 225, 227 and 229 correspond to arbiters 150(1) through 150(M) where N = M = 3.

0024Some embodiments of the present invention include output queues 252, 254 and 256 to reassemble data cells into packets. Output queues 252, 254 and 256 are not needed in embodiments that send packets directly to the switch fabric (i.e. do not segment packets into cells) because no packet reassembly is required.

0025Some embodiments of the present invention provide that the size of each crosspoint queue (e.g. queue depth) might be variable. Thus, the depth of each crosspoint queue might vary based on, for example, the round-trip time of data transfer through buffered crossbar switches 103 and 104 of <figref idref="f0001">FIG. 1</figref>. For example, a data path having a long round-trip time might require a larger crosspoint buffer to store data cells during the round-trip transfer of the packet. Thus, embodiments of the present invention provide crosspoint queues of varying depths to support different round-trip times.

0026Round-trip time is the time required for a credit to return to the input arbiter and the time required to send data to a crosspoint buffer. Initially, all crosspoint buffers are empty. Each input arbiter of the buffered crossbar switch might track how many credits are available for each crosspoint buffer. For example, the number of credits, C, available for each input port i at crosspoint buffer i, j might be represented as C(i, j). The number of credits might initially be equal to the depth of crosspoint buffer i, j. For each data cell sent to crosspoint buffer i, j from input port i, the number of credits, C, is decremented. When C(i, j) is zero, input port i must stop sending data cells, or else crosspoint buffer i, j will overflow. When the output arbiter at output j pulls a cell from crosspoint buffer (i, j), C(i, j) is incremented ("returning a credit"). For example, in a system where it takes two cycles for an input to write to a crosspoint buffer and three cycles to return a credit, the round-trip time is 5 cycles. With a deeper crosspoint buffer, the input can keep sending cells into the crosspoint buffer without exhausting its credits. In exemplary embodiments of the present invention, the depth of the crosspoint buffer might increase as credit return latency, and thus round-trip time, increases.

0027Input arbiters 214, 216 and 218 are distributed arbiters because each of N input FIFO buffers 202, 204 and 206 has one arbiter. Input arbiters are not in electrical communication with each other, thus facilitating scalability of buffered crossbar switch 226, since distributed arbiters might have a smaller physical size than a centralized arbiter. Similarly, output arbiters 225, 227 and 229 are also distributed arbiters because each of the M output queues 252, 254 and 256 has one arbiter.

0028Arbiters 214, 216, 218, 225, 227 and 229 are, for exemplary embodiments, implemented as "round-robin" schedulers implementing a round-robin algorithm: buffers are processed in ascending or descending order, and the process returns to the first buffer in the sequence after the last buffer in the sequence is served. For example, as each one of non-empty VOQs 202a, 202b and 202c is served by arbiter 214, data cells in the serviced buffer are provided to the switch fabric until the currently serviced VOQ becomes empty or the timeslot for the VOQ is over. Either such occurrence causes arbiter 214 to advance to the next VOQ in the sequence. Arbiter 214 skips empty VOQs to serve the next non-empty VOQ. Arbiters 216, 218, 225, 227 and 229 operate similarly as arbiter 214. Although a round-robin scheduling algorithm is described, the present invention is not so limited, and other scheduling algorithms might be employed, for example longest queue first scheduling wherein the queue that has held data for the longest time is given scheduling priority. Further, embodiments of the present invention provide input and output arbiters that do not necessarily employ the same scheduling algorithm and a combination of scheduling algorithms might be employed.

0029Generally, arbitration is performed as described above. Thus, if buffered crossbar switch 226 has an equal number, N, of inputs and outputs, a packet consisting of Z cells, where Z is an integer, might take N x Z clock cycles to reach the destination output queue. Exemplary embodiments of the present invention provide for cell prioritization. For example, priority might be assigned on a processor basis, wherein certain processors within processing unit 102 might be assigned higher priority than other processors, such that the associated arbiter would deliver all Z cells of a packet consecutively in Z cycles. Alternatively, priority might be assigned on a packet basis, wherein, for example, certain types of packets might be assigned higher priority than other types of packets.

0030Non-uniform output occurs with repeated access of a particular output port of buffered crossbar switch 226. Non-uniform output might limit the maximum throughput of buffered crossbar switch 226 since access to the output port is controlled by the output arbiter. For example, as shown in <figref idref="f0001">FIG. 1</figref>, each output port of buffered crossbar switch 226 corresponds to particular memory controller within memory unit 105. Typically, data transfer of a block of memory having sequential addresses is managed by one memory controller. The destination memory address determines which output of the buffered crossbar switch 226 is accessed. Thus, sequential memory accesses might cause a non-uniform output condition to arise, which might limit the maximum throughput of buffered crossbar switch 226. Some embodiments of the present invention scramble memory addresses before passing data for such addresses through buffered crossbar switch 226. Scrambling of memory addresses translates sequential memory accesses into accesses having a more uniform distribution of output port accesses.

0031<figref idref="f0003">FIG. 3</figref> shows additional detail of the buffered crossbar switch system of <figref idref="f0002">FIG. 2</figref>. As shown in <figref idref="f0003">FIG. 3</figref>, exemplary embodiments of buffered crossbar switch 301 additionally include M multiplexers, shown as 302, 304 and 306, where M is the number of outputs of the crossbar switch. As shown in <figref idref="f0003">FIG. 3</figref>, N = M = 3. For a switch having N inputs and M outputs, the switch fabric of buffered crossbar switch 301 might include M "N-to-1" multiplexers. As shown, arbiter 225 and crosspoint queues 228, 230 and 232 are in electrical communication with multiplexer 302. Arbiter 225 controls which of the crosspoint queues is asserted by multiplexer 302 to output port 246, thus accessing output queue 252. Similarly, crosspoint queues 234, 236 and 238 are in electrical communication with multiplexer 304. Arbiter 227 controls which of the crosspoint queues is asserted by multiplexer 304 to output port 248, thus accessing output queue 254. Crosspoint queues 240, 242 and 244 are in electrical communication with multiplexer 306. Arbiter 229 controls which of the crosspoint queues is asserted by multiplexer 306 to output port 250, thus accessing output queue 256.

0032While described above as being implemented as a monolithic chip, the present invention is not so limited. For example, as shown in <figref idref="f0004">FIG. 4</figref>, each buffered crossbar switch might occupy its own circuit board, shown as switch cards 406(1) through 406(L). Multiple switch cards might occupy a shelf, with multiple shelves, shown as shelves 404(1) through 404(K), forming switch chassis 400. Similarly, the processors and memories might be located on line chassis 402. Line chassis 402 might include multiple shelves, shown as shelves 409(1) through 409(K), each shelf including multiple line cards, shown as line cards 410(1) through 410(L). Each line card might contain devices, such as memory or processors, which communicate via the switch cards. A line chassis typically communicates with a switch chassis via electrical cables or optical links, shown as links 412, 414, 416 and 418.

0033<figref idref="f0005">FIG. 5</figref> shows another buffered crossbar switch system operating in accordance with exemplary embodiments of the present invention. As shown in <figref idref="f0005">FIG. 5</figref>, one or more buffered crossbar switches, shown as 500, might be used to make a chain of processors 102 and storage modules 105 where a storage module might serve as a buffer for two processors to communicate with each other. Buffered crossbar switches 500 operate as described with regard to <figref idref="f0001">FIG. 1</figref>. Buffered crossbar switches 500 might also be in electrical communication with bridge 502, thus, creating a branch along the chain. Bridge 500 might have one high-bandwidth port for communication with buffered crossbar switches 500, and lower bandwidth ports for communication with slower or legacy data modules, shown as processor 504 and storage modules 506 and 508. Thus, embodiments of the present invention provide a way for slower data modules to communicate with faster data modules without limiting the system bandwidth available to the faster data modules. Bridge 502 might employ buffered crossbar switches as described above, or might employ a design that doesn't use buffered crossbar switches.

0034It will be further understood that various changes in the details, materials, and arrangements of the parts which have been described and illustrated in order to explain the nature of this invention may be made by those skilled in the art without departing from the scope of the invention as expressed in the following claims.

0035Reference herein to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the invention. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments necessarily mutually exclusive of other embodiments. The same applies to the term "implementation."

0036While the exemplary embodiments of the present invention have been described with respect to processes of circuits, including possible implementation as a single integrated circuit, a multi-chip module, a single card, or a multi-card circuit pack, the present invention is not so limited. As would be apparent to one skilled in the art, various functions of circuit elements may also be implemented as processing blocks in a software program. Such software may be employed in, for example, a digital signal processor, micro-controller, or general purpose computer.

0037The present invention can be embodied in the form of methods and apparatuses for practicing those methods. The present invention can also be embodied in the form of program code embodied in tangible media, such as magnetic recording media, optical recording media, solid state memory, floppy diskettes, CD-ROMs, hard drives, or any other machine-readable storage medium, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention. The present invention can also be embodied in the form of program code, for example, whether stored in a storage medium, loaded into and/or executed by a machine, or transmitted over some transmission medium or carrier, such as over electrical wiring or cabling, through fiber optics, or via electromagnetic radiation, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention. When implemented on a general-purpose processor, the program code segments combine with the processor to provide a unique device that operates analogously to specific logic circuits. The present invention can also be embodied in the form of a bitstream or other sequence of signal values electrically or optically transmitted through a medium, stored magnetic-field variations in a magnetic recording medium, etc., generated using a method and/or an apparatus of the present invention.

0038A method of transferring data between data modules, is preferably provided, each of said data modules being at least one of a data processing module and a data storage module, the method comprising: passing the data through at least one crossbar switch, wherein input nodes and output nodes of each of the crossbar switches are coupled to corresponding ones of the data modules, the ith crossbar switch having an Ni-input node by Mi-output node switch fabric, wherein Ni and Mi are each positive integers greater than one and wherein each crossbar switch comprises: reading, from an input buffer at each input node having an input arbiter, data packets from each input buffer in accordance with a first scheduling algorithm; reading, from an output arbiter of a crosspoint buffer at each crosspoint of the switch fabric, the crosspoint buffer having a queue with an output arbiter, data packets from the crosspoint buffer queue in accordance with a second scheduling algorithm; and receiving, at an output node, segments of data packets from one or more corresponding crosspoint buffers.

0039The method may further comprise varying, for each crosspoint buffer queue a depth of the queue in accordance with a round-trip time of data transfer through one of the at least one crossbar switch.

0040The method may further comprise scrambling, when a data storage module employs sequential addressing of locations in memory, addresses of the locations in memory before passing the data through the at least one crossbar switch.

0041The method may further comprise partitioning the packet of data provided to the input buffer into equal length cells.

0042A machine-readable storage medium is preferably provided, having encoded thereon program code, wherein, when the program code is executed by a machine, the machine implements a method for transferring data between data modules, each of said data modules being at least one of a data processing module and a data storage module, comprising the steps of: passing the data through at least one crossbar switch, wherein input nodes and output nodes of each of the crossbar switches are coupled to corresponding ones of the data modules, the ith crossbar switch having an Ni-input node by Mi-output node switch fabric, wherein Ni and Mi are each positive integers greater than one and wherein each crossbar switch comprises: reading, from an input buffer at each input node having an input arbiter, data packets from each input buffer in accordance with a first scheduling algorithm; reading, from an output arbiter of a crosspoint buffer at each crosspoint of the switch fabric, the crosspoint buffer having a queue with an output arbiter, data packets from the crosspoint buffer queue in accordance with a second scheduling algorithm; and receiving, at an output node, segments of data packets from one or more corresponding crosspoint buffers.

0043It should be understood that the steps of the exemplary methods set forth herein are not necessarily required to be performed in the order described, and the order of the steps of such methods should be understood to be merely exemplary. Likewise, additional steps may be included in such methods, and certain steps may be omitted or combined, in methods consistent with various embodiments of the present invention.

0044As used herein in reference to an element and a standard, the term "compatible" means that the element communicates with other elements in a manner wholly or partially specified by the standard, and would be recognized by other elements as sufficiently capable of communicating with the other elements in the manner specified by the standard. The compatible element does not need to operate internally in a manner specified by the standard.

0045Also for purposes of this description, the terms "couple," "coupling," "coupled," "electrical communication," "connect," "connecting," or "connected" refer to any manner known in the art or later developed in which energy is allowed to be transferred between two or more elements, and the interposition of one or more additional elements is contemplated, although not required. Conversely, the terms "directly coupled," "directly connected," etc., imply the absence of such additional elements. Signals and corresponding nodes or ports may be referred to by the same name and are interchangeable for purposes here.

Contents4

Every citation, both ways

Citations
Document	Relation	Office	Cited during
EP1414252A2	Cites	European Patent Office (EPO)	Search report
US4982187A	Cites	United States of America	Search report
US6888841B1	Cites	United States of America	Search report

134 members in 10 offices; this record represents the family

This recordMember, by publication datePriority claim

Priority claims5

Priority claims
Document	Office	Kind	Date
430438	United States of America	–	2009-04-27
43043809	United States of America	A	2009-04-27
43043809	United States of America	A	2009-04-27
430438	–	–	–
US20090430438	–	–	–

Members134

Family members, oldest first
Document	Office	Kind
ATA197792A	Austria	A	1993-10-15
CA2107829A1	Canada	A1	1994-04-08
EP0592386A1	European Patent Office (EPO)	A1	1994-04-13
AT397745B	Austria	B	1994-06-27
JPH077797A	Japan	A	1995-01-10
EP0592386B1	European Patent Office (EPO)	B1	1997-05-28
DE59306580D1	Germany	D1	1997-07-03
DK0592386T3	Denmark	T3	1997-09-08
US5673328A	United States of America	A	1997-09-30
US2010238937A1	United States of America	A1	2010-09-23
US2010238938A1	United States of America	A1	2010-09-23
CN101873253A	China	A	2010-10-27
US2010272117A1	United States of America	A1	2010-10-28
EP2247040A2This record	European Patent Office (EPO)	A2	2010-11-03
KR20100118054A	Republic of Korea	A	2010-11-04
JP2010259045A	Japan	A	2010-11-11
US2010293312A1	United States of America	A1	2010-11-18
US2010293345A1	United States of America	A1	2010-11-18
US2010293353A1	United States of America	A1	2010-11-18
TW201108668A	Taiwan Province of China	A	2011-03-01
US2011222540A1	United States of America	A1	2011-09-15
US2011222552A1	United States of America	A1	2011-09-15
US2011222553A1	United States of America	A1	2011-09-15
US2011225168A1	United States of America	A1	2011-09-15
US2011225334A1	United States of America	A1	2011-09-15
US2011225337A1	United States of America	A1	2011-09-15
US2011225372A1	United States of America	A1	2011-09-15
US2011225376A1	United States of America	A1	2011-09-15
US2011225391A1	United States of America	A1	2011-09-15
US2011225394A1	United States of America	A1	2011-09-15
US2011225588A1	United States of America	A1	2011-09-15
US2011225589A1	United States of America	A1	2011-09-15
US2011289179A1	United States of America	A1	2011-11-24
US2011289180A1	United States of America	A1	2011-11-24
US2011289279A1	United States of America	A1	2011-11-24
EP2247040A3	European Patent Office (EPO)	A3	2011-12-07
US2012002546A1	United States of America	A1	2012-01-05
US2012005391A1	United States of America	A1	2012-01-05
US2012020210A1	United States of America	A1	2012-01-26
US2012020223A1	United States of America	A1	2012-01-26
US2012020249A1	United States of America	A1	2012-01-26
US2012020250A1	United States of America	A1	2012-01-26
US2012020251A1	United States of America	A1	2012-01-26
US2012020366A1	United States of America	A1	2012-01-26
US2012020367A1	United States of America	A1	2012-01-26
US2012020368A1	United States of America	A1	2012-01-26
US2012020369A1	United States of America	A1	2012-01-26
US2012020370A1	United States of America	A1	2012-01-26
US2012020371A1	United States of America	A1	2012-01-26
US2012023295A1	United States of America	A1	2012-01-26
US2012023498A1	United States of America	A1	2012-01-26
US2012036351A1	United States of America	A1	2012-02-09
US2012076153A1	United States of America	A1	2012-03-29
US2012084498A1	United States of America	A1	2012-04-05
US2012131283A1	United States of America	A1	2012-05-24
US2012155495A1	United States of America	A1	2012-06-21
US2012158729A1	United States of America	A1	2012-06-21
US8243737B2	United States of America	B2	2012-08-14
US8255644B2	United States of America	B2	2012-08-28
US2012230341A1	United States of America	A1	2012-09-13
US2012236857A1	United States of America	A1	2012-09-20
US8321385B2	United States of America	B2	2012-11-27
US2012300772A1	United States of America	A1	2012-11-29
US8352669B2	United States of America	B2	2013-01-08
US2013042038A1	United States of America	A1	2013-02-14
TWI390913B	Taiwan Province of China	B	2013-03-21
US8407707B2	United States of America	B2	2013-03-26
US2013086332A1	United States of America	A1	2013-04-04
US2013089098A1	United States of America	A1	2013-04-11
US2013089099A1	United States of America	A1	2013-04-11
US2013089109A1	United States of America	A1	2013-04-11
US2013091330A1	United States of America	A1	2013-04-11
US2013097345A1	United States of America	A1	2013-04-18
US2013125127A1	United States of America	A1	2013-05-16
US2013128896A1	United States of America	A1	2013-05-23
US2013142205A1	United States of America	A1	2013-06-06
US8473657B2	United States of America	B2	2013-06-25
US8489791B2	United States of America	B2	2013-07-16
US8489792B2	United States of America	B2	2013-07-16
US8489794B2	United States of America	B2	2013-07-16
US8499137B2	United States of America	B2	2013-07-30
US8505013B2	United States of America	B2	2013-08-06
US8514874B2	United States of America	B2	2013-08-20
US8515965B2	United States of America	B2	2013-08-20
US8537832B2	United States of America	B2	2013-09-17
US8539199B2	United States of America	B2	2013-09-17
US8547878B2	United States of America	B2	2013-10-01
US8565250B2	United States of America	B2	2013-10-22
US8576862B2	United States of America	B2	2013-11-05
US2013304926A1	United States of America	A1	2013-11-14
US8615013B2	United States of America	B2	2013-12-24
US8619787B2	United States of America	B2	2013-12-31
US8638805B2	United States of America	B2	2014-01-28
US8677075B2	United States of America	B2	2014-03-18
US8683221B2	United States of America	B2	2014-03-25
US8705531B2	United States of America	B2	2014-04-22
CN101873253B	China	B	2014-06-04
US2014153575A1	United States of America	A1	2014-06-05
US8761204B2	United States of America	B2	2014-06-24
JP5537956B2	Japan	B2	2014-07-02

Showing 100 of 134

72 legal events, as 10 offices reported them to INPADOC

Over the term

Point at a mark for the event

Events

Legal events, newest first
	Event	Code	Office
2020-10-30	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2020-06-30	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2020-06-03	Application deemed withdrawn, or ip right lapsed, due to non-payment of renewal feeWithdrawnR119	R119	DE
2020-05-29	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2020-03-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2019-12-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2019-11-29	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2019-10-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2019-07-24	Gb: european patent ceased through non-payment of renewal feeCeasedGBPC	GBPC	EP
2019-06-28	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2019-01-31	Annual fee paid to national office [announced via postgrant information from national office to epo]GrantedPGFP	PGFP	EP
2018-11-30	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-10-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-10-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-09-28	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-09-05	Patent lapsedLapsedMM4A	MM4A	IE
2018-08-31	Lapsed because of non-payment of the annual feeLapsedMM	MM	BE
2018-08-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-08-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-08-31	Notification of lapseLapsedST	ST	FR
2018-07-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-07-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-07-25	No opposition filedOpposition26N	26N	EP
2018-06-29	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-06-22	No opposition filed within time limitOppositionORIGINAL CODE: 0009261PLBE	PLBE	EP
2018-06-22	Information on the status of an ep patent application or granted ep patentGrantedSTATUS: NO OPPOSITION FILED WITHIN TIME LIMITSTAA	STAA	EP
2018-05-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-05-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-05-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-05-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-05-17	No opposition filed against granted patent, or epo opposition proceedings concluded without decisionGrantedR097	R097	DE
2018-04-30	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-04-30	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-04-30	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-02-28	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-02-28	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-02-28	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-02-28	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-02-28	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-02-28	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-02-28	Annual fee paid to national office [announced via postgrant information from national office to epo]GrantedPGFP	PGFP	EP
2018-01-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-01-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-01-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-01-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-01-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-01-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2018-01-15	Deletion acc. to par. 5 (withdrawal of the translation of the ep patent)MK05	MK05	AT
2018-01-10	Invalidated european patentMG4D	MG4D	LT
2017-12-20	Patent invalid in the netherlands as no translation has been filedMP	MP	NL
2017-09-28	Dpma publication of mentioned ep patent grantGrantedR096	R096	DE
2017-09-15	Reference to at number (ep patent enters austrian national phase)REF	REF	AT
2017-09-06	European patents granted designating irelandGrantedFG4D	FG4D	IE
2017-08-31	European patent takes effect as a national patent in ch/liEP	EP	CH
2017-08-16	Designated contracting statesAK	AK	EP
2017-08-16	European patent grantedGrantedFG4D	FG4D	GB
2017-07-14	(expected) grantORIGINAL CODE: 0009210GRAA	GRAA	EP
2017-07-14	Information on the status of an ep patent application or granted ep patentGrantedSTATUS: THE PATENT HAS BEEN GRANTEDSTAA	STAA	EP
2017-07-11	Grant fee paidORIGINAL CODE: EPIDOSNIGR3GRAS	GRAS	EP
2017-04-26	Intention to grant announcedINTG	INTG	EP
2017-03-27	Despatch of communication of intention to grant a patentORIGINAL CODE: EPIDOSNIGR1GRAP	GRAP	EP
2017-03-27	Information on the status of an ep patent application or granted ep patentGrantedSTATUS: GRANT OF PATENT IS INTENDEDSTAA	STAA	EP
2017-03-22	Information provided on ipc code assigned before grantRIC1	RIC1	EP
2017-02-14	Amendment of ipc main classPREVIOUS MAIN CLASS: H04L0012560000R079	R079	DE
2016-04-13	Party data changed (applicant data changed or rights of an application transferred)RAP1	RAP1	EP
2012-07-18	Request for examination filed17P	17P	EP
2011-12-07	Designated contracting statesAK	AK	EP
2011-12-07	Information provided on ipc code assigned before grantRIC1	RIC1	EP
2011-12-07	Information provided on ipc code assigned before grantRIC1	RIC1	EP
2011-11-04	Search report despatchedORIGINAL CODE: 0009013PUAL	PUAL	EP
2010-11-03	Designated contracting statesAK	AK	EP
2010-10-01	Public reference made under article 153(3) epc to a published international application that has entered the european phaseORIGINAL CODE: 0009012PUAI	PUAI	EP