IL116288A

Parallel i/o network file server architecture

Abstract

This record has no abstract on file.

Term

No projected expiry on record.

8 claims: 3 independent, 5 dependent

1
CLAIMS l.-' Network controller apparatus for use with a first data network carrying signals representinginformation packets encoded according to a firstphysical layer protocol, comprising:a first network interface unit, a first packet busand first packet memory addressable by said firstnetwork interface unit over said first packet bus, saidfirst network interface unit including means for'receiving signals over said first network represent-ί ngincoming information packets, extracting said incominginformation packets . and writing said incominginformation packets into said first packet memory oversaid first packet bus;a first packet bus port;first packet DMA means for reading data over saidfirst packet bus from said first packet memory to saidfirst packet bus port;and a local processor including means for accessing said incoming information packets in said first packet memory ' and, in response to the contents of said incoming information packets, controlling said first packet DMA means to read selected data over said first packet bus from said first packet memory to said first it,, packet bus pert, said local processor including a CPU, f i;... - 'I '·1 a CPU bus and CPU memory containing CPU instructions, 133 said local processor operating in response to said CPUinstructions, said CPU instructions being received by-said CPU over said CPU bus independently of any of saidwriting by said first network interface unit ofincoming information packets into said first packetmemory over said first packet bus and independently ofany of said reading by said first packet DMA means ofdata over said first packet bus from said first packetmemory to said first packet .bus port.
3
Apparatus according to claim i, furthercomprising a first FIFO having first and second ports,said first port of said first FIFO being said firstpacket bus port.
6
Apparatus according to claim :5, wherein saidsecond physical layer protocol is the same as saidfirst physical layer protocol. 136

Independent claims3

613 paragraphs in 45 sections, as filed

·/ D’pn mi’» imyn} zri’ipn mwFinnji

PARALLEL I/O NETWORK FILE SERVER ARCHITECTURE AUSPEX SYSTEMS, INC.C: 23778 / t t i !

I

I -1-

PARALLEL I/O NETWORK FILE SERVER ARCHITECTURE INVENTORS : EDWARD JOHN ROW, LAURENCE B. BOUCHER,

WILLIAM M. PITTS, STEPHEN E. BLIGHTMAN

The present application is related to thefollowing U.S. Patent Applications, all filedconcurrently herewith:

1. MULTIPLE FACILITY OPERATING SYSTEM ARCHITECTURE, invented by David Hitz, Allan Schwartz,Janies Lau and Guy Harris;

2. ENHANCED VMEBUS PROTOCOL UTILIZING

PSEUDOSYNCHRONOUS HANDSHAKING AND BLOCK MODE DATA TRANSFER, invented by Daryl Starr; and

3. BUS LOCKING FIFO MULTI-PROCESSOR COMMUNICATIONS

SYSTEM UTILIZING PSEUDOSYNCHRONOUS HANDSHAKING AND BLOCK MODE DATA TRANSFER invented by Daryl D. Starr,William Pitts and Stephen Blightman.

The above applications are all assigned to theassignee of the present invention and are all expresslyincorporated herein by reference.

BACKGROUND OF THE INVENTION

Field of the Invention

The invention relates to computer data networks, and more particularly, to network file server architectures for computer networks.

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 (,. -2-

Description of the Related Art"

Over the past ten years, remarkable Increases inhardware price/performance ratios have caused astartling shift in both technical and office computingenvironments. Distributed workstation-server networksare displacing the once pervasive dumb terminalattached to mainframe or minicomputer. To date,however, network I/O limitations have constrained thepotential performance available to workstation users.This situation has developed in part because dramaticjumps in microprocessor performance have exceededincreases in network I/O performance.

In a computer network, individual userworkstations are referred to as clients, and sharedresources for filing, printing, data storage and wide- area communications are referred to as servers.

Clients and servers are all considered nodes of a network. Client nodes use standard communications protocols to exchange service requests and responses with server nodes .

PreSent-day network clients and servers usuallyrun the DOS, Macintosh OS, OS/2, or Unix operatingsystems. Local networks are usually Ethernet or TokenRing at the high end, Arcnet in the midrange, or

LocalTalk or StarLAN at the low end. The client-server

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -3- communication protocols are fairly strictly dictated bythe operating system environment -- usually one ofseveral proprietary schemes for PCs (NetWare, 3Plus,Vines, LANManager, LANServer); AppleTalk forMacintoshes; and TCP/IP with NFS or RFS for Unix.These protocols are all well-known in the industry.

Unix client nodes typically feature a 16- or 32-bit microprocessor with 1-8 MB of primary memory, a640 x 1024 pixel display, and a built-in networkinterface. A 40-100 MB local disk is often optional.Low-end examples are 80286-based PCs or 68000-basedMacintosh I's; mid-range machines include 80386 PCs,Macintosh Il's, and 680X0-based Unix workstations;high-end machines include RISC-based DEC, HP, and SunUnix workstations. Servers are typically nothing morethan repackaged client nodes, configured in 19-inchracks rather than desk sideboxes. The extra space ofa 19-inch rack is used for additional backplane slots,disk or tape drives, and power supplies.

Driven by RISC and CISC microprocessordevelopments, client workstation performance hasincreased by more than a factor of ten in the last fewyears. Concurrently, these extremely fast clientshave also gained an appetite for data that remoteservers are unable to satisfy. Because the I/Oshortfall is most dramatic in the Unix environment, the

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -4- description of the preferred embodiment of the presentinvention will focus on Unix file servers. Thearchitectural principles that solve the Onix server I/Oproblem, however, extend easily to server performancebottlenecks in other operating system environments aswell. Similarly, the description of the preferredembodiment will focus on Ethernet implementations,though the principles extend easily to other types of networks .

In most Unix environments, clients and serversexchange file data using the Network File System("NFS"), a standard promulgated by Sun Microsystems andnow widely adopted by the Unix community. NFS isdefined in a document entitled, “NFS: Network FileSystem Protocol Specification," Request For Comments(RFC) 1094, by Sun Microsystems, Inc. (March 1989).This document is incorporated herein by reference inits entirety.

While simple and reliable, NFS is not optimal.Clients using NFS place considerable demands upon bothnetworks and NFS servers supplying clients with NFSdata. This demand is particularly acute for so-calleddiskless clients that have no local disks and therefore depend on a file server for applicationbinaries and virtual memory paging as well as data.For these Unix client-server configurations, the ten-

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -5- to-one increase in client power has not been matched by a ten-to-one increase in Ethernet capacity, in disk speed, or server disk-to-network I/O throughput.

The result is that the number of diskless clientsthat a single modern high-end server can adequatelysupport has dropped to between 5-10, depending onclient power and application workload. For clientscontaining small local disks for applications andpaging, referred to as dataless clients, the client-to-server ratio is about twice this, or between 10-20.

Such low client/server ratios cause piecewisenetwork configurations in which each local Ethernetcontains isolated traffic for its· own 5-10 (diskless) clients and dedicated server. For overallconnectivity, these local networks are usually joinedtogether with an Ethernet backbone or, in the future,with an FDDI backbone. These backbones are typicallyconnected to the local networks either by IP routers orMAC-level bridges, coupling the local networks togetherdirectly, or by a second server functioning as anetwork interface, coupling servers for all the localnetworkstogether.

In addition to performance considerations, the lowclient-to-server ratio creates computing problems inseveral additional ways:

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 ν. -6- 1. Sharing. Development groups of more than 5-10 people cannot share the sante server, and thus cannoteasily share files without file replication and manual,multi-server updates. Bridges or routers are a partialsolution but inflict a performance penalty due to morenetwork hops. 2. Administration. System administratorsmust maintain many limited-capacity servers rather thana few more substantial servers. This burden includes network administration, hardware maintenance, and user account administration. 3. File System Backup. System administrators oroperators must conduct multiple file system backups,which can be onerously time consuming tasks. It isalso expensive to duplicate backup peripherals on eachserver (or every few servers. if slower network backupis used). 4. Price Per Seat. With only 5-10 clientsper server, the cost of the server must be shared byonly a small number of users. The real cost of anentry-level Unix workstation is therefore significantlygreater,· often as much as 140% greater, than the cost of the workstation alone.

The widening I/O gap, as well as administrativeand economic considerations, demonstrates a need forhigher-performance, larger-capacity Unix file servers.

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7

V

-7-

Conversion of a display-less workstation into a servermay address disk capacity issues, but does nothing toaddress fundamental I/O limitations. As an NFS server,

the one-time workstation must sustain 5-10 or more5 times the network, disk, backplane, and file systemthroughout than it was designed to support as aclient. Adding larger disks, more network adaptors,extra primary memory, or even a faster processor do notresolve basic architectural I/O constraints; I/O 10 throughput does not increase sufficiently.

Other prior art computer architectures, while not specifically designed as file servers, may potentiallybe used as such. In one such well-known architecture,a CPU, a memory unit, and two I/O processors are 15 connected to a single bus. One of the I/O processorsoperates a set of disk drives, and if the architectureis to be used as a server, the other I/O processor would be connected to a network. This architecture isnot optimal as a file server, however, at least because 20 the two I/O processors cannot handle network filerequests without involving the CPU. All network filerequests that are received by the network I/O processorare first transmitted to the CPU, which makes appropriate requests to the disk-I/0 processor for 25 satisfaction of the network request.

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 k -8-

In another such computer architecture, a diskcontroller CPU manages acceds to disk drives, andseveral other CPUs, three for example, may beclustered around the disk controller CPU. Each of the other CPUs can be connected to its own network. The network CPUs are each connected to the disk controller CPU as well as to each other for interprocessorcommunication. One of the disadvantages of thiscomputer architecture is that each CPU in the systemruns its own complete operating system. Thus, networkfile server requests must be handled by an operatingsystem which is also heavily loaded with facilities andprocesses for performing a large number of other, · nonfile-server tasks. Additionally, the interprocessorcommunication is not optimized for file server typerequests.

In yet another computer architecture, a pluralityof CPUs, each having its own cache memory for data andinstruction storage, are connected to a common bus witha system memory and a disk controller. The diskcontroller and each of the CPUs have direct memoryaccess to the system memory, and one or more of theCPUs can be connected to a network. This architectureis disadvantageous as a file server because, amongother things, both file data and the instructions forthe CPUs reside in the same system memory. There will

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -9- be instances, therefore, in which the CPUs must stoprunning while they wait for large blocks of file datato be transferred between system memory and the networkCPU. Additionally, as with both of the previouslydescribed computer architectures, the entire operatingsystem runs on each of the CPUs, including the network CPU.

In yet another type of computer architecture, alarge number of CPUs are connected together in ahypercube topology. One of more of these CPUs can beconnected to networks, while another can be connectedto disk drives. This architecture is alsodisadvantageous as a file server because, among otherthings, each processor runs the entire operatingsystem. Interprocessor communication is also notoptimal for file server applications.

SUMMARY OF THE INVENTION

The present invention involves a new, server-specific I/O architecture that is optimized for a Unixfile server's most common actions — file operations.Roughly stated, the invention involves a file serverarchitecture comprising one or more networkcontrollers, one or more file controllers, one or morestorage processors, and a system or buffer memory, allconnected over a message passing bus and operating in

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -10-

10 15 20 parallel with the Unix host processor. The networkcontrollers each connect to one or more network, andprovide all protocol processing between the networklayer data format and an internal file server formatfor communicating client requests to other processorsin the server. Only those data packets which cannot beinterpreted by the network controllers, for exampleclient requests to run a client-defined program on theserver, are transmitted to the Unix host forprocessing. Thus the network controllers, filecontrollers and storage processors contain only smallparts of an overall operating system, and each isoptimized for the particular type of work to which it is dedicated.

Client requests for file operations aretransmitted to one of the file controllers which,independently of the Unix host, manages the virtualfile system of a mass storage device which is coupledto the storage processors. The file controllers mayalso control data buffering between the storageprocessors and the network controllers, through thesystem memory. The file controllers preferably eachinclude a local buffer memory for caching file controlinformation, separate from the system memory forcaching file data. Additionally, the networkcontrollers, file processors and storage processors are 25

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 V. -11- all designed to avoid any instruction fetches from thesystem memory, instead keeping all instruction memoryseparate and local. This arrangement eliminatescontention on the backplane between microprocessorinstruction fetches and transmissions of message and file data.

BRIEF DESCRIPTION OF THE DRAWINGS

The inveAtion will be described with respect toparticular embodiments thereof, and reference will bemade to the drawings, in which:

Fig. 1. is a block diagram of a prior art fileserver architecture;

Fig. 2 is a block diagram of a file serverarchitecture according to the invention;

Fig. 3 is a block diagram of one of the networkcontrollers shown in Fig. 2;

Fig. 4 is a block diagram of one of the filecontrollers shown in Fig. 2;

Fig. 5 is a block diagram of one of the storageprocessors shown in Fig. 2;

Fig. 6 is a block diagram of one of the systemmemory cards shown in Fig. 2;

Figs. 7A-C are a flowchart illustrating theoperation of a fast transfer protocol BLOCK WRITEcycle; and

Attorney Docket No..-AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -12-

Figs. 8A-C are a flowchart illustrating the

operation of a fast transfer protocol BLOCK READ cycle.

DETAILED DESCRIPTION 5 For comparison purposes and background, an illustrative prior-art file server architecture willfirst be described with respect to Fig. 1. Fig. 1 isan overall block diagram of a conventional prior-artUnix-based file server for Ethernet networks. It 10 consists of a host CPU card 10 with a singlemicroprocessor on board. The host CPU card 10 connectsto an Ethernet #1 12, and it connects via a memory management unit (MMU) 11 to a large memory array 16.The host CPU card 10 also drives a keyboard, a video 15 display, and two RS232 ports (not shown) . It alsoconnects via the MMU 11 and a standard 32-bit VME bus20 to various peripheral devices, including an SMD diskcontroller 22 controlling one or two disk drives 24, aSCSI host adaptor 26 connected to a SCSI bus 28, a 20 tape controller 30 connected to a quarter-inch tapedrive 32, and possibly a network #2 controller 34 connected to a second Ethernet 36. The SMD disk controller 22 can communicate with memory array 16 bydirect memory access via bus 20 and MMU 11, with either 25 the disk controller or the MMU acting as a bus master.

Attorney Docket No. .-AUSP7209WPl/WSW/AUSP/7209.001 8/24/89-7 i . -13-

This configuration is illustrative; many variations are available.

The system communicates over the Ethernets usingindustry standard TCP/IP and NFS protocol stacks. Adescription of protocol stacks in general can be foundin Tanenbaum, "Computer Networks" (Second Edition,Prentice Hall: 1988). File server protocol stacks aredescribed at pages 535-546. The Tanenbaum reference isincorporated herein by reference.

Basically, the following protocol layers areimplemented in the apparatus of Fig. 1:

Network Layer. The network layer converts datapackets between a formal specific to Ethernets and aformat which is independent of the particular type ofnetwork used. the Ethernet-specific format which isused in the apparatus of Fig. 1 is described in Hornig,"A Standard For The Transmission of IP Datagrams OverEthernet Networks," RFC 894 (April 1984), which isincorporated herein by reference.

The Internet Protocol (IP) Laver. This layerprovides the functions necessary to deliver a packageof bits (an internet datagram) from a source to adestination over an interconnected system of networks.For messages to be sent from the file server to aclient, a higher level in the server calls the IPmodule, providing the internet address of the

Attorney Docket No.:AUSP7209WP1/WSW/AUSP/7209.001 8/24/89-7 -14- destination client and the message to transmit. The IPmodule performs any required fragmentation of themessage to accommodate packet size limitations of anyintervening gateway, adds internet headers to eachfragment, and calls on the network layer to transmitthe resulting internet datagrams. The internet headerincludes a local network destination address (translated from the internet address) as well as otherparameters.

For messages received by the IP layer from thenetwork layer, the IP module determines from theinternet address whether the. datagram is to beforwarded to another host on another network, forexample on a second Ethernet such as 36 in Fig. 1, orwhether it is intended for the server itself. If it is

intended for another host on the second network, the IP module ' determines a local net address for the destination and calls on the local network layer forthat network to send the datagram. If the datagram isintended for an application program within the server,the IP layer strips off the header and passes theremaining portion of the message to the appropriatenext higher layer. The internet protocol standard usedin the illustrative apparatus of Fig. 1 is specified inInformation Sciences Institute, "Internet Protocol,DARPA Internet Program Protocol Specification,'* RFC 791

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7

V

-15- (September 1981), which is incorporated herein by reference. TCP/UDP Layer. This layer is a datagram servicewith more elaborate packaging and addressing options 5 than the IP layer. For example, whereas an IP datagramcan hold about 1,500 bytes and be addressed to hosts,UDP datagrams can hold about 64KB and be addressed to aparticular port within a host. TCP and UDP arealternative protocols at this layer; applications 10 requiring ordered reliable delivery of streams of datamay use TCP, whereas applications (such as NFS) whichdo not require ordered and reliable delivery may use UDP.

The prior art file server of Fig. 1 uses both TCP 15 and UDP. It uses UDP for file server-related services, and uses TCP for certain other services which theserver provides to network clients. The UDP isspecified in Postel, "User Datagram Protocol," RFC 768(August 28, 1980), which is incorporated herein by 20 reference. TCP is specified in Postel, "TransmissionControl Protocol," RFC 761 (January 1980) and RFC 793(September 1981), which is also incorporated herein by reference. XDR/RPC Laver. This layer provides functions 25 callable from higher level programs to run a designatedprocedure on a remote machine. It also provides the

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 k. -16-

decoding necessary to permit a client machine toexecute a procedure on the server. For example, acaller process in a client node may send a call messageto the server of Fig. 1. The call message includes aspecification of the desired procedure, and itsparameters. The message is passed up the stack to theRPC layer, which calls the appropriate procedure withinthe server. When the procedure is complete, a replymessage is generated and RPC passes it back down thestack and over the network to the caller client. RPC is described in Sun Microsystems, Inc., “RPC: RemoteProcedure Call Protocol Specification, Version 2," RFC1057 (June 1988), which is incorporated herein by reference. RPC uses the XDR external data representationstandard to represent information passed to and fromthe underlying UDP layer. XDR is merely a dataencoding standard, useful for transferring data betweendifferent computer architectures. Thus, on the networkside of the XDR/RPC layer, information is machine-independent; on the host application side, it may notbe. XDR is described in Sun Microsystems, Inc., “XDR:External Data Representation Standard," RFC 1014 (June1987), which is incorporated herein by reference. NFS Laver. The NFS ("network file system")layer is one of the programs available on the server

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7

V -17- which an RPC request can call. The combination of hostaddress, program number, and procedure number in an RPCrequest can specify one remote NFS procedure to be called.

Remote procedure calls to NFS on the file serverof Fig. 1 provide transparent, stateless, remote access to shared files on the disks 24. NFS assumes a file system that is hierarchical, with directories as allbut the bottom level of files. Client hosts can call any of about 20 NFS procedures including suchprocedures as reading a specified number of bytes froma specified file; writing a specified number of bytesto a specified file; creating, renaming and removingspecified files; parsing directory trees; creating andremoving directories; and reading and setting file attributes. The location on disk to which and from which data is stored and retrieved is always specifiedin logical terms, such as by a file handle or Inodedesignation and a byte offset. The details of theactual data storage are hidden from the client. TheNFS procedures, together with possible higher levelmodules such as Unix VFS and UFS, perform allconversion of logical data addresses to physical dataaddresses such as drive, head, track and sectoridentification. NFS is specified in Sun Microsystems,Inc., "NFS: Network File System Protocol

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -18-

Specification," RFC 1094 (March 1989), incorporatedherein by reference.

With the possible exception of the network layer,all the protocol processing described above is done insoftware, by a single processor in the host CPU card 10. That is, when an Ethernet packet arrives onEthernet 12, the host CPU 10 perforins all the protocolprocessing in the NFS stack, as well as the protocol processing for any other application which may be • /· running on the host 10. NFS procedures are run on thehost CPU 10, with access to memory 16 for both data andprogram code being provided via MMU 11. Logicallyspecified data addresses are converted to a much morephysically specified form and communicated to the SMDdisk controller 22 or the SCSI bus 28, via the VME bus20, and all disk caching is done by the host CPU 10through the memory 16. The host CPU card 10 also runsprocedures for performing various other functions ofthe file server, communicating with tape controller 30via the VME bus 20. Among these are client-definedremote procedures requested by client workstations.

If the server serves a second Ethernet 36, packets from that Ethernet are transmitted to the host CPU 10over the same VME bus 20 in the form of IP datagrams.Again, all protocol processing except for the networklayer is performed by software processes running on the

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -19- host CPU 10. In addition, the protocol processing forany message that is to be sent from the server out oneither of the Ethernets 12 or 36 is also done byprocesses running on the host CPU 10.

It can be seen that the host CPU 10 performs anenormous amount of processing of data, especially if 5-10 clients on each of the two Ethernets are makingfile server requests and need to be sent responses on afrequent basis. The host CPU 10 runs a multitaskingUnix operating system, so each incoming request neednot wait for the previous request to be completelyprocessed and returned before being processed.Multiple processes are activated on the host CPU 10 forperforming different stages of the processing ofdifferent requests, so many requests may be in processat the same time. But there is only one CPU on thecard 10, so the processing of these requests is notaccomplished in a truly parallel manner. The processesare instead merely time sliced. The CPU 10 thereforerepresents a major bottleneck in the processing of file server requests.

Another bottleneck occurs in MMU 11, which must transmit both instructions and data between the CPUcard 10 and the memory 16. All data flowing betweenthe disk drives and the network passes through this interface at least twice.

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -20-

Yet another bottleneck can occur' on the VME bus 20, which must transmit data among the SMD diskcontroller 22, the SCSI host adaptor 26, the host CPUcard 10, and possibly the network H2 controller 34.

PREFERRED EMBODIMENT-OVERALL HARDWARE ARCHITECTURE

In Fig. 2 there is shown a block diagram of anetwork file server 100 according to the invention. It 4k can include multiple network controller (NC) boards,one or more file controller (FC) boards, one or morestorage processor (SP) boards, multiple system memoryboards, and one or more host processors. Theparticular embodiment shown in Fig. 2 includes fournetwork controller boards HOa-llOd, two filecontroller boards 112a-112b, two storage processors114a-114b, four system memory cards 116a-116d for atotal of 192I4B of memory, and one local host processor116. The boards 110, 112, 114, 116 and 118 are connected together over a VME bus 120 on which anenhanced block transfer mode as described in theENHANCED VMEBUS PROTOCOL application identified abovemay be used. Each of the four network controllers 110shown In Fig. 2 can be connected to up to two Ethernets122, for a total capacity of 8 Ethernets 122a-122h.Each of the storage processors 114 operates tenparallel SCSI busses, nine of which can each support up

Attorney Docket No.:AUSP7209WP1/WSW/AUSP/7209.001 8/24/89-7 V. ( -21- to three SCSI disk drives each. The tenth SCSI channelon each of the storage processors 114 is used for tapedrives and other SCSI peripherals.

The host 118 is essentially a standard SunOs Unixprocessor, providing all the standard Sun Open NetworkComputing (ONC) services except NFS and IP routing.Importantly, all network requests to run a user-defined procedure are passed to the host forexecution. Each of the NC boards 110, the FC boards112 and the SP boards 114 includes its own independent32-bit microprocessor. These boards essentially off-load from the host processor 118 virtually all of theNFS and disk processing. Since the vast majority ofmessages to and from clients over the Ethernets 122involve NFS requests and responses, the processing ofthese requests in parallel by the NC, FC and SPprocessors, with minimal involvement by the local host118, vastly improves file server performance. Unix isexplicitly eliminated from virtually all network, file,and storage processing.

OVERALL SOFTWARE ORGANIZATION AND DATA FLOW

Prior to a detailed discussion of the hardware subsystems shown in Fig. 2 , an overview of the software structure will now be undertaken. The software organization is described in more detail in the above-

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -22- identified application entitled MULTIPLE FACILITYOPERATING SYSTEM ARCHITECTURE. „

Most of the elements of the software are well known in the field and are found in most networked Unixsystems, but there are two components which are not:Local NFS ("LNFS") and the messaging kernel ("MK“)operating system kernel. These two components will beexplained first.

The Messaging Kernel. The various processors infile server 100 communicate with each other through theuse of a messaging kernel running on each of theprocessors 110, 112, 114 and 118. These processors donot share any instruction memory, so task-levelcommunication cannot occur via straightforwardprocedure calls as it does in conventional Unix.Instead, the messaging kernel passes messages over VMEbus 120 to accomplish all necessary inter-processorcommunication. Message passing is preferred overremote procedure calls for reasons of simplicity andspeed.

Messages passed by the messaging kernel have afixed 128-byte length. Within a single processor,messages are sent by reference; between processors,

I they are copied by the messaging kernel and thendelivered to the destination process by reference. Theprocessors of Fig. 2 have special hardware, discussed

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -23- below, that can expediently exchange and buffer inter-processor messaging kernel messages.

The LNFS Local NFS interface. The 22-function NFSstandard was specifically designed for statelessoperation using unreliable communication. This meansthat neither clients nor server can be sure if theyhear each other when they talk (unreliability). In practice, in an Ethernet environment, this works * well.

Within the server 100, however, NFS leveldatagrams are also used for communication betweenprocessors, in particular between the networkcontrollers 110 and the file controller 112, andbetween the host processor 118 and the file controller 112. For this internal communication to be both efficient and convenient, it is undesirable andimpractical to have complete statelessness orunreliable communications. Consequently, a modifiedform of NFS, namely LNFS, is used for internalcommunication of NFS requests and responses. LNFS isused only within the file server 100; the externalnetwork protocol supported by the server is preciselystandard, licensed NFS. LNFS is described in more detail below.

The Network Controllers 110 each run an NFS serverwhich, after all protocol processing is done up to the

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 ν.

I -24- NFS layer, converts between external NFS requests andresponses and internal LNFS ^requests and responses.For example, NFS requests arrive as RPC requests withXDR and enclosed in a UDP datagram. After protocolprocessing, the NFS server translates the NFS requestinto LNFS form and uses the messaging kernel to sendthe request to the file controller 112.

The file controller runs an LNFS server whichhandles LNFS requests both from network controllers andfrom the host 118. The LNFS server translates LNFSrequests to a form appropriate for a file systemserver, also running on the file controller, whichmanages the system memory file data cache through ablock I/O layer.

An overview of the software in each of theprocessors will now be set forth.

Network Controller 110

The optimized dataflow of the server 100 beginswith the intelligent network controller 110. Thisprocessor receives Ethernet packets from clientworkstations. It quickly identifies NFS-destinedpackets and then performs full protocol processing onthem to the NFS level, passing the resulting LNFSrequests directly to the file controller 112. Thisprotocol processing includes IP routing and reassembly,

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -25- (JDP demultiplexing, XDR decoding, and NFS request

dispatching. The reverse steps are used to send an NFS reply back to a client. Importantly, these time- consuming activities are performed directly in the

Network Controller 110, not in the host 118.

The server 100 uses conventional NFS ported fromSun Microsystems, Inc., Mountain View, CA, and is NFSprotocol compatible.

Non-NFS network traffic is passed directly to itedestination host processor 118.

The NCs 110 also perform their own IP routing.Each network controller 110 supports two fully parallelEthernets. There are four network controllers in the embodiment of the server 100 shown in Fig. 2, so thatserver can support up to eight Ethernets. For the twoEthernets on the same network controller 110, IProuting occurs completely within the networkcontroller and generates no backplane traffic. Thusattaching two mutually active Ethernets to the samecontroller not only minimizes their inter-net transittime, but also significantly reduces backplanecontention on the VME bus 120. Routing table updates are distributed to the network controllers from the host processor 118, which runs either the gated or routed Unix demon.

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.00 1 8/24/89-7 -26-

While the network controller described here isdesigned for Ethernet LANs, it will be understood thatthe invention can be used just as readily with othernetwork types, including FDDI. 5 File Controller 112

In addition to dedicating a separate processor for NFS protocol processing and IP routing, the server 100 also dedicates a separate processor, the intelligent. * file controller 112, to be responsible ' for all file10 system processing. It uses conventional Berkeley Unix 4.3 file system code and uses a binary-compatible datarepresentation on disk. These two choices allow allstandard file system utilities (particularly block-level tools) to run unchanged. 15 The file controller 112 runs the shared file system used by all NCs 110 and the host processor 118.Both the NCs and the host processor communicate withthe file controller 112 using the LNFS interface. TheNCs 110 use LNFS as described above, while the host 20 processor 118 uses LNFS as a plug-in module to SunOS'sstandard Virtual File System (“VFS") interface.

When an NC receives an NFS read request from aclient workstation, the resulting LNFS request passesto the FC 112. The FC 112 first searches the system 25 memory 116 buffer cache for the requested data. If

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 found, a reference to the buffer is returned to the NC110. If not found, the LRU (least recently used) cachebuffer in system memory 116 is freed and reassigned forthe requested block. The FC then directs the SP 114 to 5 read the block into the cache buffer from a disk drivearray. When complete, the SP so notifies the FC, whichin turn notifies the NC HQ. The NC 110 then sends an NFS NFS reply, client with the data from the workstation out on the buffer, network. back to the Not$ that 10 the SP 114 transfers the data into ι system memory 116, if necessary, and the NC 110 transfers the data from system memory 116 to the networks. The process takesplace without any involvement of the host 118.

Storage Processor 15 The intelligent storage processor 114 manages all disk and tape storage operations. While autonomous,storage processors are primarily directed by the filecontroller 112 to move file data between system memory116 and the disk subsystem. The exclusion of both the 20 host 118 and the FC 112 from the actual data path helpsto supply the performance needed to service manyremote clients.

Additionally, coordinated by a Server Manager inthe host 118, storage processor 114 can execute server 25 backup by moving data between the disk subsystem and

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7

I -28- tape or channels. other archival peripherals on the SCSIFurther, if directly accessed by hostprocessor 118, SP 114 can provide a much higherperformance conventional disk interface for Unix,virtual memory, and databases. In Unix nomenclature,the host processor 118 can mount boot, storage swap,and raw partitions via the storage processors 114.

Each storage processor 114 operates ten parallel,fully synchronous SCSI channels (busses)simultaneously. Nine of these channels support threearrays of nine SCSI disk drives each, each drive in anarray being assigned to a different SCSI channel. Thetenth SCSI channel hosts up to seven tape and otherSCSI peripherals. In addition to performing reads andwrites, SP 114 performs device-level optimizationssuch as disk seek queue sorting, directs device errorrecovery, and controls DMA transfers between thedevices and system memory 116.

Host Processor 118

The local host 118 has three main purposes: to runUnix, to provide standard ONC network services forclients, and to run a Server Manager. Since Unix andONC are ported from the standard SunOS Release 4 andONC Services Release 2, the server 100 can provideidentically compatible high-level ONC services such as

Attorney Docket No. .-AUSP72O9 WPl/WSW/AUSP/7209.001 8/24/89-7

V -29-

the Yellow Pages, Lock Manager, DES Key Authenticator,Auto Mounter, and Port Mapper. Sun/2 Network diskbooting and more general IP internet services such asTelnet, FTP, SMTP, SNMP, and reverse ARP are also 5 supported. Finally, print spoolers and similar Unixdemons operate transparently.

The host processor 118 runs.the following software modules : TCP and socket layers. The Transport Control10 Protocol ("TCP"), which is used for certain serverfunctions other than NFS, provides reliable bytestraamcommunication between two processors. Sockets are used to establish TCP connections. VFS interface. The Virtual File System ("VFS")15 interface Is a standard SunOs file system interface.It paints a uniform file-system picture for both usersand the non-file parts of the Unix operating system, hiding the details of the specific file system. Thusstandard NFS, LNFS, and any local Unix file system can 20 coexist harmoniously. UFS interface. The Unix File System ("UFS")interface is the traditional and well-known Unixinterface for communication with local-to-the-processordisk drives. In the server 100, it is used to 25 occasionally mount .storage processor volumes directly,without going through the file controller 112.

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7

I -30-

Normally, the host 118 uses LNFS and goes through the file controller.

Device layer. The device layer is a standardsoftware interface between the Unix device model anddifferent physical device implementations. In theserver 100, disk devices are not attached to hostprocessors directly, so the disk driver in the host'sdevice layer uses the messaging kernel to communicatewith the storage processor 114.

Route and Port Mapper Demons. The Route and PortMapper demons are Unix user-level background processesthat maintain the Route and Port databases for packetrouting. They are mostly inactive and not in anyperformance path.

Yellow Pages and. Authentication .Demon. The YellowPages and Authentication services are Sun-ONC standardnetwork services. Yellow Pages is a widely usedmultipurpose name-to-name directory lookup service.The Authentication service uses cryptographic keys toauthenticate, or validate, requests to insure thatrequestors have the proper privileges for any actionsor data they desire.

Server Manager. The Server Manager is anadministrative application suite that controlsconfiguration, logs error and performance reports, andprovides a monitoring and tuning interface for the

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -31- system administrator. These functions can be exercised from either system console connected to the host 118, or from a system administrator's workstation.

The host processor 118 is a conventional OEM Suncentral processor card, Model 3E/120. It incorporatesa Motorola 68020 microprocessor and 4MB of on-boardmemory. Other processors, such as a SPARC-basedprocessor, are also possible.

The structure and operation of each of thehardware components of server 100 will now be described in detail.

NETWORK CONTROLLER HARDWARE ARCHITECTURE

Fig. 3 is a block diagram showing the data pathand some control paths for an illustrative one of thenetwork controllers 110a. It comprises a 20 MHz 68020microprocessor 210 connected to a 32-bit microprocessordata bus 212. Also connected to the microprocessordata "bus 212 is a 256K byte CPU memory 214. The loworder 8 bits of the microprocessor data bus 212 areconnected through a bidirectional buffer 216 to an 8-bit slow-speed data bus 218. On the slow-speed databus 218 is a 128K byte EPROM 220, a 32 byte PROM 222,and a multi-function peripheral (MFP) 224. The EPROM220 contains boot code for the network controller 110a,while the PROM 222 stores various operating parameters

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -32-

such as the Ethernet addresses assigned to each of the two Ethernet interfaces on the., board. Ethernet address information is read into the corresponding interfacecontrol block in the CPU memory 214 during 5 initialization. The MFP 224 is a Motorola 68901, andperforms various local functions such as timing,interrupts, and general purpose I/O. The MFP 224 alsoincludes a UART for interfacing to an RS232 port 226.

These functions are not critical to the invention and 10 will not be further described herein.

The low order 16 bits of the microprocessor data bus 212 are also coupled through a bidirectional buffer 230 to a 16-bit LAN data bus 232. A LAN controllerchip 234, such as the Am7990 LANCE Ethernet controller 15 manufactured by Advanced Micro Devices, Inc. Sunnyvale,CA. , interfaces the LAN data bus 232 with the firstEthernet 122a shown in Fig. 2. Control and data forthe LAN controller 234 are stored in a 512K byte LANmemory 236, which is also connected to the LAN data bus 20 232. A specialized 16 to 32 bit FIFO chip 240, referred to herein as a parity FIFO chip and describedbelow, is also connected to the LAN data bus 232. Also

connected to the LAN data bus 232 is a LAN DMA controller 242, which controls movements of packets of 25 data between the LAN memory 236 and the FIFO chip 240.

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/72O9.001 8/24/89-7 -33-

The LAN DMA controller 242 may be a Motorola M68440 DMAcontroller using channel zero'"only.

The second Ethernet 122b shown in Fig. 2 connects to a second LAN data bus 252 on the network controller card 110a shown in Fig. 3. The LAN data bus 252 connects to the low order 16 bits of the microprocessor data bus 212 via a bidirectional buffer250, and has similar components to those appearing onthe LAN data bus 232. In particular, a LAN controller254 interfaces the LAN data bus 252 with the Ethernet122b, using LAN memory 256 for data and control, and aLAN DMA controller 262 controls DMA transfer of databetween the LAN memory 256 and the 16-bit wide dataport A of the parity FIFO 260.

The low order 16 bits of microprocessor data bus212 are also connected directly to another parity FIFO270, and also to a control port of a VME/FIFO DMAcontroller 272. The FIFO 270 is used for passingmessages between the CPU memory 214 and one of theremote boards 110, 112, 114, 116 or 118 (Fig. 2) in a manner described below. The VME/FIFO DMA controller272, which supports three round-robin non-prioritizedchannels for copying data, controls all data transfersbetween one of the remote boards and any of the FIFOs240, 260 or 270, as well as between the FIFOs 240 and 260.

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7

-34- 32-bit data bus 274, which is connected to the 32-bit port B of each of the FIFOs 240, 260 and 270, isthe . data bus over which these transfers take place.Data bus 274 communicates with a local 32-bit bus 276 5 via a bidirectional pipelining latch 278, which is alsocontrolled by VME/FIFO DMA controller 272 which inturn communicates with the VME bus 120 via a bidirectional buffer 280. *

The local data bus 276 is also connected to a set10 of control registers 282, which are directlyaddressable across the VME bus 120. The registers 282are used mostly for system initialization and diagnostics.

The local data bus 276 is also coupled to the15 microprocessor data bus 212 via a bidirectional buffer284. When the NC 110a operates in slave mode, the CPUmemory 214 is directly addressable from VME bus 120.One of the remote boards can copy data directly fromthe CPU memory 214 via the bidirectional buffer 284. 20 LAN memories 236 and 256 are not directly addressed over VME bus 120.

The parity FIFOs 240, 260 and 270 each consist of an ASIC, the functions and operation of which are described in the Appendix. The FIFOs 240 and 260 are 25 configured for packet data transfer and the FIFO 270 is configured for message passing. Referring to the

Attorney Docket No.:AUSP7209WPl/WSW/AUSP/7209.001 8/24/89-7 -35-

Appendix C, the FIFOs 240 and 260 are programmed with thefollowing bit settings in the Data TransferConfiguration Register:

fill Definition S.e.tting 5 0 WD Mode N/A 1 Parity Chip N/A 2 Parity Correct Mode N/A 3 8/16 bits CPU &amp; PortA interface 16 bits (1} 4 Invert Port A address 0 no (0) 10 5 Invert Port A address 1 yes (1) 6 Checksum Carry Wrap yes (1) 7 Reset no (0) The Data Transfer Control Register is programmed as follows: 15 Bit P.efinlti.on Setting 0 Enable PortA Req/Ack yes (1) 1 Enable PortB Req/Ack yes (1) 2 Data Transfer Direction . (as desired) 3 CPU parity enable no (0) 20 4 PortA parity enable no ( 0) 5 PortB parity enable no (0) 6 Checksum Enable yes (1) 7 PortA Master yes (1) Unlike the configuration used on FIFOs 240 and 25 260, the microprocessor 210 is responsible for loading and unloading Port A directly. The microprocessor 210

Attorney Docket No.:AUSP7209WP1/WSW/AUSP/7209.001 8/24/89-7 C. -36- reads an entire 32-bit word from port A with a single

instruction using two port A*· access cycles. Port A data transfer is disabled by unsetting bits 0 (Enable

PortA Req/Ack) and 7 (PortA Master) of the Data

Transfer Control Register.

The remainder of the control settings in FIFO 270 are the same as those in FIFOs 240 and 260 described above.

The NC 110a also includes a command FIFO 290. The command FIFO 290 includes an input port coupled to thelocal data bus 276, and which is directly addressableacross the VME bus 120, and includes an output portconnected to the microprocessor data bus 212. Asexplained in more detail below, when one of the remoteboards issues a command or response to the NC 110a, itdoes so by directly writing a 1-word (32-bit) messagedescriptor into NC 110a's command FIFO 290. CommandFIFO 290 generates a "FIFO not empty" status to themicroprocessor 210, which then reads the messagedescriptor off the top of FIFO 290 and processes it.If the message is a command, then it includes a VMEaddress at which the message is located (presumably anaddress in a shared memory similar to 214 on one of theremote boards). The microprocessor 210 then programsthe FIFO 270 and the VME/FIFO DMA controller 272 to

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 τ

10 15 20 -37- copy the message from the remote location into the CPUmemory 214.

Command FIFO 290 is a conventional two-port FIFO,except that additional circuitry is included forgenerating a Bus Error signal on VME bus 120 if anattempt is made to write to the data input port whilethe FIFO is full. Command FIFO 290 has space for 256 entries . A noteworthy feature of the architecture of NC110a is that the LAN buses 232 and 252 are independentof the microprocessor data bus 212. Data packets beingrouted to or from an Ethernet are stored in LAN memory236 on the LAN data bus 232 (or 256 on the LAN data bus252), and not in the CPU memory 214. Data transferbetween the LAN memories 236 and 256 and the Ethernets122a and 122b, are controlled by LAN controllers 234and 254, respectively, while most data transfer betweenLAN memory 236 or 256 and a remote port on the VME bus120 are controlled by LAN DMA controllers 242 and 262,FIFOs 240 and 260, and VME/FIFO DMA controller 272. Anexception to this rule occurs when the size of the datatransfer- is small, e.g., less than 64 bytes, in whichcase microprocessor 210 copies it directly withoutusing DMA. The microprocessor 210 is not involved inlarger transfers except in initiating them and inreceiving notification when they are complete. 25

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -38-

The CPU memory 214 contains mostly instructionsfor microprocessor 210, messages being transmitted toor from a remote board via FIFO 270, and various datablocks for controlling the FIFOs, the DMA controllersand the LAN controllers. The microprocessor 210accesses the data packets in the LAN memories 236 and256 by directly addressing them through thebidirectional buffers 230 and 250, respectively, forprotocol processing. The local high-speed static RAMin CPU memory 214 can therefore provide zero wait statememory access for microprocessor 210 independent ofnetwork traffic. This is in sharp contrast to theprior art architecture shown in Fig. 1, in which alldata and data packets, as well as microprocessorinstructions for host CPU card 10, reside in the memory 16 and must communicate with the host CPU card 10 via the MMU 11.

While the LAN data buses 232 and 252 are shown asseparate buses in Fig. 3, it will be understood thatthey may instead be implemented as a single combined bus.

NETWORK CONTROLLER OPERATION

In operation, when one of the LAN controllers(such as 234) receives a packet of information over itsEthernet 122a, it reads in the entire packet and stores

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7

V k -39- it in corresponding LAN memory 236. The LAN controller234 then issues an interrupt to microprocessor 210 viaMFP 224, and the microprocessor 210 examines the statusregister on LAN controller 234 (via bidirectionalbuffer 230) to determine that the event causing theinterrupt was a "receive packet completed." In orderto avoid a potential lockout of the second Ethernet122b caused by the prioritized interrupt handlingcharacteristic of MFP 224, the microprocessor 210 doesnot at this time immediately process the receivedpacket; instead, such processing is scheduled for apolling function.

When the polling function reaches the processingof the received packet, control over the packet ispassed to a software link level receive module. Thelink level receive module then decodes the packetaccording to either of two different frame formats:standard Ethernet format or SNAP (IEEE 802 LCC) format.An entry in the header in the packet specifies which frame format was · used. The link level driver then determines which of three types of messages iscontained in the received packet: (1) IP, (2) ARPpackets which can be handled by a local ARP module, or.(3) ARP packets and other packet types which must beforwarded to the local host 118 (Fig. 2) forprocessing. If the packet is an ARP packet which can

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.00 1 8/24/89-7 κ -40- be handled by the NC 110a, such as a request for theaddress of server 100, then the microprocessor 210assembles a response packet in LAN memory 236 and, in aconventional manner, causes LAN controller 234 totransmit that packet back over Ethernet 122a. It isnoteworthy that the data manipulation for accomplishingthis task is performed almost completely in LAN memory236, directly addressed by microprocessor 210 ascontrolled by instructions in CPU memory 214. Thefunction is accomplished also without generating anytraffic on the VME backplane 120 at all, and withoutdisturbing the local host 118.

If the received packet is either an ARP packetwhich cannot be processed completely in the NC 110a, oris another type of packet which requires delivery tothe local host 118 (such as a client request for theserver 100 to execute a client-defined procedure),then the microprocessor 210 programs LAN DMAcontroller 242 to load the packet from LAN memory 236into FIFO 240, programs FIFO 240 with the direction ofdata transfer, and programs DMA controller 272 to readthe packet out of FIFO 240 and across the VME bus 120into system memory 116. In particular, themicroprocessor 210 first programs the LAN DMAcontroller 242 with the starting address and length ofthe packet in LAN memory 236, and programs the

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7

controller to begin transferring data from the LAN πζπη j χ

memory 236 to port Λ of parity FIFO 240 as soon as theFIFO is ready to receive data. Second, microprocessor210 programs the VME/FIFO DMA controller 272 with the5 destination address in system memory 116 and the lengthof the data packet, and instructs the controller tobegin transferring data from port B of the FIFO 260onto VME bus 120. Finally, the microprocessor 210programs FIFO 240 with the direction of the transfer to10 take place. The transfer then proceeds entirely underthe control of DMA controllers 242 and 262, without anyfurther involvement by microprocessor 210.

The microprocessor 210 then sends a message tohost 118 that a packet is available at a specified15 system memory address. The microprocessor 210 sendssuch a message by writing a message descriptor to asoftware-emulated command FIFO on the host, whichcopies the message from CPU memory 214 on the NC viabuffer 284 and into the host's local memory, in20 ordinary VME block transfer mode. The host then copiesthe packet from system memory 1.16 into the host's ownlocal memory using ordinary VME transfers.

If the packet received by NC 110a from the networkis an IP packet, then the microprocessor 210 determines25 whether it is (1) an IP packet for the server 100 whichis not an NFS packet; (2) an IP packet to be routed to

Attorney Docket No.:AUSP72O9 WP1/WSW/AUSP/7209.001 8/24/89-7 -42- a different network; or (3) an NFS packet. If it is anIP packet for the server 100 but not an NFS packet,then the microprocessor 210 causes the packet to betransmitted from the LAN memory 236 to the host 118 inthe same manner described above with respect to certainARP packets.

If the IP packet is not intended for the server100, but rather is to be routed to a client on adifferent network, then the packet is copied into theLAN memory associated with the Ethernet to which thedestination client is connected. If the destination client is on the Ethernet 122b, which is on the same NC board as the source Ethernet 122a, then themicroprocessor 210 causes the packet to be copied fromLAN memory 236 into LAN 256 and then causes LANcontroller 254 to transmit it over Ethernet 122b. (Ofcourse, if the two LAN data buses 232 and 252 arecombined, then copying would be unnecessary; themicroprocessor 210 would simply cause the LANcontroller 254 to read the packet out of the samelocations in LAN memory to which the packet waswritten by LAN controller 234.)

The copying of a packet from LAN memory 236 to LANmemory 256 takes place similarly to the copyingdescribed above from LAN memory to system memory. Fortransfer sizes of 64 bytes or more, the microprocessor

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 V. -43- 210 first programs the LAN DMA controller 242 with thestarting address and length of. the packet in LAN memory236, and programs the controller to begin transferringdata from the LAN memory 236 into port A of parity FIFO240 as soon as the FIFO is ready to receive data.Second, microprocessor 210 programs the LAN DMAcontroller 262 with a destination address in LAN memory256 and the length of the data packet, and instructsthat controller to transfer data from parity FIFO 260into the LAN memory 256. Third, microprocessor 210programs the VME/FIFO DMA controller 272 to clock wordsof data out of port B of the FIFO 240, over the databus 274, and into port B of FIFO 260. Finally, themicroprocessor 210 programs the two FIFOs 240 and 260with the direction of the transfer to take place. Thetransfer then proceeds entirely under the control ofDMA controllers 242, 262 and 272, without any furtherinvolvement by the microprocessor 210. Like thecopying from LAN memory to system memory, if thetransfer size is smaller than 64 bytes, themicroprocessor 210 performs the transfer directly, without DMA.

When each of the LAN DMA controllers 242 and 262 complete their work, they so notify microprocessor 210by a respective interrupt provided through MFP 224.When the microprocessor 210 has received both

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -44- interrupts, it programs LAN controller 254 to transmitthe packet on the Ethernet 122b in a conventional manner.

Thus, IP routing between the two Ethernets in a 5 single network controller 110 takes place over databus 27 4, generating no traffic over VME bus 120. Noris the host processor 118 disturbed for such routing,in contrast to the prior art architecture of Fig. 1.Moreover, all but the shortest copying «work is 10 performed by controllers outside microprocessor 210,•requiring the involvement of the microprocessor 210,and bus traffic on microprocessor data bus 212, onlyfor the supervisory functions of programming the DMAcontrollers and the parity FIFOs and instructing them 15 to begin. The VME/FIFO DMA controller 272 isprogrammed by loading control registers viamicroprocessor data bus 212; the LAN DMA controllers242 and 262 are programmed by loading control registerson the respective controllers via the microprocessor 20 data bus 212, respective bidirectional buffers 230 and250, and respective LAN data buses 232 and 252, and theparity FIFOs 240 and 260 are programmed as set forth in the Appendix C. ΐ

I

If the destination workstation of the IP packet to 25 be routed is on an Ethernet connected to a differentone of the network controllers 110, then the packet Is

Attorney Docket No.:AUSP7209WPl/WSW/AUSP/7209.001 8/24/89-7 -45- copied into the appropriate LAN memory on the NC 110 towhich that Ethernet is connected. Such copying isaccomplished by first copying the packet into systemmemory 116, in the manner described above with respectto certain ARP packets, and then notifying thedestination NC that a packet is available. When an NCis so notified, it programs its own parity FIFO and DMAcontrollers to copy the packet from system memory 116into the appropriate LAN memory. It is noteworthy thatthough this type of IP routing does create VME bustraffic, it still does not involve the host CPU 118.

If the IP packet received over the Ethernet 122aand now stored in LAN memory 236 is an NFS packetintended for the server 100, then the microprocessor210 performs all necessary protocol preprocessing toextract the NFS message and convert it to the local NFS(LNFS) format. This may well involve the logicalconcatenation of data extracted from a large number ofindividual IP packets stored in LAN memory 236,resulting in a linked list, in CPU memory 214, pointingto the different blocks of data in LAN memory 236 inthe correct sequence.

The exact details of the LNFS format are not important for an understanding of the invention, except to note that it includes commands to maintain a directory of files which are stored on the disks

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -46- attached to the storage processors 114, commands forreading and writing data to ^and from a file on thedisks, and various configuration management anddiagnostics control messages. The directorymaintenance commands which are supported by LNFSinclude the following messages based on conventionalNFS: get attributes of a file (GETATTR); set attributesof a file (SETATTR) ; look up a file (LOOKUP); created afile (CREATE); remove a file (REMOVE); rename a file(RENAME); created a new linked file (LINK); create asymlink (SYMLINK); remove a directory (RMDIR); andreturn file system statistics (STATFS). The datatransfer commands supported by LNFS include read from afile (READ); write to a file (WRITE); read from adirectory (READDIR); and read a link (READLINK) . LNFSalso supports a buffer release command (RELEASE), fornotifying the file controller that an NC is finishedusing a specified buffer in system memory. It alsosupports a VOP-derived access command, for determiningwhether a given type access is legal for specifiedcredential on a specified file.

If the LNFS request includes the writing of filedata from the LAN memory 236 to disk, the NC 110a firstrequests a buffer in system memory 116 to be allocatedby the appropriate FC 112. When a pointer to thebuffer is returned, microprocessor 210 programs LAN DMA

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 k. -47-

controller 242, parity FIFO 240 and VME/FIFO DMA controller 272 to transmit the entire block of filedata to system memory 116. The only difference betweenthis transfer and the transfer described above for transmitting IP packets and ARP packets to systemmemory 116 is that these data blocks will typicallyhave portions scattered throughout LAN memory 236. Themicroprocessor 210 accommodates that situation byprogramming LAN DMA controller 242 successively foreach portion of the data, in accordance with the linkedlist, after receiving notification that the previousportion is complete. The microprocessor 210 canprogram the parity FIFO 240 and the VME/FIFO DMAcontroller 272 once for the entire message, as long asthe entire data block is to be placed contiguously insystem memory 116. If it is not, then themicroprocessor 210 can program the DMA controller 272

for successive blocks in the same manner LAN DMA controller 242.

If the network controller 110a receives a messagefrom another processor in server 100, usually from filecontroller 112,. that file data is available in systemmemory 116 for transmission on one of the Ethernets,for example Ethernet 122a, then the network controller110a copies the file data into LAN memory 236 in amanner similar to the copying of file data in the

Attorney Docket No. :ACJSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -48-

opposite direction. In particular, the microprocessor210 first programs VME/FIFO DMA controller 272 with thestarting address and length of the data in systemmemory 116, and programs the controller to begin 5 transferring data over the VME bus 120 into port B ofparity FIFO 240 as soon as the FIFO is ready to receivedata. The microprocessor 210 then programs the LAN DMAcontroller 242 with a destination address in LAN memory236 and then length of the file data, and instructs 10 that controller to transfer data from the parity FIFO240 into the LAN memory 236. Third, microprocessor210 programs the parity FIFO 240 with the direction ofthe transfer to take place. The transfer then proceeds entirely under the control of E >MA controllers 242 and 15 272, without any further involvement by the microprocessor 210. Again, if the file data is scattered in multiple blocks in system memory 116, the microprocessor 210 programs the VME/FIFO DMA controller272 with a linked list of the blocks to transfer in the 20 proper order.

When each of the DMA controllers 242 and 262complete their work, they so notify microprocessor 210through MFP 224 . The microprocessor 210 then performsall necessary protocol processing on the LNFS message 25 in LAN memory 236 in order to prepare the message fortransmission over the Ethernet 122a in the form of

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.00 1 8/24/89-7 k -49-

Ethernet IP packets. As set forth above, this protocolprocessing is performed entirely in network controller110a, without any involvement of the local host 118.

It should be noted that the parity FIFOs aredesigned to move multiples of 128-byte blocks mostefficiently. The data transfer size through port B isalways 32-bits wide, and the VME address correspondingto the 32-bit data must be quad-byte aligned. The datatransfer size for port £ can be either 8 or 16 bits.For bus utilization reasons, it is set to 16 bits whenthe corresponding local start address is double-bytealigned, and is set at 8 bits otherwise. The TCP/IPchecksum is always computed in the 16 bit mode.Therefore, the checksum word requires byte swapping ifthe local start address is not double-byte aligned.

Accordingly, for transfer from port B to port A ofany of the FIFOs 240, 260 or 270, the microprocessor210 programs the VME/FIFO DMA controller to pad thetransfer count to the next 128-byte boundary. Theextra 32-bit word transfers do not involve the VME bus,and only the desired number of 32-bit words will beunloadedfrom port A.

For transfers from port A to port B of the parityFIFO 270, the microprocessor 210 loads port A word-by-word and forces a FIFO full indication when it isfinished. The FIFO full indication enables unloading

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -50-

from port B. The same procedure also takes place fortransfers from port A to port &amp; of either of the parityFIFOs 240 or 260, since transfers of fewer than 128bytes are performed under local microprocessor controlrather than under the control of LAN DMA controller 242 or 262. For all of the FIFOs, the VME/FIFO DMAcontroller is programmed to unload only the desired number of 32-bit words.

FILE CONTROLLER HARDWARE ARCHITECTURE 10 The file controllers (FC) 112 may each be a standard off-the-shelf microprocessor board, such asone manufactured by Motorola Inc. Preferably,however, a more specialized board is used such as thatshown in block diagram form in Fig. 4. 15 Fig. 4 shows one of the FCs 112a, and it will be understood that the other FC can be identical. In manyaspects it is simply a scaled-down version of the NC110a shown in Fig. 3, and in some respects it is scaledup. Like the NC 110a, FC 112a comprises a 20MHz 68020 20 microprocessor 310 connected to a 32-bit microprocessordata bus' 312. Also connected to the microprocessordata bus 312 is a 256K byte shared CPU memory 314.The low order 8 bits of the microprocessor data bus 312are connected through a bidirectional buffer 316 to an .25 8-bit slow-speed data bus 318. On slow-speed data bus

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -51-

318 ace a 128K byte FROM 320, and a multifunctionperipheral (MFP) 324. The functions of the FROM 320and MFF 324 are the same as those described above withrespect to EPROM 220 and MFP 224 on NC 110a. FC 112a 5 does not include PROM like the PROM 222 on NC 110a,but does include a parallel port 392. The parallelport 392 is mainly for testing and diagnostics.

Like the NC 110a, the FC 112a is connected to the VME bus 120 via a bidirectional buffer 380 and a 32- 10 bit local data bus 376. A set of control registers. 382are connected to the local data bus 376, and directlyaddressable across the VME bus 120. The local data bus 376 is also coupled to the microprocessor data bus 312via a bidirectional buffer 384. This permits the 15 direct addressability of CPU memdry 314 from VME bus 120 .

FC 112a also includes, a command FIFO 390, whichincludes an input port coupled to the local data bus376 and which is directly addressable across the VME 20 bus 120. The command FIFO 390 also includes an outputport connected to the microprocessor data bus 312. Thestructure, operation and purpose of command FIFO 390are the same as those described above with respect to command FIFO 290 on NC 110a. 25 The FC 112a omits the LAN data buses 232 and 252 which’ are present in NC 110a, but instead includes a 4

Attorney Docket No.sAUSP7209WPl/WSW/AUSP/7209.00 1 8/24/89-7 c c -52- megabyte 32-bit wide FC memory 396 coupled to themicroprocessor data bus 312 via a bidirectional buffer394. As will be seen, FC memory 396 is used as a cachememory for file control information, separate from thefile data information cached in system memory 116.

The file controller embodiment shown in Fig. 4does not include any DMA controllers, and hence cannotact as a master for transmitting or receiving data inany block transfer mode, over the VME bus 120. Blocktransfers do occur with the CPU memory 314 and the FCmemory 396, however, with the FC 112a acting as an VMEbus slave. In such transfers, the remote masteraddresses the CPU memory 314 or the FC memory 396directly over the VME bus 120 through the bidirectionalbuffers 384 and, if appropriate, 394.

FILE CONTROLLER OPERATION

The purpose of the FC 112a is basically to providevirtual file system services in response to requestsprovided in LNFS format by remote processors on theVME bus 120. Most requests will come from a networkcontroller 110, but requests may also come from the local host 118.

The file related commands supported by LNFS areidentified above. They are all specified to the FC112a in terms of logically identified disk data blocks.

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -53-

For example, the LNFS command for reading data from afile includes a specification df the file from which toread (file system ID (FSID) and file ID (inode)), abyte offset, and a count of the number of bytes toread. The FC 112a converts that identification intophysical form, namely disk and sector numbers, in orderto satisfy the command.

The FC 112a runs a conventional Fast File System(FFS or UFS), which is based on the Berkeley 4.3 VAXrelease. This code performs the conversion and alsoperforms all disk data caching and control datacaching. However, as previously mentioned, controldata caching is performed using the FC memory 395 on FC112a, whereas disk data caching is performed using thesystem memory 116 (Fig. 2). Caching this file controlinformation within the FC 112a avoids the VME buscongestion and speed degradation which would result iffile control information was cached in system memory116. "

The memory on the FC 112a is directly accessedover the VME bus 120 for three main purposes. First,and by far the most frequent, are accesses to FC memory396 by an SP 114 to read or write cached file controlinformation. These are accesses requested by FC 112ato write locally modified file control structuresthrough to disk, or to read file control structures

Attorney Docket No. :AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -54- from disk. Second, the FC's CPU memory 314 is accesseddirectly by other processors for message transmissionsfrom the FC 112a to such other processors. Forexample, if a data block in system memory is to betransferred to an SP 114 for writing to disk, the FC112a first assembles a message in its local memory 314requesting such a transfer. The FC 112a then notifiesthe SP 114, which copies the message directly from theCPU memory 314 and executes the requested transfer. A third type of direct access to the FC's local memory occurs when an LNFS client reads directory entries. When FC 112a receives an LNFS request to read directory entries, the FC 112a formats the requested directory entries in FC memory 396 and notifies the requestor of their location. The requestor then directly accesses FC memory 396 to read the entries .

The version of the UFS code on FC 112a includes some modifications in order to separate the two caches.In particular, two sets of buffer headers aremaintained, one for the FC memory 396 and one for thesystem memory 116. Additionally, a second set of thesystem buffer routines (GETBLK(), BRELSE(), BREAD(),BWRITE(), and BREADA()) exist, one for buffer accessesto FC Mem 396 and one for buffer accesses to systemmemory 116. The UFS code is further modified to call

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -55- the appropriate buffer routines for FC memory 396 foraccesses to file control information, and to call theappropriate buffer routines for the system memory 116for the caching of disk data. A description of UFS maybe found in chapters 2, 6, 7 and 8 of "Kernel Structureand Flow," by Rieken and Webb of . sh consulting (SantaClara, California: 1988), incorporated herein by reference.

When a read command is sent to the FC by arequestor such as a network controller, the FC firstconverts the file, offset and count information intodisk and sector information. It then locks the systemmemory buffers which contain that information,instructing the storage processor 114 to read them fromdisk if necessary. When the buffer is ready, the FCreturns a message to the requestor containing both theattributes of the designated file and an array ofbuffer descriptors that identify the locations insystem memory 116 holding the data.

After the requestor has read the data out of thebuffers, it sends a release request back to the FC.The. release request is the same message that wasreturned by the FC in response to the read request; the FC 112a uses the information contained therein to determine which buffers to free.

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 ν. V.

ίο 15 20 -56- A write command is processed by FC 112a similarlyto the read command, but the caller is expected towrite to (instead of read from) the locations in systemmemory 116 identified by the buffer descriptorsreturned by the FC 112a. Since FC 112a employs write-through caching, when it receives the release commandfrom the requestor, it instructs storage processor 114to copy the data from system memory 116 onto theappropriate disk sectors before freeing the systemmemory buffers for possible reallocation.

The READDIR transaction is similar to read and write, but the request is satisfied by the FC 112adirectly out of its own FC memory 396 after formattingthe requested directory information specifically forthis purpose. The FC 112a causes the storageprocessor read the requested directory information fromdisk if it is not already locally cached. Also, thespecified offset is a "magic cookie" instead of a byteoffset, identifying directory entries instead of anabsolute byte offset into the file. No file attributes are returned.

The· READLINK transaction also returns no file attributes, and since links are always read in theirentirety, it does not require any offset or count.

For all of the disk data caching performed throughsystem memory 116, the FC 112a acts as a central

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 25 -57-

10 15 20 authority for dynamically allocating, deallocating andkeeping track of buffers. If there are two or more FCs112, each has exclusive control over its own assignedportion of system memory 116. In all of thesetransactions, the requested buffers are locked duringthe period between the initial request and the releaserequest. This prevents corruption of the data by other clients.

Also in the situation where there are two or more FCs, each file system on the disks is assigned to aparticular one of the FCs. FC #0 runs a process calledFC_VICE_PRESIDENT, which maintains a list of which filesystems are assigned to which FC. When a clientprocessor (for example an NC 110) is about to make anLNFS request designating a particular file system, itfirst sends the fsid in a message to theFC_VICE_PRESIDENT asking which FC controls thespecified file system. The FC_VICE_PRESIDENT responds,and the client processor sends the LNFS request to thedesignated FC. The client processor also maintains itsown list of fsid/FC pairs as it discovers them, so asto minimize the number of such requests to theFC_VICE_PRESIDENT.

STORAGE PROCESSOR HARDWARE ARCHITECTURE

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -sa-

in the file server 100; each of the storageprocessors 114 can Interface the VME bus 120 with up to10 different SCSI buses. Additionally, it can do so atthe full usage rate of an enhanced block transfer 5 protocol of S5MB per second.

Fig. 5 is a block diagram of one of the SPs 114a.SP 114b is identical. SP 114a comprises amicroprocessor 510, which may be a Motorola 68020microprocessor operating at 20MHz. The microprocessor 10 510 is coupled over a 32-bit microprocessor data bus 512 with CPU memory 514, which may include up to 1MB ofstatic RAM. The microprocessor 510 accessesInstructions, data and status on its own private bus512, with no contention from any other source. The 15 microprocessor 510 is the only master of bus 512.

The low order 16 bits of the microprocessor data bus 512 interface with a control bus 516 via a bidirectional buffer 518. The low order 8 bits of the control bus 516 interface with a slow speed bus 520 via 20 another bidirectional buffer 522. The slow speed bus520 connects to an MFP 524, similar to the MFP 224 inNC 110a (Fig. 3), and with a PROM 526, similar to PROM220 on NC 110a. The PROM 526 comprises 128K bytes of EPROM which contains the functional code for SP 114a. 25 Due to the width and speed of the PROM 526, the

Attorney Docket No.:AUSP7209WPl/WSW/AUSP/7209.00 1 8/24/89-7 -59-

10 15 functional code is copied to CPU memory 514 upon reset for faster execution. MPP 524, like the MFP 224 on NC 110a, comprises aMotorola 68901 multifunction peripheral device. Itprovides the functions of a vectored interruptcontroller, individually programmable I/O pins, fourtimers and a UART. The UART functions provide serialcommunications across an RS 232 bus (not shown in Fig.5) for debug monitors and diagnostics. Two of the fourtiming functions may be used as general-purpose timersby the microprocessor 510, either independently or incascaded fashion. A third timer function provides therefresh clock for a DMA controller described below, andthe fourth timer generates the UART clock. Additionalinformation on the MFP 524 can be found in "MC 68901Multi-Function Peripheral Specification," by Motorola,Inc., which is incorporated herein by reference.

The eight general-purpose I/O bits- provided by MFP524 are configured according to the following table:

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 X.. -60-

Power Failure is Imminent functions as an early warning.

Blt Direction Definition 7 input

This input input output SCSI Attention - A composite of theSCSI. Attentions from all 10 SCSIchannels .

Channel Operation Done - A composite ofthe channel done bits from all 13channels of the DMA controller,described below. DMA Controller Enable.Controller to run.

Enables the DMA input VMEbus Interrupt Done - Indicates thecompletion of a VMEbus Interrupt. input Command Available - Indicates that theSP'S Command Fifo, described below,contains one or more command pointers. output External Interrupts Disable. Disablesexternally generated interrupts to themicroprocessor 510. output Command Fifo Enable. Enables operationof the SP'S Command Fifo. Clears theCommand Fifo when reset. bus

Commands are provided to the SP 114a.120 via a bidirectional buffer 530,

from the VME a local data bus 532, and a command FIFO 534. The command FIFO 534 is similar to the command FIFOs 290 and 390 on NC 110a and FC 112a, respectively, and has a depth of 256 32-bit entries. The command FIFO 534 is a write-onlyregister as seen on the VME bus 120, and as a read-only register as seen by microprocessor 510. If theFIFO is full at the beginning of a write from the VMEbus, a VME bus error is generated. Pointers are

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -61-

10 15 20 removed from the command FIFO 534 in the order received, and only by the microprocessor 510. Command available status is provided through I/O bit 4 of the MFP 524, and as a long as one or more command pointers are still within the command FIFO 534, the command available status remains asserted.

As previously mentioned, the SP 114a supports upto 10 SCSI buses or channels 540a-540j. In the typicalconfiguration, buses 540a-540i support up to 3 SCSIdisk drives each, and channel 540j supports other SCSIperipherals such as tape drives, optical disks, and soon. Physically, the SP 114a connects to each of the SCSI buses with an ultra-miniature D sub connector and round shielded cables. Six 50-pin cables provide 300conductors which carry 18 signals per bus and 12grounds. The cables attach at the front panel of the SP 114a and to a commutator board at the disk drive array. Standard 50-pin cables connect each SCSI device to the commutator board. Termination resistors are installed on the SP 114a.

The SP 114a supports synchronous parallel datatransfers, up to 5MB per second on each of the SCSIbuses 540, arbitration, and disconnect/reconnect services. Each SCSI bus 540 is connected to arespective SCSI adaptor 542, which in the presentembodiment is an AIC 6250 controller IC manufactured by 25

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -62-

Adaptec Inc., Milpitas, California, operating in thenon-multiplexed address bus snode. The AIC 6250 isdescribed in detail in HAIC-6250 FunctionalSpecification," by Adaptec Inc., which is incorporatedherein by reference. The SCSI adaptors 542 each providethe necessary hardware interface and low-levelelectrical protocol to implement its respective SCSIchannel.

The 8-bit data port of each of the SCSI adaptors542 is connected to port A of a respective one of a setof ten parity FIFOs 544a-544 j. The FIFOs 544 are thesame as FIFOs 240, 260 and 270 on NC 110a, and areconnected and configured to provide parity covered datatransfers between the 8-bit data port of the respectiveSCSI adaptors 542 and a 36-bit (32-bit plus 4 bits ofparity) common data bus 550. The FIFOs 544 providehandshake, status, word assembly/disassembly and speedmatching FIFO buffering for this purpose. The FIFOs544 also generate and check parity for the 32-bit bus,and for RAID 5 implementations they accumulate and check redundant data and accumulate recovered data.

All- of the SCSI adaptors 542 reside at a singlelocation of ‘the address space of the microprocessor510, as do all of the parity FIFOs 544. Themicroprocessor 510 selects individual controllers andFIFOs for access in pairs, by first programming a pair

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -63-

select register (not shown) to point to the desiredpair and then reading from or writing to the controlregister address of the desired chip in the pair. Themicroprocessor 5 10 communicates with the control 5 registers on the SCSI adaptors 542 via the control bus516 and an additional bidirectional buffer 546, andcommunicates with the control registers on FIFOs 544via the control bus 516 and a bidirectional buffer 552.Both the SCSI adaptors 542 and FIFOs 544 employ 8-bit 10 control registers, and register addressing of the FIFOs544 is arranged such that such registers alias inconsecutive byte locations. This allows themicroprocessor 510 to write to the registers as asingle 32-bit register, thereby reducing instruction 15 overhead.

The parity FIFOs 544 are each configured in theirAdaptec 6250 mode. Referring to the Appendix C,>theFIFOs 54 4 are programmed with the following bitsettings in the Data Transfer Configuration Register: 20 Bit Definition Setting 0 WD Mode (0) 1 Parity Chip (1) 2 Parity Correct Mode (0) 3 8/16 bits CPU &amp; PortA interface (0) 25 4 Invert Port A address 0 (1) 5 Invert Port A address 1 (1) Attorney Docket Mo.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89 6 -64- Checksum Carry Wrap β (0) 7 Reset >. (0) The Data Transfer Control Register is programmed as follows : Bit Definition Settino . 0 Enable PortA Req/Ack (1) 1 Enable PortB Req/Ack (1) 2 Data Transfer Direction as desired 3 CPU parity enable (0) 4 PortA parity enable (1) 5 PortB parity enable (1) 6 Checksum Enable (0) 7 PortA Master (0) In addition, bit 4 of the RAM Access Control

Register (Long Burst) is programmed for 8-byte bursts.SCSI adaptors 542 each generate a respective interrupt signal, the status of which are provided tomicroprocessor 510 as 10 bits of a 16-bit SCSIinterrupt register 556. The SCSI interrupt register 556 is connected to the control bus 516.Additionally, a composite SCSI interrupt is providedthrough the MFP 524 whenever any one of the SCSIadaptors 542 needs servicing.

An additional parity FIFO 554 is also provided inthe SP 114a, for message passing. Again referring to

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -65- the Appendix C, the parity FIFO 554 is programmed withthe following bit settings in the Data TransferConfiguration Register: fiiX Pgfjpj.tig.n Sst-ting 5 0 WD Mode (0) 1 Parity Chip (1) 2 Parity Correct Mode (0) 3 8/16 bits CPU &amp; PortA Interface (1, 4 Invert Port A address 0 CD- io 5 Invert Port A address 1 (1) 6 Checksum Carry Wrap (0) 7 Reset • *.* (0) ft The Data Transfer Control Register Is programmed as follows: 15 ait Definition Satting 0 Enable PortA Req/Ack (0) 1 Enable PortB Req/Ack (1) 2 Data Transfer Direction as desired 3 CPU parity enable (0) 20 4 PortA parity enable (0) 5 PortB parity enable (1) 6 Checksum Enable (0) 7 PortA Master (0)

In addition, bit 4 of the RAM Access Control25 Register (Long Burst) is programmed for 8-byte bursts.

Attorney Docket No.:AUSP7209 ! WPl/WSW/AUSP/7209.001 8/24/89-7 i ' ί i. I ’ -66-

Port A of FIFO 554 is connected to the 16-bit control bus 516, and port B is connected to the commondata bus 550. FIFO 554 provides one means by which themicroprocessor 510 can communicate directly with theVME bus 120, as is described in more detail below.

The microprocessor 510 manages data movementusing a set of 15 channels, each of which has an uniquestatus which indicates its current state. Channels areimplemented using a channel enable register 560 and achannel status register 562, both connected to thecontrol bus 516. The channel enable register 560 is a16-bit write-only register, whereas the channel statusregister 562 is a 16-bit read-only register. The tworegisters reside at the same address to microprocessor510. The microprocessor 510 enables a particularchannel by setting its respective bit in channel enableregister 560, and recognizes completion of thespecified operation by testing for a "done" bit in thechannel status register 562. The microprocessor 510then resets the enable bit, which causes the respective"done" bit in the channel status register 562 to be cleared.'

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -67-

The channels are defined as follows:

CHANNEL FUNCTION 0:9 These channels control data movement to and from the respective FIFOs 544 via the commondata bus 550. When a FIFO is enabled and arequest is received from it, the channelbecomes ready. Once the channel has beenserviced a status of done is generated. 11:10 These channels control data movement betweena local data buffer 564, described below,and the VME bus 120. When enabled thechannel becomes ready. Once the channel hasbeen serviced a status of done is generated. 12 When enabled, this channel causes the DRAM inlocal data buffer 564 to be refreshed basedon a clock which is generated by the MFP 524.The refresh consists of a burst of 16 rows.This channel does not generate a status ofdone. 13 The microprocessor's communication FIFO 554is serviced by this channel. When enable isset and the FIFO 554 asserts a request thenthe- channel becomes ready. This channelgenerates a status of done. 14 Low latency writes from microprocessor 510onto the VME bus 120 are controlled by thischannel. When this channel is enabled data ismoved from a special 32 bit register,described below, onto the VME bus 120. Thischannel generates a done status. 15 This is a null channel for which neither aready status nor done status is generated.

Channels are. prioritized to allow servicing of the more critical requests first. Channel priority is assigned in a descending order starting at channel 14.

That is, in the event that all channels are requesting service, channel 14 will be the first one served.

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -68-

The common data bus 550 is coupled via abidirectional register 570 to a 36-bit junction bus572. Λ second bidirectional register 574 connects thejunction bus 572 with the local data bus 532. Local 5 data buffer 564, which comprises 1MB of DRAM, withparity, is coupled bidirectionally to the junction bus572. It is organized to provide 256K 32-bit words withbyte parity. The SP 114a operates the DRAMs in pagemode to support a very high data rate, which requires 10 bursting of data instead of random single-wordaccesses. It will be seen that the local data buffer564 is used to implement a RAID (redundant array ofinexpensive disks) algorithm, and is not used fordirect reading and writing between the VME bus 120 and 15 a peripheral on one of the SCSI buses 540. A read-only register 576, containing all zeros, Is also connected to the junction bus 572. This registeris used mostly for diagnostics, initialization, andclearing of large blocks of data in system memory 116. 20 The movement of data between the FIFOs 544 and

554 , the local data buffer 564, and a remote entitysuch as the system memory 116 on the VME bus 120, isall controlled by a VME/FIFO DMA controller 580. TheVME/FIFO DMA controller 580 is similar to the VME/FIFO 25 DMA controller 272 on network controller 110a (Fig. 3),and is described in the Appendix A. Briefly, it

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -69-

10 15 20 includes a bit slice engine 582 and a dual-port staticRAM 584. One port of the dual-port static RAM 584communicates over the 32-bit microprocessor data bus512 with microprocessor 510, and the other portcommunicates over a separate 16-bit bus with the bitslice engine 582. The microprocessor 510 placescommand parameters in the dual-port RAM 584, and usesthe channel enables 560 to signal the VME/FIFO DMAcontroller 580 ’to proceed with the command. TheVME/FIFO DMA controller is responsible for scanning thechannel status and servicing requests, and returningending status in the dual-port RAM 584. The dual-portRAM 584 is organized as IK x 32 bits at the 32-bit portand as 2K x 16 bits at the 16-bit port. An example showing the method by which the microprocessor 510controls the VME/FIFO DMA controller 580 is as follows.First, the microprocessor 510 writes into the dual-portRAM 584 the desired command and associated parametersfor the desired channel. For example, the commandmight be, "copy a block of data from FIFO 544h out intoa block of system memory 116 beginning at a specifiedVME address." Second, the microprocessor sets thechannel enable bit in channel enable register 560 for the desired channel.

At the time the channel enable bit is set, theappropriate FIFO may not yet be ready to send data. 25

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -70-

Only when the VME/FIFO DMA controller 580 does receive a "ready" status from the channel, will the controller

580 execute the command. In the meantime, the DMA controller 580 is free to execute commands and move data to or from other channels.

When the DMA controller 580 does receive a statusof "ready" from the specified channel, the controllerfetches the channel command and parameters from thedual-ported RAM 584 and executes. When the command iscomplete, for example all the requested data has beencopied, the DMA controller writes status back into thedual-port RAM 584 and asserts "done" for the channel inchannel status register 562. The microprocessor 510 isthen interrupted, at which time it reads channel statusregister 562 to determine which channel interrupted.The microprocessor 510 then clears the channel enablefor the appropriate channel and checks the endingchannel status in the dual-port RAM 584. 'In this way a high-speed data transfer can takeplace under the control of DMA controller 580, fully inparallel with other activities being performed bymicroprocessor 510. The data transfer takes place overbusses different from microprocessor data bus 512,thereby avoiding any interference with microprocessor instruction fetches.

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -71-

10 15 20

The SP 114a also includes a high-speed register590, which is coupled between., the microprocessor databus 512 and the local data bus 532. The high-speedregister 590 is used to write a single 32-bit word toan VME bus target with a minimum of overhead. Theregister is write only as viewed from themicroprocessor 510. In order to write a word onto theVME bus 120, the microprocessor 510 first writes theword into the register 590, and the desired VME targetaddress into dual-port RAM 584. When themicroprocessor 510 enables the appropriate channel inchannel enable register 560, the DMA controller 580transfers the data from the register 590 into the VMEbus. address specified in the dual-port RAM 584. TheDMA controller 580 then writes the ending status to thedual-port RAM and sets the channel "done" bit inchannel status register 562.

This procedure is very efficient for transfer of asingle word of data, but becomes inefficient for largeblocks of data. Transfers of greater than one word ofdata, typically for message passing, are usuallyperformed using the FIFO 554.

The SP 114a also includes a series of registers592, similar to the registers 282 on NC 110a (Fig. 3)and the registers 382 on FC 112a (Fig. 4). The details 25

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -72- of these registers are not important for an understanding of the present invention.

STORAGE PROCESSOR OPERATION

The 30 SCSI disk drives supported by each of theSPs 114 are visible to a client processor, for exampleone of the file controllers 112, either as three large,logical disks or as 30 independent SCSI drives,depending on configuration. When the drives arevisible as three logical disks, the SP uses RAID 5design algorithms to distribute data for each logicaldrive on nine physical drives to minimize disk armcontention. The tenth drive is left as a spare. TheRAID 5 algorithm (redundant array of inexpensivedrives, revision 5) is described in "A Case For aRedundant Arrays of Inexpensive Disks (RAID)", byPatterson et al., published at ACM SIGMOD. Conference,Chicago, Ill., June 1-3, 1988, incorporated herein by reference.

In the RAID 5 design, disk data are divided intostripes. Data stripes are recorded sequentially oneight different disk drives. A ninth parity stripe, theexclusive-or of eight data stripes, is recorded on aninth drive. If a stripe size is set to 8K bytes, aread of 8K of data involves only one drive. A write of 8K of data involves two drives: a data drive and a

Attorney Docket No.:AUSP7209WP1/WSW/AUSP/7209.001 8/24/89-7 i

V -73-

10 15 20 parity drive. Since a write requires the reading backof old data to generate a new parity stripe, writes arealso referred to as modify writes. The SP 114asupports nine small reads to nine SCSI drivesconcurrently. When stripe size is set to 8K, a read of64K of data starts all eight SCSI drives, with eachdrive reading one 8K stripe worth of data. The paralleloperation is transparent to the caller client.

The parity stripes are rotated among the ninedrives in order to avoid drive contention during, writeoperations. The parity stripe is used to improveavailability of data. When one drive is down, the. SP114a can reconstruct the missing data from a paritystripe. In such case, the SP 114a is running in errorrecovery mode. When a bad drive is repaired, the SP114a can be instructed to restore data on the repaireddrive while the system is on-line.

When the SP 114a is used to attach thirtyindependent SCSI drives, no parity stripe is createdand the client addresses each drive directly.

The SP 114a processes multiple messages(transactions, commands) at one time, up to 200messages per second. The SP 114a does not initiate anymessages after initial system configuration. Thefollowing SP 114a operations are defined: 25

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -74-

01 Mo Op 02 Send Configuration Data03 Receive Configuration Data 05 Read and Write Sectors 5 06 Read and Write Cache Pages 07 IOCTL Operation08 Dump SP 114a Local Data Buffer09 Start/Stop Λ SCSI Drive0C Inquiry * 10 0E Read Message Log Buffer OF Set SP 114a Interrupt

The above transactions are described in detail inthe above-identified application entitled MULTIPLE FACILITY OPERATING SYSTEM ARCHITECTURE. For an, 15 understanding of the invention, it will be useful todescribe the function and operation of only two ofthese commands: read and write sectors, and read and write cache pages.

Read and Write Sectors 20 This command, issued usually by an FC 112, causes the SP 114a to transfer data between a specified blockof system memory and a specified series of contiguoussectors on the SCSI disks. As previously described Inconnection with the file controller 112, the particular 25 sectors are Identified in physical terms. In

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7

I

I -75- particular, the particular disk sectors are identified by SCSI channel number (0-9) SCSI ID on that channel number (0-2), starting sector address on the specified drive, and a count of the number of sectors to read or write. The SCSI channel number is zero if the SP 114a is operating under RAID 5.

The SP 114a can execute up to 30 messages on the30 SCSI drives simultaneously. Unlike most of thecommands to an SP 114, which are processed bymicroprocessor 510 as soon as they appear on thecommand FIFO 534, read and write sectors commands (aswell as read and write cache memory commands) are firstsorted and queued. Hence, they are not served in the order of arrival.

When a disk access command arrives, themicroprocessor 510 determines which disk drive istargeted and inserts the message in a queue for thatdisk drive sorted by the target sector address. Themicroprocessor 510 executes commands on all the queuessimultaneously, in the order present in the queue for each disk drive. In order to minimize disk armmovements, the microprocessor 510 moves back and forthamong queue entries in an elevator fashion.

If no error conditions are detected from the SCSI disk drives, the command is completed normally. When adata check error condition occurs and the SP 114a is

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7

X

X -76- configured for RAID 5, recovery actions usingredundant data begin automatically. When a drive isdown while the SP 114a is configured for RAID 5,recovery actions similar to data check recovery take 5 place.

Read/Write Cache Pages

This command is similar to read and write sectors,except that multiple VME addresses are provided fortransferring disk data to and from system memory 116. 10 Each VME address points to a cache page in systemmemory 116, the size of which is also specified in thecommand. When transferring data from a disk to systemmemory 116, data are scattered to different cachepages; when writing data to a disk, data are gathered 15 from different cache pages in system memory 116.Hence, this operation is referred to as a scatter-gather function.

The target sectors on the SCSI disks are specifiedin the command in physical terms, in the same manner 20 that they are specified for the read and write sectors command. .· Termination of the command with or without error conditions is the same as for the read and write sectors command.

The dual-port RAM 584 in the DMA controller 580 25 maintains a separate set of commands for each channel

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 V.

-77- controlled by the bit slice engine 582. As eachchannel completes its previous operation, themicroprocessor 510 writes a new DMA operation into thedual-port RAM 584 for that channel in order to satisfy 5 the next operation on a disk elevator queue.

The commands written to the DMA controller 580 include an operation code and a code indicatingwhether the operation is to be performed in non-blockmode, in standard VME block mode, or in enhanced block 10 mode. The operation codes supported by DMA controller 580 are as follows: OP CODE OPERATION 0 NO-OP 15 1 ZEROES -> BUFFER Move zeros from zerosregister 576 to localdata buffer 564. 20 2 ZEROES -> FIFO Move zeros from zerosregister 576 to thecurrently selected FIFOon common data bus 550. 25 3 " ZEROES -> VMEbus Move zeros from zerosregister 576 out onto the . VME bus 120. Used for initializing cache .. buffers in system memory116.

Attorney Docket No,:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -78- 4 VMEbus -> BUFFER Move data from - the VMEbus 120 to the local databuffer 564. This operation is used duringa write, to move targetdata intended for a downdrive into the buffer forparticipation in redundancy generation. Used only for RAID 5application. 5 VMEbus -> FIFO Mew data to be writtenfrom VME bus onto a drive. Since RAID 5 >3·^ * requires redundancy data . .+ * ’ ..0 • «ί to be generated from datathat is buffered in local . .. 7 « / ,:ί · z data buffer 564, this '4.* operation Will be used only if the SP 114a is i not configured for RAID 5. 6 VMEbus 5-> . / BUFFER &amp; FIFO Target data is moved from VME bus 120 to a SCSIdevice and is alsocaptured in the local data buffer 564 forparticipation in redundancy generation. Used only if SP 114a is configured for RAID 5operation. 7 BUFFER -> VMEbus This operation is notused. 8 BUFFER -> FIFO Participating data istransferred to createredundant data or recovered data on a diskdrive. Used only in RAID 5 applications. 9 FIFO -> VMEbus This operation is used tomove target data directlyfrom a disk drive ontothe VME bus 120. Attorney Docket Mo. :AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 V.

V -79- A FIFO -> BUFFER Used to move participating data forrecovery and modifyoperations. Used only inRAID 5 applications.

B FIFO -> VMEbus &amp; BUFFER

This operation is used tosave target data forparticipation in datarecovery. Used only inRAID 5 applications.

SYSTEM MEMORY

Fig. 6 provides a simplified block diagram of thepreferred, architecture of one of the system memorycards 116a. Each of the other system memory cards arethe same. Each memory card 116 operates as a slave onthe enhanced VME bus 120 and therefore requires no on-board CPU. Rather, a timing control block 610 issufficient to provide the necessary slave controloperations. In particular, the timing control block610, in response to control signals from the controlportion of the enhanced VME bus 120, enables a 32-bitwide buffer 612 for an appropriate direction transfer of 32-bit data between the enhanced VME bus 120 and a multiplexer unit 614. The multiplexer 614 provides amultiplexing and demultiplexing function, depending ondata transfer direction, for a six megabit by seventy-two bit word memory array 620. An error correctioncode (ECC) generation and testing unit 622 is also connected to the multiplexer 614 to generate or verify,Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -80- again depending on transfer direction, eight bits ofECC data. The status of ECC verification is providedback to the timing control block 610.

ENHANCED VHE BUS PROTOCOL 5 VME bus 120 is physically the same as an ordinary VME bus, but each of the NCs and SPs include additionalcircuitry and firmware for transmitting data using anenhanced VME block transfer protocol. The Enhanced protocol is described in detail in the above-identified *

10 application entitled ENHANCED VMEBOS PROTOCOL UTILIZING

PSEUDOSYNCHRONOUS HANDSHAKING AND BLOCK MODE DATA TRANSFER, and summarized in the Appendix B hereto.Typically transfers of LNFS file data between NCs andsystem memory, or between SPs and system memory, and 15 transfers of packets being routed from one NC toanother through system memory, are the only types oftransfers that use the enhanced protocol in server 100.

All other data transfers on VME bus 120 use either conventional VME block transfer protocols or ordinary 20 non-block transfer protocols.

MESSAGE PASSING

As is evident from the above description, thedifferent processors in the server 100 communicate withi each other via certain types of messages. In software,

Attorney Docket No.:AUSP7209WPl/WSW/AUSP/7209.001 8/24/89-7 V. -81- these messages are all handled by the messaging kernel,described in detail in the MULTIPLE FACILITY OPERATINGSYSTEM ARCHITECTURE application cited above. Inhardware, they are implemented as follows.

Each of the NCs 110, each of the FCs 112, and each

of the SPs 114 includes a command or communication FIFO such as 290 on NO 110a. The host 118 also includes acommand FIFO, but since the host is an unmodifiedpurchased processor board, the FIFO is emulated insoftware. The write port of the command FIFO in eachof the processors is directly addressable from any ofthe other processors over VME bus 120.

Similarly, each of the processors except SPs 114also includes shared memory such as CPU memory 214 onNC 110a. This shared memory is also directlyaddressable by any of the other processors in the server 100.

If one processor, for example network controller110a, is to send a message or command to a secondprocessor, for example file controller 112a, then itdoes so as follows. First, it forms the message in itsown shared memory (e.g., in CPU memory 214 on NC 110a).Second, the microprocessor in the sending processordirectly writes a message descriptor into the commandFIFO in the receiving processor. For a command beingsent from network controller 110a to file controller

Attorney Docket No.:AUSP7209WP1/WSW/AUSP/7209.001 8/24/89-7 ν -82- 112a, the microprocessor 210 would perform the writevia buffer 284 on NC 110a, VMS' bus 120, and buffer 384 on file controller 112a.

The command descriptor is a single 32-bit wordcontaining in its high order 30 bits a VME addressindicating the start of a quad-aligned message in the sender's shared memory. The low order two bits indicate the message type as ι follows: Type Description 0 Pointer to a new message being sent 1 Pointer to a reply message 2 Pointer to message to be forwarded 3 Pointer to message to be freed; also message acknowledgmentAll messages are 128-bytes long.

When the receiving processor reaches the commanddescriptor on its command FIFO, it directly accessesthe sender's shared memory and copies it into thereceiver's own local memory. For a command issued fromnetwork controller 110a to file controller 112a, thiswould be an ordinary VME block or non-block modetransfer from NC CPU memory 214, via buffer 284, VMEbus 120 and buffer 384, into FC CPU memory 314. The FCmicroprocessor 310 directly accesses NC CPU memory 214for this purpose over the VME bus 120.

When the receiving processor has received thecommand and has completed its work, it sends a replyAttorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -83- message back to the sending processor. The replymessage may be no more them the original commandmessage unaltered, or it may be a modified version ofthat message or a completely new message. If the replymessage is not identical to the original commandmessage, then the receiving processor directly accesses the original sender's shared memory to modify the original command message or overwrite it completely. For replies from the FC 112a to the NC 110a, this involves an ordinary VME block or non -block mode transfer from the FC 112a, via buffer 384, VME bus 120, buffer 284 and into NC CPU memory 214. Again, the PCmicroprocessor 310 directly accesses NC CPU memory 214for this purpose over the VME bus 120.

Whether or not the original command message hasbeen changed, the receiving processor then writes areply message descriptor directly into the originalsender's command FIFO. The reply message descriptorcontains the same VME address as the original commandmessage descriptor, and the low order two bits of theword are modified to indicate that this is a replymessage. For replies from the FC 112a to the NC 110a,the message descriptor write is accomplished bymicroprocessor 310 directly accessing command FIFO 290via buffer 384, VME bus 120 and buffer 280 on the NC.Once this is done, the receiving processor can free the

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -84- buffer in its local memory containing the copy of the command message.

When the original sending processor reaches thereply message descriptor on its command FIFO, it wakesup the process that originally sent the message andpermits it to continue. After examining the replymessage, the original sending processor can free theoriginal command message buffer in its own local shared memory.

As mentioned above, network controller 1'iOa usesthe buffer 284 data path in order to write messagedescriptors onto the VME bus 120, and uses VME/FIFO DMAcontroller 272 together with parity FIFO 270 in orderto copy messages from the VME bus 120 into CPU memory214. Other processors read from CPU memory 214 usingthe buffer 284 data path.

File controller 112a writes message descriptorsonto the VME bus 120 using the buffer 384 data path,and copies messages from other processors' sharedmemory via the same data path. Both take place underthe control of microprocessor 310. Other processorscopy messages from CPU memory 314 also via the buffer384 data path.

Storage processor 114a writes message descriptorsonto the VME bus using high-speed register 590 in themanner described above, and copies messages from other

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7

10 15 -85- processors using DMA controller 580 and FIFO 554. TheSP 114a has no shared memory'; however, so it uses abuffer in system memory 116 to emulate that function.That is, before it writes a message descriptor intoanother processor's command FIFO, the SP 114a firstcopies the message into its own previously allocatedbuffer in system memory 116 using DMA controller 580and FIFO 554. The VME address included in the messagedescriptor then reflects the VME address of the messagein system memory 116.

In the host 118, the command FIFO and sharedmemory are both emulated in. software.

The invention has been described with respect toparticular embodiments thereof, and it will be understood that numerous modifications and variations are possible within the scope of the invention.

Attorney Docket No.:AUSP7209WPl/WSW/AUSP/7209.001 8/24/89-7 -86-

APPENDIX A VME/FIFO DMA Controller

In storage processor 114a, DMA controller 580manages the data path under the direction of themicroprocessor 510. The DMA controller 580 is amicrocoded 16-bit bit-slice implementation executingpipelined instructions at a rate of one each 62.5ns.It is responsible for scanning the channel status 562and servicing request with parameters stored in thedual-ported ram 584 by the microprocessor 510. Endingstatus is returned in the ram 584 and interrupts aregenerated for the microprocessor 510.

Control Store. The control store contains the

microcoded instructions which control the - DMA controller 580. The control store consists of 6 IK x 8 proms configured to yield a IK x 48 bit microword.Locations within the control store are addressed by thesequencer and data . is presented at the input of thepipeline registers.

Sequencer. The sequencer controls program flow bygenerating control store addresses based upon pipeline data and various status’ bits. The control store address consists of 10 bits. Bits 8:0 of the control store address derive from a multiplexer having as itsinputs either an ALU output or the output of anincrementer. The incrementer can be preloaded with

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7

V -87- pipeline register bits 8:0, or it can be incremented asa result of a test condition. The IK address range isdivided into two pages by a latched flag such that themicroprogram can execute from either page. Branches,however remain within the selected page. Conditionalsequencing is performed by having the test conditionincrement the pipeline provided address. A falsecondition allows execution from the pipeline address while a true condition causes execution from the address + 1. The alu output is selected as an addresssource in order to directly vector to a routine or inorder to return to a calling routine. Note that whencalling a subroutine the calling routine must residewithin the same page as the subroutine or the wrongpage will be selected on the return. ALU. The alu comprises a single IDT49C402Aintegrated circuit. It is 16 bits in width and mostclosely resembles four 2901s with 64 registers. The aluis used primarily for incrementing, decrementing,addition and bit manipulation. All necessary controlsignals originate in the control store. The IDT HIGHPERFORMANCE CMOS 1988 DATA BOOK, incorporated byreference herein, contains additional information about the alu.

Microword. The 48 bit microword comprises several fields which control various functions of the

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -88- DMA controller 580. The format of the microword isdefined below along with mnemonics and a description of each function. AI<8:0> 47:39 (Alu Instruction bits 8:0) The Al bitsprovide the instruction for the 49C402Aalu. Refer to the IDT data book for acomplete definition of the aluinstructions. Note that the 19 signalinput of the 49C402A is always low.

CIN 38 (Carry INput) This bit forces the carryinput to the alu. R2V<5:0> 37:32 (Register A address bits 5:0) These bitsselect one of 64 registers as the "A"operand for the alu„ These bits alsoprovide literal bits 15:10 for the alubus. RB<5:0> 31:26 (Register B address bits 5:0) These bitsselect one of 64 registers as the "B"operand for the alu. These bits alsoprovide literal bits 9:4 for the alubus. LFD 25 (Latched Flag Data) When set this bitcauses the selected latched flag to beset. When reset this bit causes theselected latched flag to be cleared.This bits also functions as literal bit3 for the alu bus. LFS<2:0> 24:22 (Latched Flag Select bits 2:0) Themeaning of these bits is dependent uponthe selected source for the alu bus. Inthe event that the literal field isselected as the bus source thenLFS<2:0> function as literal bits <2:0>otherwise the bits are used to selectone of the latched flags.

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.00 1 8/24/89-7 k. k -89- LFS<2:0>

SELECTED FLAG 0 This value selects a null flag. 1 When set this bit enables thebuffer clock. When reset thisbit disables the bufferclock. 2 When this bit is cleared VMEbus transfers, bufferoperations and RAS are alldisabled.

3 NOT USED 4 When set this bit enables VMEbus transfers. 5 When set this bit enablesbuffer operations. 6 When set this bit asserts therow address strobe to the drambuffer. 7 When set this bit selects page0 of the control store. SRC<l,0> 20,21 (alu bus SouRCe select bits 1,0) Thesebits select the data source to beenabled onto the alu bus. SRC<1.0> Selected Source 0 alu 1 dual ported ram 2 literal 3 reserved-not defined PF<2:0> 19:17 (Pulsed Flag select bits 2:0) These bitsselect a flag/signal to be pulsed.

Attorney Docket No.:AUSP7209WPl/WSW/AUSP/7209.001 8/24/89-7 k -90- PF<;2;0> Flag 0 null '

1 SGL_CLK generates a single transitionof buffer clock.

2 SET_VB forces vine and buffer enableto be set.

3 CL_PERR clears buffer parity errorstatus.

4 SET_DN set channel done status forthe currently selectedchannel.

5 INC_ADR increment dual ported ramaddress.

6:7 RESERVED - NOT DEFINED DEST<3:0> 16:13 (DESTination select bits 3:0) These bits select one of 10 destinationsto be loaded from the alu bus. DEST<3:0> Destination 0 null 1 WR_RAM causes the data on the alu bus to be written to the dual 2 ported ram. D<15:0> -> ram<15:0> WR_BADD loads the data from the alu bus into the dram address counters. D<14:7> -> mux addr<8:0>

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7

V V. -91-

3 WR_VADL loads the data from the alubus iilto the least significant2 bytes of the VME addressregister. D<15:2> -> VME addr<15:2>

DI -> ENB_ENH

DO -> ENB_BLK

4 WR_VADH loads the most significant 2 bytes of the VME address register. D<15:0> -> VME addr<31:16>

5 WR_RADD loads the dual ported ramaddress counters. D<10:0> -> ram addr <10:0>

6 WR_WCNT loads the word counters. D15 -> count enable* D<14:8> -> count <6:0>

7 WR_CO loads the co-channel selectregister. D<7:4> -> C0<3:0>

8 WR_NXT loads the next-channel selectregister. D<3:0> -> NEXT<3:0>

9 WR_CUR loads the current-channelselect register. D<3:0> -> CURR <3:0>

10:14 RESERVED - NOT DEFINED

15 JUMP causes the control storesequencer to select the aludata bus . D<8:0> -> CS_A<8:0>

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -92- TEST<3:0> 12:9 (TEST condition select bits 3:0) Select one of 16 ^inputs to the test multiplexor to” be used as the carry input to the incrementer. TEST<3:0> Condition NEXT A<8:0> 0 FALSE -always false 1 TRUE -always true 2 ALU_COUT -carry output of alu 3 ALU_EQ -equals output ofalu 4 ALU_OVR -alu overflow 5 ALU_NEG -alu negative 6 XFR_DONE -transfer complete 7 PAR_ERR -buffer parity error 8 TIMOUT -bus operation timeout 9 ANY_ERR -any error status 14:10 RESERVED -NOT DEFINED 15 CH_RDY -next channel ready (NEXT Address bits 8:0) Selects an instructions from the current page ofthe control store for execution.

Dual Ported Ram. The dual ported ram is themedium by which command, parameters and status are communicated between the DMA controller 580 and the microprocessor 510. The ram is organized as IK x 32 atthe master port and as 2K x 16 at the DMA port. The rammay be both written and read at either port.

The ram is addressed by the DMA controller 580 by loading an 11 bit address into the address counters.Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -93-

Data is then read into bidirectional registers and the s. address counter is incremented to allow read of the next location.

Writing the ram is accomplished by loading datafrom the processor into the registers after loading theram address. Successive writes may be performed onevery other processor cycle.

The ram contains current block pointers, endingstatus, high speed bus address and parameter blocks.The following is the format of the ram:

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -94- OFFSET 31 0 0 ICURR POINTER 0 | STATUS 0 1 I 4 1 INITIAL POINTER 0 1 1 58 |CURR POINTER B | STATUS B 1 1 5C | INITIAL POINTER B I 1 60 | not used | not used 1 1 64 | not used ! not used 1 1 68 ICURR POINTER D | STATUS D I 1 6C | INITIAL POINTER D 1 1 70 I not used j STATUS E 1 1 74 {HIGH SPEED BUS ADDRESS 31:2|0 01 78 | PARAMETER BLOCK 0 1 1 ?? i PARAMETER BLOCK n i

The Initial Pointer is a 32 bit value which pointsthe first command block of a chain. The current pointeris a sixteen bit value used by the DMA controller 580to point to the current command block. The currentcommand block pointer should be initialized to 0x0000by the microprocessor 510 before enabling the channel.Upon detecting a value of 0x0000 in the current blockpointer the DMA controller 580 will copy the lower 16bits from the initial pointer to the current pointer.Once the DMA controller 580 has completed the specifiedAttorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7

-95-

operations for the parameter block the current pointerwill be updated to point to the next block. In theevent that no further parameter blocks are availablethe pointer will be set to 0x0000.

The status byte indicates the ending status forthe last channel operation performed. The followingstatus bytes are defined: 10

STATUS MEANING

0 NO ERRORS

1 ILLEGAL OP CODE

2 BUS OPERATION TIMEOUT

3 BUS OPERATION ERROR

4 DATA PATH PARITY ERROR

The format of the parameter block is: 15 20

OFFSET 0 4 8

C 25 30 C+(4Xn) 31 0 FORWARD LINK NOT USED . J WORD COUNT VME ADDRESS 31:2, ENH, BLK ί TERM 0 I OP 0 | BUF ADDR 0 • ί TERM n ί OP n j BUF ADDR n 1 1 35 FORWARD LINK - The forward link points to thefirst word of the next parameter block for execution.It allows several parameter blocks to be initialized

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -96- and chained to execution. The format: create a sequence of operations for ’o. forward pointer has the following A31:A2/0/0

The format dictates that the parameter block must starton a quad byte boundary. A pointer of 0x00000000 is aspecial case which indicates no forward link exists. WORD COUNT - The word count specifies the numberof quad byte words that are to be transferred to orfrom each buffer address or to/from the VME address. Aword count of 64K words may be specified byinitializing the word count with the value of 0. Theword count has the following format: jD15JD14jD13!D12jDI1j010 JD9i08SD7jD6jD5|D4jD3jD2j01jDOj

The word count is updated by the DMA controller580 at the completion of a transfer to/from the lastspecified buffer address. Word count is not updatedafter transferring to/from each buffer address and is therefore not an accurate indicator of the total data moved to/from the buffer. Word count represents the amount of data transferred to the VME bus or one of the FIFOs 544 or 554. VME ADDRESS - The VME address specifies thestarting address for data transfers. Thirty bits allowsthe address to start at any quad byte boundary.

Attorney Docket No.:AUSP7209WPl/WSW/AUSP/7209.001 8/24/89-7 c c -97-

ENH - This bit when set selects the enhancedblock transfer protocol described in the above-citedENHANCED VMEBUS PROTOCOL UTILIZING PSEUDOSYNCHRONOUS HANDSHAKING AND BLOCK MODE DATA TRANSFER application, to be used during the VME bus transfer. Enhanced protocol will be disabled automatically when performing any transfer, to or from 24 bit or 16 bit address space, when the starting address is not 8 byte aligned or when the word count is not even. BLK - This bit when set selects the conventionalVME block mode protocol to be used during the VME bustransfer. Block mode will be disabled automaticallywhen performing any transfer to or from 16 bit addressspace. ( BUF ADDR - The buffer address specifies thestarting buffer address for the adjacent operation.

Only 16 bits are available for a 1M byte buffer and asa result the starting address always falls.on a 16 byteboundary. The programmer must ensure that the startingaddress is on a modulo 128 byte boundary. The bufferaddress is updated by the DMA controller 580 aftercompletion of each data burst.

*A19 J A18 ' A17 j A16 ' A15 J A14' A13' A12 * Al 1 * A10! A9 j A8 ' A7 * A6 j A5 * A4J TERM - The last buffer address and operationwithin a parameter block is identified by the terminal bit. The DMA controller 580 continues to fetch buffer

Attorney Docket No.:AUSP7209 WP1/WSW/AUS P/7209.001 8/24/89-7 c c 10 -98- addresses and operations to perforin until this bit isencountered. Once the last operation within theparameter block is executed the word counter is updatedand if not equal to zero the series of operations isrepeated. Once the word counter reaches zero theforward link pointer is used to access the nextparameter block. ίΟ|0!0!0!0|0|0!0!Τ! OP - Operations are specified by the op code. Theop code byte has the following format: j 0 i 0}0 I 0 j OP3|OP2|OP1j ΟΡΟ ί

The op codes are listed below ("FIFO" refers to any ofthe FIFOs 544 or 554):

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 c c -99-

QR-GO-QS OPERATION • 0 NO-OP 1 ZEROES -> BUFFER 2 ZEROES -> FIFO 5 3 ZEROES -> VMEbus 4 VMEbus -> BUFFER 5 VMEbus -> FIFO 6 VMEbus -> BUFFER 7 BUFFER -> VMEbus 10 8 BUFFER -> FIFO 9 FIFO -> VMEbus A FIFO -> BUFFER B FIFO -> VMEbus C RESERVED 15 D RESERVED E RESERVED F RESERVED

&amp; FIFO

&amp; BUFFER

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 ( ( -100- APPENDIX Β

Enhanced VME Block Transfer Protocol

The enhanced VME block transfer protocol is aVMEbus compatible pseudo-synchronous fast transferhandshake protocol for use on a VME backplane bushaving a master functional module and a slave functional module logically interconnected by a data * . ·'.# -100- transfer bus. The data transfer bus includes a o A* strobe signal line and a data transfer acknowledgesignal line. To accomplish the handshake, the mastertransmits a data strobe signal of a given duration on the data strobe line. The master then awaits thereception of a data transfer acknowledge signal fromthe slave module on the data transfer acknowledgesignal line. The slave then responds by transmittingdata transfer acknowledge signal of a given duration onthe data transfer acknowledge signal line.

Consistent with the pseudo-synchronous nature ofthe handshake protocol, the data to be transferred isreferenced to only one signal depending upon whetherthe transfer operation is a READ or WRITE operation.In transferring data from the master functional unit tothe slave, the master broadcasts the data to betransferred. The master asserts a data strobe signaland the slave, in response to the data strobe signal,captures the data broadcast by the master. Similarly,

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -101- in transferring data from the slave to the master, the slave broadcasts the data to be transferred to the master unit. The slave then asserts a data transferacknowledge signal and the master, in response to thedata transfer acknowledge signal, captures the databroadcast by the slave.

The fast transfer protocol, while not essential tothe present invention, facilitates the rapid transferof large amounts of data across a VME backplane bus bysubstantially increasing the data transfer rate. Thesedata rates are achieved by using a handshake whereinthe data strobe and data transfer acknowledge signalsare functionally decoupled and by specifying highcurrent drivers for all data and control lines.

The enhanced pseudo-synchronous method of datatransfer (hereinafter referred to as "fast transfermode") is implemented so as to comply and be compatiblewith the IEEE VME backplane bus standard. Theprotocol utilizes user-defined address modifiers,defined in the VMEbus standard, to indicate use of the fast transfer mode. Conventional VMEbus functional units, capable only of implementing standard VMEbusprotocols, will ignore transfers made using the fasttransfer mode and, as a result, are fully compatiblewith functional units capable of implementing the fast transfer mode.

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 (. c -102-

10 15 20

The fast transfer mode reduces the number of bus . propagations required to accomplish a handshake fromfour propagations, as required under conventionalVMEbus protocols, to only two bus propagations.Likewise, the number of bus propagations required to effect a BLOCK READ or BLOCK WRITE data transfer is reduced. Consequently, by reducing the propagationsacross the VMEbus to accomplish handshaking and datatransfer functions, the transfer rate is materially 4» increased.

The enhanced protocol is described in detail inthe above-cited ENHANCED VMEBUS PROTOCOL application,and will only be summarized here. Familiarity with the conventional VME bus standards is assumed.

In the fast transfer mode handshake protocol, onlytwo bus propagations are used to accomplish ahandshake, rather than four as required by theconventional protocol. At the initiation of a datatransfer cycle, the master will assert and deassertDSO* in the form of a pulse of a given duration. Thedeassertion of DSO* is accomplished without regard asto whether a response has been received from the slave.The master then waits for an acknowledgement from theslave. Subsequent pulsing of DSO* cannot occur until aresponsive DTACK* signal is received from the slave.Upon receiving the slave's assertion of DTACK*, the 25

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -103- master can then immediately reassert data strobe, if so desired. The fast transfer mode protocol does not require the master to wait for the deassertion of DTACK* by the slave as a condition precedent to subsequent assertions of OSO*. In the fast transfermode, only the leading edge (i.e., the assertion) of asignal is significant. Thus, the deassertion of eitherDSO* or DTACK* is completely irrelevant for completionof a handshake. The fast transfer protocol does notemploy the DS1* line for data strobe purposes at all.

The fast transfer mode protocol may becharacterized as pseudo-synchronous as it includes bothsynchronous and asynchronous aspects. The fasttransfer mode protocol is synchronous in character dueto the fact that DSO* is asserted and deassertedwithout regard to a response from the slave. Theasynchronous aspect of the fast transfer mode protocolis attributable to the fact that the master may notsubsequently assert DSO* until a response to the priorstrobe is received from the slave. Consequently,because the protocol includes both synchronous andasynchronous components, it is most accuratelyclassified as "pseudo-synchronous."

The transfer of data during a BLOCK WRITE cycle inthe fast transfer protocol is referenced only to DSO*.The master first broadcasts valid data to the slave,

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -104- and then asserts OSO to the slave. The slave is given s. a predetermined period of time after the assertion ofOSO* in which to capture the data. Hence, slavemodules must be prepared to capture data at any time, 5 . as DTACK* is not referenced during the transfer cycle.

Similarly, the transfer of data during a BLOCK READ cycle in the fast transfer protocol is referencedonly to DTACK*. The master first asserts DSO*. The slave then broadcasts data to the master and then 10 asserts DTACK*. The master is given a predetermined period of time after the assertion of DTACK in whichto capture the data. Hence, master modules must beprepared to capture data at any time as DSO is notreferenced during.the transfer cycle. 15 Fig. 7, parts A through C, is a flowchart illustrating the operations involved in accomplishingthe fast transfer protocol BLOCK WRITE cycle. Toinitiate a BLOCK WRITE cycle, the master broadcaststhe memory address of the data to be transferred and 20 the address modifier across the DTB bus. The master also drives interrupt acknowledge signal (IACK*) highand the LWORD* signal low 701. A special addressmodifier, for example "IF," broadcast by the master indicates to the slave module that the fast transfer 25 protocol will be used to accomplish the BLOCK WRITE.

The starting memory' address of the data to be

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -105- transferred should reside on a 64-bit boundary and thesize of block of data to be ''transferred should be amultiple of 64 bits. In order to remain in compliancewith the VMEbus standard, the block must not cross a 5 256 byte boundary without performing a new address cycle.

The slave modules connected to the DTB receive the address and the address modifier broadcast by themaster across the bus and receive LWORD* low and IACK* 10 high 703. Shortly after broadcasting the address and address modifier 701, the master drives the AS* signallow 705. The slave modules receive the AS* low signal707. Each slave individually determines whether itwill participate in the data transfer by determining 15 whether the broadcasted address is valid for the slave in question 709. If the address is not valid, the datatransfer does not involve that particular slave and itignores the remainder of the data transfer cycle.

The master drives WRITE* low to indicate that the 20 transfer cycle about to occur is a WRITE operation 711'.

The slave receives the WRITE* low signal 713 and,knowing that the data transfer operation is a WRITE operation, awaits receipt of a high to low transition on the DSO* signal line 715. The master will wait 25 until both DTACK* and BERR* are high 718, which

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7

V

I -106- indicates that the previous slave is no longer driving the DTB.

The master proceeds to place the first segment ofthe data to be transferred on data lines D00 throughD31, 719. After placing data on D00 through D31, themaster drives DSO* low 721 and, after a predeterminedinterval, drives DSO* high 723.

In response to the transition of DSO* from high tolow, respectively 721 and 723, the slave latches thedata being transmitted by the master over data linesD00 through D31, 725. The master places the next segment of the data to be transferred on data lines D00through D31, 727, and awaits receipt of a DTACK* signalin the form of a high to low transition signal, 729 inFig. 7B.

Referring to Fig. 7B, the slave then drives DTACK*low, 731, and, after a predetermined period of time,drives DTACK high, 733. The data latched by the slave,725, Is written to a device, which has been selected to store the data 735. The slave also increments the device address 735. The slave then waits for another transition of DSO* from high to low 737.

To commence the transfer of the next segment ofthe block of data to be transferred, the master drivesDSO* low 739 and, after a predetermined period oftime, drives DSO* high 741. In response to the

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -107- transition of DSO* from high to low, respectively 739and 741, the slave latches the data being broadcast bythe master over data lines D00 through D31, 743. Themaster places the next segment of the data to betransferred on data lines 000 through D31, 745, and awaits receipt of a DTACK* signal in the form of a highto low transition, 747.

The slave then drives DTACK* low, 749, and, aftera predetermined period of time, drives DTACK* high,751. The data latched by the slave, 743, is written tothe device selected to store the data and the device address is incremented 753. The slave waits for another transition of DSO* from high to low 737.

The transfer of data will continue in the above- described manner until all of the data has been transferred from the master to the slave. After all of the data has been transferred, the master will releasethe address lines, address modifier lines, data lines,IACK*' line, LWORD* line and DSO* line, 755. Themaster will then wait for receipt of a DTACK* high tolow transition 757. The slave will drive DTACK* low,759 and, after a predetermined period of time, driveDTACK* high 761. In response to the receipt of theDTACK* high to low transition, the master will driveAS* high 763 and then release the AS* line 765.

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7

V -108-

10 15 20

Fig. 8, parts A through C, is a flowchartillustrating the operations involved in accomplishingthe fast transfer protocol BLOCK READ cycle. Toinitiate a BLOCK READ cycle, the master broadcasts thememory address of the data to be transferred and the address modifier across the DTB bus 801. The masterdrives the LWORD* signal low and the IACK* signal high801. As noted previously, a special address modifierindicates to the slave module that the fast transferprotocol will be used to accomplish the BLOCK READ.

The slave modules connected to the DTB receive the address and the address modifier broadcast by the master across the bus and receive LWORD* low and IACK* high 803. Shortly after broadcasting the address andaddress modifier 801, the master drives the AS* signallow 805. The slave modules receive the AS* low signal807. Each slave individually determines whether itwill participate in the data transfer by determiningwhether the broadcasted address is valid for the slave in question 809. If the address is not valid, the datatransfer does not involve that particular slave and itignores the remainder of the data transfer cycle.

The master drives WRITE* high to indicate that thetransfer cycle about to occur is a READ operation 811.The slave receives the WRITE* high signal 813 and,knowing that the data transfer operation is a READ 25

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -109- operation, places the first segment of the data to betransferred on data lines 000 through D31 819. Themaster will wait until both DTACK* and BERR* are high818, which indicates that the previous slave is nolonger driving the DTB.

The master then drives DSO* low 821 and, after apredetermined interval, drives DSO* high 823. Themaster then awaits a high to low transition on theDTACK* signal line 82 4. As shown in Fig. 8B, th4‘ slavethen drives the DTACK* signal low 825 and, after apredetermined period of time, drives the DTACK* signalhigh 827.

In response to the transition of DTACK* from highto low, respectively 825 and 827, the master latchesthe data being transmitted by the slave over data linesD00 through D31, 831. The data latched by the master,831, is written to a device, which has been selected to store the data the device address is incremented 833. *

The slave places the next segment of the data tobe transferred on data lines D00 through D31, 829, andthen waits for another transition of DSO* from high tolow 835.

To commence the transfer of the next segment ofthe block of data to be transferred, the master drivesDSO* low 839 and, after a predetermined period of

Attorney Docket No.:AUSP7209WPl/WSW/AUSP/7209.001 8/24/89-7

V -110- time, drives DSO* high 841. The master then waits forthe DTACK* line to transition from high to low, 843.

The slave drives DTACK* low, 845, and, after apredetermined period of time, drives DTACK* high, 847.

In response to the transition of DTACK* from high tolow, respectively 839 and 841, the master latches thedata being transmitted by the slave over data lines D00through D31, 845. The data, latched by the master, 845,is written to the device selected to store the data, 851 in Fig. 8C, and the device address is incremented.The slave places the next segment of the data to betransferred on data lines D00 through D31, 849.

The transfer of data will continue in the above- described manner until all of the data to be transferred from the slave to the master has been written into the device selected to store the data.

After all of the data to be transferred has been written into the storage device, the master willrelease the address lines, address modifier lines, data lines, the IACK* line, the LWORD line and DSO* line852. The master will then wait for receipt of a DTACK*high to ’ low transition 853. The slave will driveDTACK* low 855 and, after a predetermined period oftime, drive DTACK* high 857. In response to thereceipt of the DTACK* high to low transition, the

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7

I

ίο 15 -111 - master will drive AS* high 859 and release the AS* line 861.

To implement the fast transfer protocol, a conventional 64 mA tri-state driver is substituted for the 48 mA open collector driver conventionally used inVME slave modules to drive DTACK*. Similarly, theconventional VMEbus data drivers are replaced with 64mA tri-state drivers in SO-type packages. The lattermodification reduces the ground lead inductance of theactual driver package itself and, thus, reduces “groundbounce** effects which contribute to skew between data,DSO* and DTACK*. In addition, signal return inductancealong the bus backplane is reduced by using a connectorsystem having a greater number of ground pins so as tominimize signal return and mated-pair pin inductance.One such connector system is the "High Density Plus"connector, Model No. 420-8015-000, manufactured byTeradyne Corporation.

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 k — 112 —

APPENDIX C

Parity FIFO

The parity FIFOs 240, 260 and 270 (on the networkcontrollers 110), and 544 and 554 (on storageprocessors 114) are each implemented as an ASIC. Allthe parity FIFOs are identical, and are configured onpower-up or during normal operation for the particularfunction desired. The parity FIFO is designed to allowspeed matching between buses of different speed, andto perform the parity generation and correction forthe parallel SCSI drives.

The FIFO comprises two bidirectional data ports,Port A and Port B, with 36 x 64 bits of RAM buffer between them. Port A is 8 bits wide and Port B is 32bits wide. The RAM buffer is divided into . two parts,each 36 x 32 bits, designated RAM X and RAM 7. The twoports access different halves of the buffer alternatingto th'e other half when available. When the chip isconfigured as a parallel parity chip (e.g. one of theFIFOs 544 on SP 114a), all accesses on Port B aremonitored and parity is accumulated in RAM X and RAM Yalternately.

The chip also has a CPU interface, which may be 8or 16 bits wide. In 16 bit mode the Port A pins areused as the most significant data bits of the CPU

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 c -113-

10 15 interface and are only actually used when reading orwriting to the Fifo Data Register inside the chip. A REQ, ACK handshake is used for data transfer onboth Ports A and B. The chip may be configured aseither a master or a slave on Port A in the sense that,in master mode the Port A ACK / RDY output signifiesthat the chip is ready to transfer data on Port A, andthe Port A REQ input specifies that the slave isresponding. In slave mode, however, the Port A REQinput specifies that the master requires a datatransfer, and the chip responds with Port A ACK / RDYwhen data is available. The chip is a master on Port Bsince it raises Port B REQ and waits for Port B ACK toindicate completion of the data transfer.

SIGNAL DESCRIPTIONS

Port A 0-7, P

Port A is the 8 bit data port. Port A P, if used,is the odd parity bit for this port. A Req, A Ack/Rdy 20 These two signals are used in the data transfer mode to control the handshake of data on Port A.

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7

— 114 —

uP Data 0-7, uP Data P, uPAdd 0-2, CS

These signals are used 'by a microprocessor to address the programmable registers within the chip.The odd parity signal uP Data P is only checked whendata is written to the Fifo Data or Checksum Registersand microprocessor parity is enabled.

Clk

The clock input is used to generate some of thechip timing. It is expected to be in the 10-20 Mhz 10 range.

Read En, Write En

During microprocessor accesses, while CS is true,these signals determine the direction of themicroprocessor accesses. During data transfers in the 15 WD mode these signals are data strobes used in conjunction with Port A Ack.

Port B 00-07, 10-17, 20-27, 30-37, 0P-3P

Port B is a 32 bit. data port. There is one oddparity bit for each byte. Port B OP is the parity of 20 bits 00-07, PortB IP is the parity of bits 10-17, Port B 2P- is the parity of bits 20-27, and Port Β 3P is theparity of bits 30-37.

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 c -115- Β Select, Β Req, Β Ack, Parity Sync, B Output Enable

These signals are used in the data transfer modeto control the handshake of data on Port B. Port B Reqand Port B Ack are both gated with Port B Select.The Port B Ack signal is used to strobe the data on thePort B data lines. The parity sync signal is used toindicate to a chip configured as the parity chip to indicate that the last words of data involved in theparity accumulation are on Port B. The Port B datalines will only be driven by the Fifo chip if all ofthe following conditions are met: a. the data transfer is from Port A to Port B; b. the Port B select signal is true; c. the Port B output enable signal is true; and d. the chip is not configured as the parity chipor it is in parity correct mode and theParity Sync signal is true.

Reset

This signal resets all the registers within thechip 'and causes all bidirectional pins to be in a highimpedance state.

DESCRIPTION OF OPERATION

Normal Operation. Normally the chip acts as asimple FIFO chip. A FIFO is simulated by using two RAMbuffers in a simple ping-pong mode. It is intended,but not mandatory, that data is burst into or out of

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 ί -116- the FIFO on Port B. This is done by holding Port B Selsignal low and pulsing the Port B Ack signal. Whentransferring data from Port B to Port A, data is firstwritten into RAM X and when this is full, the datapaths will be switched such that Port B may startwriting to RAM Y. Meanwhile the chip will beginemptying RAM X to Port A. When RAM Y is full and RAMX empty the data paths will be switched again such thatPort B may reload RAM X and Port A may empty RAM Y.

Port A Slave Mode. This is the default mode and the chip is reset to this condition. In this mode thechip waits for a master such as one of the SCSI adapterchips 542 to raise Port A Request for data transfer.If data is available the Fifo chip will respond withPort A Ack/Rdy.

Port A WD Mode. The chip may be configured torun in the WD or Western Digital mode. In this modethe chip must be configured as a slave on Port A. Itdiffers from the default slave mode in that the chipresponds with Read Enable or Write Enable asappropriate together with Port A Ack/Rdy. This mode isintended to allow the chip to be interfaced to theWestern Digital 33C93A SCSI chip or the NCR 53C90 SCSIchip.

Port A Master Mode. When the chip is configuredas a master, it will raise Port A Ack/Rdy when it is

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7

L ( -117- ready for data transfer. This signal is expected to betied to the Request input of a DMA controller whichwill respond with Port A Req when data is available.In order to allow the DMA controller to burst, the PortA Ack/Rdy signal will only be negated after every 8 or16 bytes transferred.

Port B Parallel Write Mode. In parallel writemode, the chip is configured to be the parity chip fora parallel transfer from Port B to Port A. In thismode, when Port B Select and Port B Request areasserted, data is written into RAM X or RAM 7 eachtime the Port B Ack signal is received. For the firstblock of 128 bytes data is simply copied into theselected RAM. The next 128 bytes driven on Port B willbe exclusive-ORed with the first 128 bytes. Thisprocedure will be repeated for all drives such that theparity is accumulated in this chip. The Parity Syncsignal should be asserted to the parallel chip togetherwith 'the last block of 128 bytes. This enables thechip to switch access to the other RAM and startaccumulating a new 128 bytes of parity.

Port B Parallel Read Mode - Check Data. This mode is set if all drives are being read and parity isto be checked. In this case the Parity Correct bit inthe Data Transfer Configuration Register is not set.The parity chip will first read 128 bytes on Port A as

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -118-

10. 15 20 in a- normal read mode and then raise Port B Request.While it has this signal asserted the chip will monitorthe Port B Ack signals and exclusive-or the data onPort B with the data in its selected RAM. The ParitySync should again he asserted with the last block of128 bytes. In this mode the chip will not drive thePort B data lines but will check the output of itsexclusive-or logic for zero. If any bits are set atthis time a parallel parity error will be flagged.

Port B Parallel Read Mode - Correct Data. Thismode is set by setting the Parity Correct bit in theData Transfer Configuration Register. In this case thechip will work exactly as in the check mode except thatwhen Port B Output Enable, Port B Select and ParitySync are true the data is driven onto the Port B datalines and a parallel parity check for zero is notperformed.

Bvte Swap. In the normal mode it is expectedthat Port B bits 00-07 are the first byte, bits 10-17the second byte, bits 20-27 the third byte, and bits30-37 the last byte of each word. The order of thesebytes may be changed by writing to the byte swap bitsin the configuration register such that the byteaddress bits are inverted. The way the bytes arewritten and read also depend on whether the CPUinterface is configured as 16 or 8 bits. The following 25

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 k. -119- table shows the byte alignments for the different poss abilities for data transfer using the Port A Request / Acknowledge handshake: CPU Invert Invert Port B Port B Port B Port B I/F. Addr 1 Addr 0 00-07 10-17 20-27 30-37 8 False False Port A Port A Port A Port A byte 0 byte 1 byte 2 byte 1 8 False True Port A Port A Port A Port A byte 1 byte 0 byte 3 byte 2 8 True False Port A Port A Port A Port A byte 2 byte 3 byte 0 byte 1 8 True True Port A Port A Port A Port A byte 3 byte 2 byte 1 byte 0 16 False False Port A uProc Port A uProc byte 0 byte 0 byte 1 byte 1 16 False True uProc Port A uProc Port A byte 0 byte 0 byte 1 byte 1 16 True False Port A uProc Port A uProc byte 1 byte 1 byte 0 byte 0 16 True True uProc Port A uProc Port A byte 1 byte 1 byte 0 byte 0 When the Fifo is accessed by reading or writing the Fifo Data Register through the microprocessor portin 8 bit mode, the bytes are in the same order as thetable above but the uProc data port is used instead ofPort A. In 16 bit mode the table above applies.

Odd Length Transfers. If the data transfer is not a multiple of 32 words, or 128 bytes, themicroprocessor must manipulate the internal registersof the chip to ensure all data is transferred. Port AAck and Port B Req are normally not asserted until all

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -120-

10 15 20 32 words of the selected RAM are available. Thesesignals may be forced by writing to the appropriate RAMstatus bits of the Data Transfer Status Register.

When an odd length transfer has taken place themicroprocessor must wait until both ports are quiescentbefore manipulating any registers. It should thenreset both of the Enable Data Transfer bits for Port A and Port B in the Data Transfer Control Register. Itmust then determine by reading their Address Registersand the RAM Access Control Register whether RAM X orRAM Y holds the odd length data. It should then setthe corresponding Address Register to a value of 20hexadecimal, forcing the RAM full bit and setting theaddress to the first word. Finally the microprocessor should set the Enable Data Transfer bits to allow the chip to complete the transfer.

At this point the Fifo chip will think that thereare now a full 128 bytes of data in the RAM and willtransfer 128 bytes if allowed to do so. The fact thatsome of these 128 bytes are not valid must berecognized externally to the FIFO chip.

Attorney Docket No.:AUSP7209WPl/WSW/AUSP/7209.001 8/24/89-7 V. V. — 121 —

PROGRAMMABLE REGISTERS

Data Transfer Configuration Register LRead/Write)

This register is cleared by the reset signal.

Bit

Bit 2

Bit 3 WD Mode. Set if data transfers are touse the Western Digital WD33C93Aprotocol, otherwise the Adaptec 6250protocol will be used.

Parity Chin. Set if this chip isaccumulate Port B parities.

Parity Correct Mode. Setparity chip is to correctparity on Port B. to if theparallel CPU Interface 16 bits wide. If set,the microprocessor data bits arecombined with the Port A data bits toeffectively produce a 16 bit Port. Allaccesses by the microprocessor as wellas all data transferred using the Port ARequest and Acknowledge handshake willtransfer 16 bits.

Bit 4 Invert Port A bvte address 0. Set to invert the least significant bit of Port A byte address. Bit 5 Invert Port A bvte address 1. Set to invert the most significant bit of PortA byte address.

Bit 6 Checksum Carry Wrap. Set to enable thecarry out of the 16 bit checksum adderto carry back into the least significantbit of the adder.

Bit·· 7 Reset. Writing a 1 to this bit willreset the other registers. This bitresets itself after a maximum of 2clock cycles and will therefore normallybe read as a 0. No other registershould be written for a minimum of 4clock cycles after writing to this bit.

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -122- ki k.

Data Transfer Control Register fRead/Write)

Bit 0 Enable Data Transfer on Port A. Set to enable the Port A Reg/Ack handshake.

Bit 1 Enable Data Transfer on Port B. Set to enable the Port B Req/Ack handshake.

Bit 2 Port A to Port B. If set, datatransfer is from Port A to Port B. Ifreset, data transfer is from Port B toPort A. In order to avoid any glitcheson the request lines, the state of thisbit should not be altered at the sametime as the enable data transfer bits 0or 1 above.

Bit 3 uProcessor Parity Enable. Set if parityis to be checked on the microprocessorinterface. It will only be checked whenwriting to the Fifo Data Register orreading from the Fifo Data or ChecksumRegisters, or during a Port ARequest/Acknowledge transfer in 16 bitmode. The chip will, however, alwaysre-generate parity ensuring thatcorrect parity is written to the RAM orread on the microprocessor interface.

Bit 4 Port A Parity Enable. Set if parity isto be checked on Port A. It is checkedwhen accessing the Fifo Data Register in16 bit mode, or during a Port ARequest/Acknowledge transfer. The chipwill, however, always re-generate parityensuring that correct parity is writtento the RAM or read on the Port Ainterface.

Bit 5 Port B Parity Enable. Set if Port Bdata has valid byte parities. If it isnot set, byte parity is generatedinternally to the chip when writing tothe RAMs. Byte parity is not checkedwhen writing from Port B, but alwayschecked when reading to Port B.

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7

V -123-

Bit 6 Checksum Enable. Set to enable writingto the 16 bit checksum register. Thisregister accumulates a 16 bit checksumfor all RAM accesses, includingaccesses to the Fifo Data Register, aswell as all writes to the checksumregister. This bit must be reset beforereading from the Checksum Register.

Bit 7 Port A Master. Set if Port A is tooperate in the master mode on Port Aduring the data transfer.

Data Transfer Status Register (Read Only)

Bit 0 Data in RAM X or RAM Y. Set if any bitsare true in the RAM X, RAM Y, or Port Abyte address registers.

Bit 1 uProc Port Parity Error. Set if theuProc Parity Enable bit is set and aparity error, is detected on themicroprocessor interface during any RAMaccess or write to the Checksum Registerin 16 bit mode.

Bit 2

Bit 3

Port A Parity Error. Set if the Port AParity Enable bit is set and a parityerror is detected on the Port Ainterface during any RAM access or writeto the Checksum Register.

Port B Parallel Parity Error . Set ifthe chip is configured as the paritychip, is not in parity correct mode, anda non zero result is detected when theParity Sync signal is true. It is alsoset whenever data is read out onto PortB and the data being read back throughthe bidirectional buffer does notcompare.

Bits 4-7

Port B Bvtes 0-3 Parity Error. Set whenever the data being read out of the RAMs on the Port B side has bad parity.

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -124-

Ram Access Control Register_(Read/Write),

Enable Data Transfer bits in the Data Transfer Control 5 Register must be reset before attempting to write to this register, else the write will be ignored. 10 Bit 0 Port A bvte address 0. This bit is theleast significant byte address bit. Itis read directly bypassing any inversiondone by the invert bit in the DataTransfer Configuration Register. 15 Bit 1 Port A bvte address 1. This bit is themost significant byte address bit. Itis read directly bypassing any inversiondone by the invert bit in the DataTransfer Configuration Register. Bit 2 Port A to RAM Y. Set if Port A is accessing RAty Y, and. reset if it isaccessing RAM X . 20 Bit 3 Port B to RAM Y. Set if Port B is accessing RAM Y, and reset if it isaccessing RAM X . 25 30 Bit 4 Lone Burst. If the chin is configuredto transfer data on Port A as a master,and this bit is reset, the chip willonly negate Port A Ack/Rdy after every 8bytes, or 4 words in 16 bit mode, havebeen transferred. If this bit is set,Port A Ack/Rdy will be negated every 16bytes,'or 8 words in 16 bit mode. Bits .T 5-7 Rpt ...Used· RAM X Address Register (Read/Write)

Register Address 4. This register is cleared by the reset signal or by writing to the reset bit. The35 Enable Data Transfer bits in the Data Transfer Control

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -125-

Register must be reset before attempting to this register, else the write will be ignored. Bits 0-4 RAM X word address Bit 5 RAM X full Bits 6-7 Not Used RAM Y Address Register (Read/Writel

Register Address 5. This register is cleared by the reset signal or by writing to the reset bit. TheEnable Data Transfer bits in the Data Transfer Control

Register must be reset this register, else the Bits 0-4 RAM Bit 5 RAM Bits 6-7 Not before attempting to write towrite will be ignored. 7 word address Y full

Used 15 Fifo Data Register (Read/Writel

Register Address 6. The Enable Data Transfer bits in the Data Transfer Control Register must be resetbefore attempting to write to this register, else thewrite will be ignored. The Port A to Port B bit in the 20 Data Transfer Control register must also be set before writing this register. If it is not, the RAM controls will be incremented but no data will be written to the RAM. For consistency, the Port A to PortB should bereset prior to reading this register.

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -126-

Bits 0-7 are Fifo Data. The microprocessor mayaccess the FIFO by reading or writing this register.The RAM control registers are updated as if the accesswas using Fort A. If the chip is configured with a 16bit CPU Interface the most significant byte will usethe Port A 0-7 data lines, and each Port A access willincrement the Port A byte address by 2.

Port A Checksum Register (Read/Write)

Bits 0-7 are Checksum Data. The chip will accumulate a 16 bit checksum for all Port A accesses.If the chip is configured with a 16 bit CPU interface,the most significant byte is read on the Port A 0-7 15 data lines. If data is written directly to this register it is added to the current contents ratherthan overwriting them. It is important to note thatthe Checksum Enable bit in the Data Transfer Control

PROGRAMMING THE FIFO CHIP

In general the fifo chip is programmed by writingto the data transfer configuration and controlregisters to enable a data transfer, and by reading

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 V. -127- the data transfer status register at the end of thetransfer to check the completion status. Usually thedata transfer itself will take place with both the PortA and the Port B handshakes enabled, and in this casethe data transfer itself should be done without anyother microprocessor interaction. In someapplications, however, the Port A handshake may not beenabled, and it will be necessary for themicroprocessor to fill or empty the fifo by repeatedlywriting or reading the Fifo Data Register.

Since the fifo chip has no knowledge of any bytecounts, there is no way of telling when any datatransfer is complete by reading any register withinthis chip itself. Determination of whether the datatransfer has been completed must therefore be done bysome other circuitry outside this chip.

The following C language routines illustrate howthe parity FIFO chip may be programmed. The routinesassume that both Port A and the microprocessor port areconnected to the system microprocessor, and return asize code of 16 bits, but that the hardware addressesthe Fifo., chip as long 32 bit registers.

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -128- scrucc FIFO_regs ( unsigned char config,al,a2la3 ; unsigned char control,bl,b2,b3; unsigned char status,c1,c2,c3; unsigned char ram_access_concrol,dl,d2,d3; unsigned char ram_X_addr,el,e2,e3; unsigned char ram_Y_addr,f1,C2,f3; unsigned long data; unsigned inc checksum, hl; }; //define FIFO1 ((struct FIFO_regs*) FIFO_BASE_ADDRESS) //define //define /(define /(define /(define /(define //define //define //define //define FIFOJRESET 0x80 FIFO_16_BITS 0x08FIFO_CARRY_WRAP 0x40FIFO_PORT_A_ENABLE 0x01FIFO_PORT_B_ENABLE 0x02FIFO_PORT_ENABLES 0x03FIFO_PORT_A_TO_B 0x04FIFO_CHECKSUM_ENABLE 0x40FIFO_DATA_IN_RAM 0x01FIFO FORCE_RAM FULL 0x20 //define PORT_A_TO_PORT_B(fifo) ((fifo-> control ) &amp; 0x04)//define PORT_A_BYTE_ADDRESS (f ifo) ((fifo->ram_access_concrol) &amp; 0x03) ^define P0RT_A_T0_RAM_Y ( fifo) ((fifo->ram_access_control ) &amp;0x04) //define PORT_B_TO_RAM_Y(f ifo) ((fifo-> ram_access_control ) &amp;0x08) /Λ**********************************************************

The following routine initiates a Fifo data transfer using two values passed to ic. conf'ig_data This is the data to be written to theconfiguration register. control_daca This is the data co be written to the Data

Transfer Control Register. If the data transferis to take place automatically using both ChePort Aand Port B handshakes, both data transferenables bits should be set in this parameter. FIFO_initiate_daca_cransfer(config_daca, control_data)unsigned char config_data, control data; { FIF01->config = config_data | FIFO_RESET; /* Set

Configuration value &amp; Reset */

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 -129-

10 FIF01->control - concrol_data &amp; ("FIFO_PORT_ENABLES); /* Set everything but enables */ FIF01->control = concroi_data ; /* Set data transfer enables */ } 7Λ*Λ*Λϋτ*Λ**Λ*Λ******ΛΛ*ΛΑ***ίΐΓ***<ρΛΛΛΑ*ΛΛ·**ΛΛΛ*ΛΛΛΛ**ΓΛΛ****ΛΛ

The following routine forces the transfer of any odd bytes that have been left in the Fifo at the end of a data transfer.

It first disables both ports, then forces the Ram Full bics, andthen re-enables che appropriate Port. \ 15 20 25 30 35 EIFO_force_odd_length_transf er () ( FIF01->control &amp;= "FIFO_PORT_ENABI.ES; /* Disable Ports A Se B */if (PORT_A_T0_P0RT_B (FIF01) ) { if (P0RT_A_T0_RAM_Y(FIF01)) {

FIF01->ram_Y_addr = FIFO_FORCE_RAM_FULL; /* Sec RAM Y full */ )

else FIF01->ram_X_addr = FIFO_FORCE_RAM_FULL ; /* Set RAM X full */ FIFOl->control |= FIFO_PORT_B_ENABLE ; /* Re-Enable

Port B */ } else { if (PORT_B_TO_RAM_Y(FIFO1)) { FIF01->ram_Y_addr = FIFO_FORCE_RAM_FULL ; /* Set RAM Y full */

J

else FIF01->ram_X_addr = FIFO_FORCE_RAM_FULL ; /- Sec RAM X full */ FIF01->control 1° FIF0_P0RT_A_ENABLE. ; /- Re-Enable

Port A */ } /Λ**********************************************************

The following routine returns how many odd bytes have been left in the Fifo at che end of a data transfer. 40 inc FIFO count odd bytes () { int number_odd_bytes;number_odd_bytes=0; if (FIF01->status &amp; FIF0_DATA_IN_RAM) {if (PORT_A_TO_PORT_B(FIFOi)) { number_odd_bytes = (P0RT_A_BYTE_ADDRESS(FIF01)) ;if (PORT_A_TO_RAM_Y(FIFOT)) number_odd_bytes += (FIF01->ram_Y_addr) * 4 ; 45

Attorney Docket No.:AUSP7209 WP1/WSW/AUSP/7209.001 8/24/89-7 -130-

10 ι

IS 20 25 ia 30 35 40 else number_odd_bytes +- (FIF01->ram_X_addr) * 4 ; } else { i f (PORT_B_TO_RAM_Y(FIFO1)) number_odd_bytes - (FIF01->ram_Y_addr) * 4 ; else number_odd_byces - (FIF01->ram X_addr) * 4 ; ) } return (number_odd_bytes); } /***********************************************************

The following roucine cescs the microprocessor interface of

Che chip. It first writes and reads the first 6 registers. Itthen writes Is, 0s, and an address pattern to che SAM, reading thedata back and checking ic.

The test returns a bit significant error code where eachbit represents che address of che registers chat failed.

Bit 0 = config register failed

Bit 1 - control register failed

Bit 2 - status register failed

Bit 3 - ram access control register failed

Bit 4 = ram X address register failed

Bit 5 = ram Y address register failed

Bit 6 - data register failed

Bit 7 = checksum register failed ***********************************************************/ //define RAM_DEPTH 64 /* number of long words in Fifo Ram */ reg_expected_data[6] = { 0x7F, OxFF, 0x00, OxlF, 0x3F, 0x3F }; char FIFO_uprocessor_interface_test() { unsigned long cest_data;char *register_addr;int i; char j,error; FIF01->config = FIFO_RESET; /* reset the chip */ error-0; register_addr -(char *) FIFO1;j-i; /* first test registers 0 thru 5 */ for (i=0; i<6; i++) { *register_addr = OxFF; /* write test data */ if (*register_addr != reg_expected_data[ij) error != j; *register_addr = 0; /* write 0s to register */ if (*regiscer_addr) error != j; 45

Attorney Docket No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7

X -131- *register_addr “ OxFF; /* write teat data again */ if (*regiscer_addr 1 = reg_expected_data(i)) error J = j;FIFOl->config~= FIFO_RESET; /* reset the chip */ if (*register_addr) error j*· j; /* register should be 0 */register_addr++; /* go to next register */ j «-1; } /* now test Ram data &amp; checksum registerstest Is throughout Ram &amp; then test Os */ for (test_data = -1; test_data ! = 1; test_data++) { /* test for Is &amp; Os */ FIF01->config - FIFO_RESET i FIFO_16_BITS ; FIF01->control = FIFO_PORT_A_TO_B; for (i=O;i<RAM_DEPTH;i++) /* write data to RAM */ FIF01->data = test_data; FIF01->control = 0;for (i=0; i<RAM_DEPTH; i++) if (FIF01->data 1= test_data) error J- j; /* read &amp; check data */ if (FIF01->checksum) error | = 0x80; /* checksum should = 0 */ } /* now test Ram data with address patternuses a different pattern for every byte */ test_data=0x00010203; /* address pattern start */ FIF01->config = FIFO_RESET j FIFO_16_BITS i FIFO_CARRY_WRAP;FIFO1—>control = FIFO_PORT_A_TO_B | FIFO_CHECKSUM_ENABLE;for (i=0;i<RAM_DEPTH;i++) { FIF01->daca = test_data; /* write address pattern */ test_data += 0x04040404; 1 test_data=0x00010203; /* address pattern start */ FIFO'l->control = FIFO_CHECKSUM_ENABLE;for (i=0;i<RAM_DEPTH;i++) { if (FIF01->status 1= FIFO_DATA_IN_RAM) error |= 0x04; /* should be data in ram */ if (FIF01->data != test_data) error |= j; /* read &amp;check address pattern */ test_data += 0x04040404; } if (FIF01->checksum != 0x0102). error J= 0x80; /* test checksum of address pattern */ FIF01->config = FIFO_RESET J FIFO_16_BITS ; /* inhibit carry wrap */ FIF01->checksum = OxFEFE; /* writing adds to checksum */

Attorney Oockec No.:AUSP7209 WPl/WSW/AUSP/7209.001 8/24/89-7 I ·; -132- if (FIFOl->checksum) errorif (FIF01->status) error J1return (error); 2-0x80; 0x04; /* checksum should be 0 *//* status should be 0 */

Attorney Docket No.:AUSP7209WPl/WSW/AUSP/7209.001 8/24/89-7

Contents45

36 members in 11 offices

This recordMember, by publication datePriority claim1 more office is in the list below

Priority claims6

Priority claims
Document	Office	Kind	Date
40495989	United States of America	A	1989-09-08
10764690	Israel	A	1990-08-21
107646	–	–	–
40495989	–	–	–
IL19900107646	–	–	–
US19890404959	–	–	–

Members36

Family members, oldest first
Document	Office	Kind
CA2066443A1	Canada	A1	1991-03-09
WO9103788A1	World Intellectual Property Organization (WIPO)	A1	1991-03-21
AU6412590A	Australia	A	1991-04-08
IL95447D0	Israel	D0	1991-06-30
EP0490973A1	European Patent Office (EPO)	A1	1992-06-24
US5163131A	United States of America	A	1992-11-10
KR920704227A	Republic of Korea	A	1992-12-19
JPH05502525A	Japan	A	1993-04-28
EP0490973A4	European Patent Office (EPO)	A4	1993-10-20
IL107643D0	Israel	D0	1994-02-27
IL107644D0	Israel	D0	1994-02-27
IL107645D0	Israel	D0	1994-02-27
IL107646D0	Israel	D0	1994-02-27
AU647414B2	Australia	B2	1994-03-24
IL95447A	Israel	A	1994-05-30
AU6590594A	Australia	A	1994-09-01
US5355453A	United States of America	A	1994-10-11
IL107644A	Israel	A	1995-12-31
IL116288D0	Israel	D0	1996-03-31
IL107643A	Israel	A	1996-05-14
AU670376B2	Australia	B2	1996-07-11
IL107645A	Israel	A	1996-09-12
IL107646A	Israel	A	1997-03-18
IL116288AThis record	Israel	A	1998-02-08
EP0490973B1	European Patent Office (EPO)	B1	1998-02-25
AT163485T	Austria	T	1998-03-15
ATE163485T1	Austria	T1	1998-03-15
DE69032069D1	Germany	D1	1998-04-02
EP0837402A2	European Patent Office (EPO)	A2	1998-04-22
US5802366A	United States of America	A	1998-09-01
KR100201574B1	Republic of Korea	B1	1999-06-15
HK1011772A1	Hong Kong, China	A1	1999-07-16
US5931918A	United States of America	A	1999-08-03
EP0837402A3	European Patent Office (EPO)	A3	2000-08-09
US2002083111A1	United States of America	A1	2002-06-27
CA2066443C	Canada	C	2003-10-21

9 legal events, as the office reported them to INPADOC

Over the term

Point at a mark for the event

Events

Legal events, newest first
	Event	Code
2008-11-27	Patent renewedKB	KB
2004-11-10	Patent renewedKB	KB
2004-06-20	Patent renewedKB	KB
2004-01-04	Application for restoration - patent lapsed through non-payment of renewal fees (section 60, patents law, 5727-1967)LapsedNE	NE
2003-02-12	CorrigendumHK	HK
2002-08-14	Patent not in force due to non-payment of renewal feesMM9K	MM9K
2000-06-29	Patent renewedKB	KB
1998-12-27	Patent renewedKB	KB
1998-07-15	Patent grantedGrantedFF	FF

Abstract

Term

I

I -1-

WILLIAM M. PITTS, STEPHEN E. BLIGHTMAN

CROSS-REFERENCE TO RELATED APPLICATIONS

2. ENHANCED VMEBUS PROTOCOL UTILIZING

3. BUS LOCKING FIFO MULTI-PROCESSOR COMMUNICATIONS

BACKGROUND OF THE INVENTION

V

SUMMARY OF THE INVENTION

BRIEF DESCRIPTION OF THE DRAWINGS

V

PREFERRED EMBODIMENT-OVERALL HARDWARE ARCHITECTURE

OVERALL SOFTWARE ORGANIZATION AND DATA FLOW

V -29-

I -30-

NETWORK CONTROLLER HARDWARE ARCHITECTURE

NETWORK CONTROLLER OPERATION

I

FILE CONTROLLER OPERATION

STORAGE PROCESSOR HARDWARE ARCHITECTURE

STORAGE PROCESSOR OPERATION

V -73-

I

X

SYSTEM MEMORY

MESSAGE PASSING

V V. -91-

10:14 RESERVED - NOT DEFINED

STATUS MEANING

0 NO ERRORS

1 ILLEGAL OP CODE

2 BUS OPERATION TIMEOUT

3 BUS OPERATION ERROR

4 DATA PATH PARITY ERROR

OFFSET 0 4 8

V

V -108-

I

APPENDIX C

SIGNAL DESCRIPTIONS

DESCRIPTION OF OPERATION

PROGRAMMABLE REGISTERS

V -123-

PROGRAMMING THE FIFO CHIP

J

Priority claims6

Members36

Over the term

Events

Numbers

Titles

Classification