Method and device for establishing links in an ad hoc wireless system

<u>Field of the Invention</u>

0001The present invention relates to the field of communications, and more particularly, to a network of mobile communication systems operating with directional antennas.

<u>Background of the Invention</u>

0002Time division multiple access (TDMA) is one example of an access scheme used for establishing communication links between wireless mobile communication systems. Communication links between the wireless mobile communication systems are established within a series of time frames. Each time frame is divided into time slots, with each wireless mobile communication system being assigned at least one time slot.

0003An omni-directional antenna is typically used by a wireless mobile communication system so that information transmitted by one mobile communication system is received by all the other mobile communication systems. When the mobile communication systems are operating at a fixed frequency, they must take turns transmitting within their respective time slots to prevent channel interference.

0004To improve quality of a communications link between two wireless communication systems, a directional antenna may be used. The directional antenna provides an increased antenna gain in a desired area that is limited in coverage while decreasing the antenna gain towards the remaining area.

0005Document <patcit id="pcit0001" dnum="US6317436B"><text>US6317436</text></patcit> discloses a method and system for managing communication resources between nodes in a network, and more particularly to a dynamic distributed multichannel time division multiple access (TDMA) slot assignment method is presented. The method and apparatus include a set of higher level heuristics that enable the wireless channel access scheme to address predetermined characteristics of the wireless channel access system. These predetermined heuristics include using bootstrap slots, adaptive broadcast cycles, channelized neighborhoods, standby slots, speculation slots, neighbor segregation, hard circuits, and soft circuits.

0006Document XP010376001, YOUNG-BAE KO ET AL: "Medium access control protocols using directional antennas in ad hoc networks", discloses MAC protocol for use with wireless ad-hoc networks and directional antenna. The protocol disclosed therein features sending ACK by a directional antenna. According to the MAC protocol disclosed therein, if the directional antenna at a certain node received an RTS or CTS regarding ongoing communication between two other nodes, then said node will not use said antenna for a transfer until the ongoing transfer is completed.

0007Document <patcit id="pcit0002" dnum="EP0682425A"><text>EP0682425</text></patcit> discloses a communication method including the retransmission of signaling messages that data terminals (e.g., modems) might exchange prior to exchanging user data, wherein it is controlled by arranging a data terminal that receives multiple copies of the same signaling message so that it saves only the first copy thereof, but transmits a response to the sending terminal following receipt of each such copy. The sending terminal terminates its transmission of the signaling message upon receipt of a response from the receiving terminal.

0008Document <patcit id="pcit0003" dnum="WO0239668A"><text>WO0239668</text></patcit> discloses an apparatus, a method, and a computer program product for effective communication routing of unicast and broadcast data traffic in wireless ad-hoc networks are presented. The routing technique separates the signaling and data transmission portions of data frame such that the length of the signaling portion is independent of the length of the data portion. In the signaling portion, reservations are able to be performed and confirmed, while the data portion also includes a reservation confirmation portion which allows reservations made in during the signaling portion of the frame to be confirmed immediately prior to transmission of the data. In addition, the present invention also provides a mechanism for dynamically adjusting reservations.

0009Document <patcit id="pcit0004" dnum="WO02063836A"><text>WO 02063836</text></patcit> discloses a data communication system and a method for use in a data communication system is disclosed. The system comprises a plurality of nodes provided with means for wireless data communication. At least one directed wireless communication link (DCHB) can be formed between a node and another node for data communication. In the method control information is generated, said information associating with at least one node of the plurality of nodes. Said control information includes at least a capacity parameter that associates with the capacity of said at least one node. Data communications between the nodes are scheduled based on said control information and data is communicated between the nodes via said at least one directed link based on said scheduling.

0010Document <patcit id="pcit0005" dnum="WO0128170A"><text>WO 01/28170</text></patcit> discloses a scheduling protocol according to which scheduling packets are exchanged among neighboring nodes of a computer network. The scheduling packets include descriptions of a transmitting node's 2-hop neighborhood within the computer network, and nodes are able to determine transmission schedules from information received via said scheduling packets. Transmission schedules may be determined, at least in part, because nodes advertise their availability using the scheduling packets. The identification of reserved communication times and/or channels is made after eliminating any conflicting scheduled transmissions for those communication times and/or channels.

0011International Patent Application Publication No. <patcit id="pcit0006" dnum="WO02063836A"><text>WO 02/063836</text></patcit> discloses communication between nodes in a network based on a combination of TDMA and space division multiple access (SDMA). As part of this methodology, TDMA is used to define the time slot allocation s for transmissions and SDMA defines the allocation of transmission resources such that data associated with more than a single transmission may be carried in a single time slot.

0012<patcit id="pcit0007" dnum="US5767807A"><text>U.S. Patent No. 5, 767, 807 to Pritchett</text></patcit> discloses phased array antennas being used for establishing communication links within a network of wireless communication systems. The phased array antenna includes parasitic elements for selectively controlling the antenna pattern. The phased array antenna radiates an omni-directional signal when all of the parasitic elements are in a high impedance state, and radiates a directional signal when a selected number of parasitic elements are placed in a lower impedance state in response to switching circuits.

0013More particularly, the Pritchett '807 patent discloses the acquisition, by a fixed initiating wireless communication system from a fixed receiving wireless communication system, of a list of the wireless communication systems operating in the network and a corresponding respective time slot list for each wireless communication system. A table is then created based upon the list for scheduling time slots among the wireless communication systems.

0014Scheduling time slots for wireless communication systems operating with directional antennas, particularly when the wireless communication systems are mobile, is complex. In such a dynamic network, mobile communication systems are continuously entering into and dropping out of the network. Furthermore, reliable confirmation messages are needed.

<u>Summary of the Invention</u>

0015It is an object of the present invention to schedule time slots and provide reliable confirmation messages for communication link demands in a wireless mobile ad hoc network.

0016This and other objects, advantages and features in accordance with the present invention are provided by a wireless communication network including a plurality of mobile nodes each including a transceiver, a directional antenna, e.g. a phased array antenna, connected to the transceiver, and a controller connected to the transceiver. The controller schedules a respective semi-permanent time slot for each time frame to establish a communication link with each neighboring mobile node and leaving at least one available time slot in each time frame, and schedules the at least one available time slot to also serve the communication link with a neighboring mobile node based upon link communications demand. An initiating mobile node transmits a request for time slots to the receiving mobile node, which transmits a reply to the initiating mobile node. The initiating mobile node transmits a confirmation to the receiving mobile node, and the receiving mobile node transmits the reply again if the confirmation is not received. Alternatively, the receiving mobile node may transmit an acknowledgment to the initiating mobile node, and the initiating mobile node transmits the confirmation again if the acknowledgment is not received.

<u>Brief Description of the Drawings</u>

0017<ul id="ul0001" list-style="none" compact="compact"><li><figref idref="f0001">FIG. 1</figref> is a diagram illustrating a wireless mobile ad hoc network in accordance with the present invention.</li><li><figref idref="f0001">FIG. 2</figref> is a more detailed block diagram illustrating a wireless mobile node in accordance with the present invention.</li><li><figref idref="f0002">FIG. 3</figref> is a diagram illustrating a frame of time slots in accordance with the present invention.</li><li><figref idref="f0002">FIG. 4</figref> illustrates the scheduling of available time slots to the network diagram illustrated in <figref idref="f0001">FIG. 2</figref> in accordance with the present invention.</li><li><figref idref="f0003">FIG. 5</figref> is a top-level state diagram for the scheduling of semi-permanent time slots and available time slots in accordance with the present invention.</li><li><figref idref="f0003">FIG. 6</figref> is a diagram illustrating a semi-permanent time slot scheduling process in accordance with the present invention.</li><li><figref idref="f0004">FIG. 7</figref> is a diagram illustrating a semi-permanent time slot being scheduled for a new communication link in accordance with the present invention.</li><li><figref idref="f0004">FIG. 8</figref> is a diagram illustrating an available time slot scheduling process in accordance with the present invention.</li><li><figref idref="f0005">FIG. 9</figref> is a diagram illustrating an available time slot being added to a communications link in accordance with the present invention.</li><li><figref idref="f0005">FIGS. 10 and 11</figref> are diagrams illustrating a semi-permanent time slot being scheduled for a new communications link based upon multiple simultaneous antenna beams from a phased array antenna in accordance with the present invention.</li></ul>

<u>Detailed Description of the Preferred Embodiments</u>

0018The present invention will now be described more fully hereinafter with reference to the accompanying drawings in which preferred embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Like numbers refer to like elements throughout and prime notations are used in alternate embodiments. The dimensions of layers and regions may be exaggerated in the figures for greater clarity.

0019Referring initially to <figref idref="f0001">FIGS. 1-2</figref>, a wireless mobile communication network <b>10</b> comprises a plurality of wireless mobile nodes <b>12a-12h</b>. Each mobile node <b>12a-12h</b> comprises a transceiver <b>14</b>, a directional antenna <b>16</b> connected to the transceiver, and a controller <b>18</b> connected to the transceiver.

0020The controller <b>18</b> includes a semi-permanent time slot unit <b>18a</b> for scheduling a respective semi-permanent time slot for each time frame for establishing a communication link with each neighboring mobile node while leaving at least one available time slot in each time frame. An available time slot unit <b>18b</b> schedules the at least one available time slot to also serve the communication link with a neighboring mobile node based upon link communications demand. In addition, the controller <b>18</b> includes an antenna aiming unit <b>18c</b> for aiming the directional antenna toward each neighboring mobile node during communication therewith.

0021Parallel operations can reduce time slot allocation delay. Accordingly, the semi-permanent time slot unit <b>18a</b> may initiate one or more semi-permanent time slot requests for respective time frames to establish the communication link with each neighboring mobile node and leaving at least one available time slot in each time frame, while processing multiple received semi-permanent time slot requests from neighboring mobile nodes. The available time slot unit <b>18b</b> may initiate one or more available time slot request to also serve the communication link with the neighboring mobile node based upon link communications demand, while processing multiple received available time slot requests from neighboring mobile nodes.

0022In other words, a node can have one or more pending demand available requests and semi-permanent requests that it initiated while processing multiple received requests. This may sometimes result in temporarily allocating a given time slot to more than one neighbor. However, this conflict may be eliminated by confirmation messages which indicate the selection of one neighbor node for the time slot, as is discussed in more detail below.

0023Reliable confirmation messages may be provided with a couple of different approaches. An initiating mobile node transmits a request for time slots to the receiving mobile node, which transmits a reply to the initiating mobile node. The initiating mobile node transmits a confirmation to the receiving mobile node, and the receiving mobile node transmits the reply again if the confirmation is not received. Alternatively, the receiving mobile node may transmit an acknowledgment to the initiating mobile node, and the initiating mobile node transmits the confirmation again if the acknowledgment is not received.

0024If two nodes simultaneously initiate time slot requests to each other, the time slot request collisions should be handled reliably. The controller <b>18</b> waits a time period to resend another request upon simultaneously sending a request to an other mobile node and receiving a request from the other mobile node without receiving a corresponding reply. During this time period, incoming time slot requests may be processed. After the period ends, a new request may be sent to the other node if no request was already received from that node or if no time slot allocation was made to that node. If a delayed request reaches the front of the queue, the controller <b>18</b> checks to see if a time slot allocation has already been made to that node. If so, the delayed request is discarded. Also, if the request is destined for a node that is no longer a neighbor by the time the request reaches the front of the queue, the delayed request is discarded.

0025An interference detection unit <b>18d</b> is included to detect interference in time slots for communication with neighboring mobile nodes. The controller <b>18</b> coordinates the scheduling of time slots based upon detected interference. The interference detection unit <b>18d</b> may measure a signal-to-interference ratio and/or a packet error rate. The packet error rate may be based upon cyclic redundancy check (CRC) failures. Also, the interference detection unit <b>18d</b> may compare detected interference to a threshold. Preferably, the controller switches an order of communication between nodes for a given time slot based upon detected interference, and may also coordinate scheduling of new time slots based upon detected interference after switching the order of communication.

0026A traffic coordination unit <b>18e</b> coordinates communication with each neighboring mobile node by allocating time slots to the time slot unit based upon link communications demand. The controller <b>18</b> coordinates the scheduling of time slots based upon based upon allocated time slots. The traffic coordination unit <b>18e</b> may allocate a bulk set of time slots based upon an increased link communications demand, and/or may request a bulk set of time slots from neighboring mobile nodes based upon an increased link communications demand. Also, the traffic coordination unit <b>18e</b> may increase a maximum number of time slots, reallocate time slots, and/or allocate half time slots based upon increased link communications demand. Such increased link communications demand may include streaming video and/or high rate sensor data.

0027The wireless mobile nodes <b>2a-12h</b> are operating in a mobile environment. These systems may be ground based and/or airborne, whereby they are continuously entering into and dropping out of the network <b>10.</b> The directional antenna <b>16</b> may be a phased array, a dish or horn antennas, for example. Transmission via a directional antenna <b>16</b> enables the RF signal to be focused in a desired direction.

0028By selectively controlling the direction of the antenna pattern between a pair of wireless mobile communication systems for establishing a communications link therebetween, additional communication links may be established between other wireless communication systems within the same scheduled semi-permanent time slot. This is illustrated by communication link <b>27</b> operating in time slot 1 between mobile nodes <b>12c</b> and <b>12e</b>, and communication link <b>29</b> also operating in time slot 1 between mobile nodes <b>12a</b> and <b>12b</b>, as best illustrated in <figref idref="f0001">FIG. 1</figref>. This feature of the present invention advantageously allows the resources of the wireless mobile communication network <b>10</b> to be better utilized.

0029The controller <b>18</b> limits the number of communication links for each wireless mobile node <b>12a-12h</b> within each time frame based upon a total number of time slots within the frame. The advantage of limiting the number of communication links to a fraction of the total number of time slots within the time frame significantly simplifies the scheduling of time slots with neighboring nodes.

0030The number of communication links for each wireless mobile node <b>12a-12h</b> within each time frame is less than or equal to N, and the total number of time slots within each frame is greater than or equal to 2N-1. In addition to simplifying the scheduling of time slots, this type of distributed scheduling avoids conflicts.

0031Distributed scheduling allows any two pair of wireless mobile nodes, such as <b>12a</b> and <b>12b,</b> for example, to schedule a semi-permanent time slot without having to communicate with any other wireless mobile node. In other words, there is no centralized master/slave type of coordination with all of the wireless mobile nodes <b>12a-12h</b> for scheduling the semi-permanent time slots. Since the time slots among the wireless mobile nodes <b>12a-12h</b> are scheduled in a distributed fashion, there is no single point of failure in the wireless mobile communication network <b>10</b>.

0032The controller <b>18</b> may prioritize the communication links and drop one of the communication links based upon the prioritization for making available a semi-permanent time slot for establishing a communication link with a new neighboring mobile node. Prioritization of the communication links will be addressed in greater detail below. In addition, the controller <b>18</b> may also prioritize the communication links and schedule the at least one available time slot based upon this prioritization.

0033The controller <b>18</b> may also schedule one of the semi-permanent time slots as an available time slot if a number of the communication links is less than N. This advantageously supports communication link demands on an as needed basis for the existing communication links. However, the controller <b>18</b> may reschedule the demand assigned time slot back to a semi-permanent time slot if the number of the communication links is again equal to N, as will also be discussed in greater detail below.

0034Each communication link is formed by an initiating mobile node, such as node <b>12a</b>, and a receiving mobile node, such as node <b>12b</b>, and the initiating mobile node transmits a list of available semi-permanent time slots to the receiving mobile node. The receiving mobile node <b>12b</b> then transmits selection of one of the semi-permanent time slots to the initiating mobile node. The initiating mobile node <b>12a</b> then confirms selection of the selected semi-permanent time slot to the receiving mobile node.

0035Each mobile node may further comprise an omni-directional antenna <b>20</b> connected to the transceiver <b>14</b> for exchanging positional information with other neighboring mobile nodes. Other information that may be exchanged includes resource requirements and detection of the presence of a potential new neighbor node. In addition, the phased array antenna <b>16</b> may simultaneously generate multiple antenna beams, wherein the controller <b>18</b> aims the phased array antenna to multiple neighboring mobile nodes within a scheduled semi-permanent time slot.

0036The interference detection unit <b>18d</b> detects and avoids interference for collinear node pairs within the beamwidth and allocated the same time slot. For example, referring to <figref idref="f0001">Fig. 1</figref>, nodes <b>12a</b> and <b>12e</b> transmitting to nodes <b>12b</b> and <b>12c</b>, respectively during their half of the same assigned time slot 1. With wide enough antenna beamwidths, both nodes <b>12b</b> and <b>12c</b> may simultaneously hear transmissions from both nodes <b>12a</b> and <b>12e</b>. The interference detection unit <b>18d</b> may measure the Signal-to-Interference Ratio (SINR) at the physical layer during time slot usage. Alternatively, the packet error rate can be measured at the link layer based upon CRC check failures. If these measurements violate a specified threshold, the slot may be declared bad.

0037However, because fading may cause a single slot to fail this test, it may be desired to declare excessive interference in a slot if m of n trials of this slot suffer degradation. At this point, the controller <b>18</b> attempts to avoid the interference. The order of Tx/Rx at both ends of the link may be switched for the time slot. If such switching fails, a new time slot may be coordinated. Of course both of these changes should be made probabilistically to reduce the likelihood that both node pairs try to make the same change at the same time and thus remain in conflict.

0038The traffic coordination unit <b>18e</b> manages unbalanced traffic loads that may be generated by streaming video or high rate sensor data. Coordination mechanisms are provided to permit each half-duplex link to allocate a time slot in any Tx/Rx split of traffic. Also, the maximum number of time slots may be increased to a number above the minimum to create more demand time slots. Subslotting would permit an effective increase or decrease in the maximum number of time slots as nodes may "steal" subslots from a semi-permanent allocated time slot to reallocate to a demand time slot. Moreover, a reservation protocol could be used together with link scheduling procedures to indicate allocation of resources for a high rate stream at each node along a path from a source to a destination node by requesting and allocating a bulk set of time slots and/or subslots at each node along the path to accommodate the high rate stream. For reserved resources, separate queues and a queue service discipline may be necessary to insure that the capacity required by the stream is delivered.

0039The invention is also directed to a method for establishing communication links for a plurality of mobile nodes <b>12a-12h</b>, with each mobile node comprising a transceiver <b>14,</b> a phased array antenna <b>16</b> connected to the transceiver, and a controller <b>18</b> connected to the transceiver. The method comprises for each mobile node <b>12a-12h</b> scheduling a respective semi-permanent time slot for each time frame to establish a communication link with a neighboring mobile node and leaving at least one available time slot in each time frame.

0040The at least one available time slot is preferably scheduled to serve the communication link with a neighboring mobile node based upon link communications demand. The phased array antenna <b>16</b> is aimed toward each neighboring mobile node <b>12a-12h</b> during communication therewith. Each time frame may have up to N semi-permanent time slots and at least 2N-1 available time slots.

0041The method may also include initiating one or more semi-permanent time slot requests for respective time frames to establish a communication link with each neighboring mobile node and leaving at least one available time slot in each time frame, while processing multiple received semi-permanent time slot requests from neighboring mobile nodes, and initiating at least one available time slot request to also serve the communication link with a neighboring mobile node based upon link communications demand, while processing multiple received available time slot requests from neighboring mobile nodes.

0042The directional/phased array antenna <b>16</b> is aimed toward each neighboring mobile node <b>12a-12h</b> during communication therewith, interference is detected in time slots for communication with neighboring mobile nodes, and the scheduling of new time slots is coordinated based upon detected interference. The interference detection unit <b>18d</b> may measure a signal-to-interference ratio and/or a packet error rate. The packet error rate may be based upon cyclic redundancy check (CRC) failures. Also, the interference detection unit <b>18d</b> may compare detected interference to a threshold. Preferably, the controller <b>18</b> switches an order of communication between nodes for a given time slot based upon detected interference, and may also coordinate scheduling of new time slots based upon detected interference after switching the order of communication. Also, communication with each neighboring mobile node <b>12a-12h</b> may be coordinated by allocating time slots for scheduling based upon link communications demand.

0043The method further includes having each node prioritize the communication links and drop one of the communication links based upon the prioritization for making available a semi-permanent time slot for establishing a communication link with a new neighboring mobile node. In addition, an available time slot that is currently scheduled to serve a particular communication link may be reassigned to another communication link based on link demand. This advantageously allows any mobile node to accommodate variations in communication link demands.

0044Scheduling of the semi-permanent time slots and the available time slots will now be discussed in greater detail. Details on steering the directional antennas <b>16</b> toward a receiving mobile node <b>12a-12h</b> will be omitted since this feature of the present invention is readily understood by one skilled in the art.

0045For purposes of discussion, it will be assumed that the directional antenna <b>16</b> is a phased array antenna. As readily understood by one skilled in the art, a phased array antenna <b>16</b> includes a plurality of antenna elements and respective phase shifters that can be adjusted for producing a steerable antenna beam in a desired direction. The phased array antenna <b>16</b> steers or scans the antenna pattern without physically moving the antenna.

0046Also for purposes of discussion, a number of assumptions about the wireless mobile communication network <b>10</b> are made. First, there is a single frequency band that is a high data rate channel that is shared by all the wireless mobile nodes <b>12a-12h</b>. This type of transmission channel is time shared between all the wireless mobile nodes <b>12a-12h</b> for both transmit and receive. All transmission slots are scheduled in advance.

0047An assumption is also made that a separate low data rate overhead channel is provided. This overhead channel can be used for node discovery, net entry, and exchange of various other data link control overhead information including resource requests. This overhead channel is provided via an omni-directional antenna <b>20</b>. Good global timing reference is also known at all nodes. The terms wireless mobile nodes and wireless mobile communications systems <b>12a-12h</b> are interchangeable throughout the following discussion.

0048The wireless mobile communication network <b>10</b> also includes the capability for locating and tracking mobile nodes so that the phased array antennas <b>16</b> can be pointed accurately when a scheduled time slot is available. As noted above, a detailed discussion on the pointing/tracking will not be provided herein.

0049An assumption is also made that the phased array antennas <b>16</b> have zero beamwidth. This assumption will be relaxed later. Consequently, we can assume that a transmission by a given mobile node will be received only by the neighbor mobile node to which it is attempting to transmit. This allows a less restrictive set of constraints on the scheduling of time slots. Each communications link will be labeled with a number which represents a scheduled time slot for transmitting and receiving data therein.

0050The constraints are as follows. No node may have more than one communications link labeled with the same time slot number. A given time slot assignment will apply to a half duplex link between two mobile nodes, and be used alternately by the two nodes for transmit and receive. These two constraints imply that a time slot assigned by a mobile node to one of its neighboring nodes is constrained by the previous time slot assigned by that node to other links.

0051The scheduling of time slots for the phased array antenna <b>16</b> is illustrated in <figref idref="f0001">FIG. 1</figref>, which shows a network <b>10</b> with link connectivity based upon scheduled time slots. The time slots are scheduled so that the wireless mobile nodes <b>12a-12h</b> know when to point their respective phased array antenna <b>16</b> toward a neighboring wireless mobile node.

0052The communication links are assumed to be bidirectional and are used in a half duplex fashion where each time slot number represents a time slot and a transmission opportunity in each direction occurring in that time slot. The term N_frame will be used to denote the maximum link index or the maximum number of time slots within a frame. In the case of this example, N_frame = 6.

0053<figref idref="f0002">FIG. 3</figref> illustrates a representative frame of time slots. In the simplest formulation, each epoch or frame has n slots and the value of n is set to N_frame. In the figure we also show how a time slot is used for the link connecting to nodes labeled as nodes A and B. Each time slot is divided into two mini-slots <b>22a</b>, <b>22b.</b> The first mini-slot <b>22a</b> (e.g., half of the time slot) is used for transmissions from node A to B. Then the direction of the link is reversed and the second mini-slot <b>22b</b> is used for transmissions from node B to A.

0054During the transmission periods, multiple packets can be transmitted. As indicated, each mini-slot <b>22a</b>, <b>22b</b> also contains a guard time <b>24a</b>, <b>24b</b> selected according to the following considerations. The maximum range between any pair of nodes determines the maximum propagation delay that must be accommodated. A maximum range of 100 miles corresponds to about 0.5 ms of propagation delay. A guard time is allocated for each mini-slot <b>22a</b>, <b>22b</b> to accommodate uncertainty of propagation delay and unequal propagation delays between all pairs of nodes.

0055At a maximum range of 100 miles, a guard time of 0.5 ms is needed. The guard time allocation for a maximum range of 100 miles implies the need to make the mini-slots <b>22a, 22b</b> on the order of 2 to 4 ms to minimize the channel efficiency loss. As an example, if we assume a 50 Mb/s data rate on the communication links and a maximum range of 100 miles, then a 4 ms mini-slot implies 200,000 bits/mini-slot (250 mini-slots per second). Then the mini-slot would contain a 25,000 bit guard time and 175,000 bits of mission data.

0056The controller <b>18</b> may also bias each established link to assign priority when the available time slots are scheduled. As will be discussed in greater detail below, semi-permanent (SP) time slots and available or demand assigned (DA) time slots are provided within each frame. A stated objective is to increase reuse of time slots among several nodes at the same time. While the mobile network <b>10</b> in <figref idref="f0001">FIG. 1</figref> is limited in the total number of nodes and communication links, there are a number of cases of parallel usage of time slots. For example, time slots 1 and 2 are simultaneously each used on 3 different communication links, and time slot 6 is used on only one link. All the other time slots are assigned to two communication links. We can define a reuse factor which indicates the average level of reuse as a ratio of the total number of time slot assignments in the network (N_frame) to the number of assigned time slots (Num_Slots_Assigned): <maths id="math0001" num="(1)"><math display="block"><mi>R</mi><mo>=</mo><mfrac><mi mathvariant="italic">Num_Slots_Assigned</mi><msub><mi mathvariant="italic">N</mi><mi mathvariant="italic">frame</mi></msub></mfrac></math><img file="EP1559211B1_D0001.tif" /></maths> For the example network <b>10</b> in <figref idref="f0001">FIG. 1</figref>, the reuse approach provides a reuse factor of R = 14/6 = 2.333, indicating that on the average there are slightly more than two simultaneous users of each time slot in the network schedule. It is obvious that the reuse factor calculated for any specific scheduling algorithm will be highly dependent on the network size and topology. A full comparative evaluation should consider a variety of network sizes, and topologies.

0057A lower bound on the value of N_frame for any graph can be determined by noting that each node requires at least as many time slots as the node has neighbors, i.e., the node requires a number of time slots at least equal to its degree. Then N_frame must be at least as great as the maximum node degree over the entire graph. Thus, denoting the degree of node i by d_i the lower bound on N_frame is <maths id="math0002" num="(2)"><math display="block"><msub><mi>N</mi><mi mathvariant="italic">frame</mi></msub><mo>≥</mo><msub><mi>max</mi><mi>i</mi></msub><mfenced open="{" close="}"><msub><mi mathvariant="italic">d</mi><mi mathvariant="italic">i</mi></msub></mfenced></math><img file="EP1559211B1_D0002.tif" /></maths> For the example network <b>10</b> illustrated in <figref idref="f0001">FIG. 2</figref> the reuse portion is assigned the scheduling with N_frame equal to the minimum number of time slots that must be used according to equation (2). Note that several nodes, namely all nodes but node 1, are assigned less than the full set of time slot. Thus, an enhanced scheduling algorithm may be able to assign additional slots to some of the links without introducing conflicts in scheduling.

0058The following discussion focuses primarily on the scheduling of time slots for generating the link schedules. Other parts of the overall phased array network problem that ultimately must be addressed include: 1) node and neighbor discovery, 2) net entry, 3) overhead channel format and protocol including protocol exchanges for scheduling updates, and 4) tracking and location of neighbor nodes (may include assistance of phased array antenna <b>16</b>), and 5) a routing algorithm for a dynamic network topology.

0059The approach for scheduling time slots according to the present invention is based upon the following principles. First, a specified number of time slots are allocated as semi-permanent (SP) time slots scheduled for a given link. The rest of the available time slots (DA) may be allocated on a demand-assigned basis to those nodes/links that need them most. This allows flexibility in shifting the schedule on an as needed basis. Secondly, as discussed above, a limit on the maximum number of semi-permanently assigned time slots is established. This limit is a parameter that is selected based upon a specific network. This limit is also the upper limit on the number of allowable neighbor nodes, with a single SP time slot per node.

0060Third, as also discussed above, a limit on the maximum number of time slots per frame is established. This limit is a parameter that is also selected based upon a specific network. This limit is important for establishing a limit on latency since it determines the maximum revisit time for a link transmit opportunity.

0061Fourth, the relationship between the number of total time slots per frame, N_frame, and the limit on the maximum number of semi-permanently assigned time slots per frame is chosen so that the scheduling of the semi-permanently assigned time slots is greatly simplified and scheduling conflicts may be significantly avoided even with distributed scheduling.

0062By limiting the maximum number of semi-permanently assigned time slots per node to a certain fraction to the total number of time slots per frame, the process of distributively assigning semi-permanently assigned time slots is greatly simplified. The upper limit on the number of the semi-permanently assigned time slots (and, therefore, the maximum number of allowable neighbor nodes) will be denoted by N. We will consider values of N_frame such that: <maths id="math0003" num="(3)"><math display="block"><msub><mi>N</mi><mi mathvariant="italic">frame</mi></msub><mo>≥</mo><mn>2</mn><mo>⁢</mo><mi mathvariant="normal">N</mi><mo>-</mo><mn>1</mn></math><img file="EP1559211B1_D0003.tif" /></maths>

0063Assume that all nodes <b>12a-12h</b> in the network <b>10</b> are connected by directional links, where each node has a single beam phased array antenna <b>16</b> with beam sharing by time hopping and pointing to its neighbor nodes. Further, assume that the number of neighbors is equal to N, and the limit on the allowable number of semi-permanent time slots (with one SP time slot allocated per neighbor) is fixed.

0064If the fixed value of N_frame satisfies equation (3), then all nodes can select a different semi-permanent time slot for each of these links by mutual agreement with the neighbor for that link without regard to what links other nodes are selecting more than one-hop away. This allows each node to select its semi-permanent time slot for the link to a neighbor node in a very direct fashion by communicating only with that neighbor node. This process can be followed for up to N neighbor nodes.

0065The key is recognizing that as the value of N_frame increases for a fixed value of N, there are fewer constraints on the ability of a node to select a time slot that does not conflict with a neighbor's choice of a time slot. A node selecting a time slot for a new link must select a time slot that it is not currently being used and that the neighbor is not currently using.

0066If a node currently has m neighbors with a single time slot assigned to each of these links to the neighbors and is adding a link to a new neighbor node, then the neighbor node can be using at most (N-1) time slots. Thus, if N_frame is greater than (m+N-1), then there will be at least one more time slot available that the node can assign to the new link. The worst case in this assignment process is when the node already has (N-1) neighbors and is assigning the time slot for the N^th neighbor node. In this case N_frame must satisfy equation (3) for an additional time slot to be guaranteed to be available for assignment to the link to the N^th neighbor.

0067Some additional observations will be made about how this property can be exploited in the disclosed time slot scheduling approach. First, a node need only coordinate the selection of the semi-permanent time slot to be assigned for a directional link to a neighbor with that neighbor. The node requesting the link might, for example, send to the neighbor the list of suggested time slots for the link. This is based upon those time slots not being used for SP assignments. There could be some ordering of this list based upon other factors to be discussed below, but this is not necessary. The neighbor node can then select from this list the time slot it prefers and return a reply with this selection. This allows us to define a straightforward, fully distributed algorithm for scheduling the semi-permanent time slots.

0068If a node has less than N neighbors, then more than one of its N allowed semi-permanent time slots could be assigned on individual links. However, in this case there is no guarantee that all N assignments can be made via neighbor-to-neighbor node coordination without some conflicts. For example, if N=6 and a node had only 3 neighbors but each of these neighbors each had 6 neighbors, then the node would be able to assign only one time slot to each of the links with its 3 neighbors. In order to simplify our algorithm, we will not allow scheduling of more than one SP time slot per link. However, all unused time slots may be allocated as available time slots.

0069For certain networks with very large numbers of nodes where the number of potential neighbors will be much larger than the limit N, there will also be a topology control problem to deal with. The node will be faced with choosing, from among the potential neighbors, those neighbors that create the optimum network topology. This topology control problem also is related to the concept of optimizing an energy efficient network. In the case where the number of potential neighbors is much larger than the limit N, a topology control function can be used to select the neighbor node to connect to.

0070If we assign to N_frame the minimum value allowed by (3), then each node will be allowed to have a maximum of N semi-permanent time slots and a total of (2N-1) time slot assignments. The demand assigned time slots will be assigned on a basis to best accommodate the traffic load. Of course, assigning a much larger value of N_frame is also an option. In this case, there will be many more time slots available for demand assignment. There may be applications for which this is a desirable way to configure the network.

0071As with the semi-permanent time slots, the node need only coordinate the selection of the available time slots to be assigned for a directional link to a neighbor with that neighbor. This means that a neighbor will send a request to the neighbor for the time slot assignment over the directional link, and receive either a grant of the assignment or a denial of the request over the same link.

0072A node requesting the allocation of an available time slot DA from a neighbor node will do so based upon a perceived need for additional capacity on that link. This may be prompted by a high link utilization (queue buildup) based on short and long term measurements. The request will contain the number of slots requested and a metric, which indicates the priority to be attached to the request. The metric might indicate the queue length as a measure of the need for the time slot allocation.

0073The node receiving the request may also receive requests from other neighbor nodes, which may contend for allocation of the same time slot. In order to simplify the protocol, a node must complete processing one thread of an available time slot DA allocation before considering the next allocation. These allocations may not persist for a long period of time because they are constantly subject to preemption to become reallocated as semi-permanent time slots as a result of topology changes or subject to reallocation due to shifting traffic demand.

0074Neighbor and link discovery will now be discussed. The distributed link scheduling algorithm requires support from an omni-directional overhead channel for certain protocol exchanges that must occur with a potential neighbor node prior to the establishment of the directional link with that node. Such messages include the REQ_SPTS which requests the allocation of a semi-permanent time slot on the directional link to that node.

0075In addition to supporting protocol message exchanges which directly support the protocol defined herein, the omni-directional overhead channel must support the function of neighbor and link discovery. This is usually done through periodic omni transmissions by each node via an omni-directional antenna <b>20</b> that alerts any other node that move within range that the two nodes can be neighbor nodes. Several ad hoc routing protocols (including OLSR) have defined such a supporting protocol. These previously defined protocols could be adapted to support this distributed link scheduling algorithm. The primary function that must be performed by such a protocol is to discover new potential neighbor nodes and to report these to the topology control function.

0076One approach for node and link discovery includes each node periodically transmitting beacon messages over the control channel to notify neighbor nodes of its presence and its position. In addition, link state messages are transmitted periodically to notify neighbor nodes of the identity of its beacon neighbors (BN list) and its PA neighbor nodes (PAN list) and the time slots assigned to these nodes.

0077The link discovery portion of the algorithm continually compares the bi-directional beacon neighbors (BBN) list with the PAN list to see if there are any nodes on the BBN list that are not on the PAN list. Any such neighbor node becomes a candidate for link testing to determine if a PA link is possible. According to this approach, after an exchange of control messages the directional link is tested to determine if reliable communication is possible. If communication is reliable, the new neighbor node is added to the PAN list.

0078This validates communication in the testing time slot, but not necessarily in the time slot that may be assigned to the link on a semi-permanent basis. One approach is to do it this way or another approach is to wait until an SP time slot is assigned and test it in this time slot.

0079The topology control function can be a very straightforward function if it does not have to do topology optimization. The purpose of this function is to take the list of nodes in the PAN list, the information about the reliability of these links, and the information about the network topology, and use this information to determine which nodes on the PAN list should become PA neighbors. This is the function that should optimize the network topology if there are constraints such as the number of PA neighbors that do not allow all nodes in the PAN list to become PA neighbors.

0080With the proposed constraints of a fixed value for N_frame and a fixed value for N (the maximum number of semi-permanent time slots per node), the potential exists for having some concern about network topology utilization. This would certainly be the case if these values were selected to be very small numbers. For example, if N = 3 were selected with N_frame = 5, it may be difficult to expect a well connected network topology when we could have no more than 3 neighbors for any node, unless an intelligent topology control function carefully utilized the topology prior to adding new PA neighbor nodes. This may be particularly so for a large network.

0081Thus, the topology control function should create a neighbor priority (NP) list, which is the PAN list ordered in order of desirability as potential PA neighbors. This list will direct the priority order in which potential PA neighbors are scheduled time slots. However, our initial problem is that of a small network with perhaps 15 nodes. In this case, we could specify N to have a value in the range of 5 to 8 and still have low latency. There is very little likelihood that there will be any topology utilization issues since allowing for 5 to 8 neighbor nodes will allow almost all possible neighbors to be PA neighbors.

0082A second purpose of the topology control function is to generate the topology change event that causes the link scheduler process to change state and perform the reallocation process for the SP time slots.

0083A top-level scheduling algorithm structure will now be discussed. The scheduling process was formulated with the objective of minimizing the complexity of the process while taking advantage of the overall approach outlined above. A key to controlling this scheduling is maintaining an accurate data structure at each node reflecting the state of time slot schedules for future time slots assigned to the link with each neighbor node.

0084Two data structures are proposed: a slot assignment DB and a link message DB. The possible states of links in the data structure for a given time slot in the epoch are listed in TABLE 1. The table describes each possible state and gives the notation for that state. TABLE 2 shows an example slot assignment DA and the contents indicating the timeslots for N_frame =9 (N = 5), the state assignments for each state, and example assigned neighbor IDs for each time slot.

0085In this example, 4 neighbors have been assigned SP time slots so one additional neighbor may be connected with these constraints. There is one free time slot which may be allocated as a DB time slot or offered with the DB time slots to be allocated as an SP time slot if a new neighbor node is possible. The use of the link message DB will be discussed later in the detailed protocol explanation. The example also indicates the use of sub-slots, e.g., 2 sub-slots per slot.

0086This is a concept to be used with the DA allocations to allow finer granularity. The meaning in this case would be that an allocation of time slot k, sub-slot 1 would be an allocation to a link of time slot k on the odd numbered frames. Conversely, sub-slot 2 would indicate an allocation of the time slot on the even numbered frames. <tables id="tabl0001" num="0001"><table frame="all"><title>TABLE 1</title><tgroup cols="2"><colspec colnum="1" colname="col1" colwidth="147mm" /><colspec colnum="2" colname="col2" colwidth="19mm" /><thead><row><entry valign="top"><b>Time Slot State in DB</b></entry><entry valign="top"><b>Notation</b></entry></row></thead><tbody><row rowsep="0"><entry>• Free</entry><entry>Free</entry></row><row rowsep="0"><entry>• SP Allocated Time Slot</entry><entry>SP_Alloc</entry></row><row rowsep="0"><entry>• DA Allocated Time Slot (May Be Preempted by SP Allocation Process or by DA Reallocation)</entry><entry>DA_Alloc</entry></row><row rowsep="0"><entry>• SP Allocation Request Message Sent</entry><entry>SP_Req</entry></row><row rowsep="0"><entry>• SP Allocation Reply Message Sent</entry><entry>SP_Reply</entry></row><row><entry rowsep="0">• DA Allocation Request Message Sent (May Be Preempted by SP Allocation Process or by DA Reallocation)</entry><entry rowsep="0">DA_Req</entry></row><row><entry>• DA Allocation Reply Message Sent (May Be Preempted by SP Allocation Process or by DA Reallocation)</entry><entry>DA_Reply</entry></row></tbody></tgroup></table></tables><tables id="tabl0002" num="0002"><table frame="all"><title>TABLE 2</title><tgroup cols="4"><colspec colnum="1" colname="col1" colwidth="20mm" /><colspec colnum="2" colname="col2" colwidth="16mm" /><colspec colnum="3" colname="col3" colwidth="18mm" /><colspec colnum="4" colname="col4" colwidth="38mm" /><thead><row><entry valign="middle"><b>Time Slot</b></entry><entry valign="middle"><b>Subslot</b></entry><entry valign="middle"><b>State</b></entry><entry valign="middle"><b>Assigned Neighbor ID</b></entry></row></thead><tbody><row><entry align="center" valign="middle">1</entry><entry valign="middle">--</entry><entry valign="middle">Free</entry><entry valign="middle">--</entry></row><row><entry align="center" valign="middle">2</entry><entry valign="middle">--</entry><entry valign="middle">SP_Alloc</entry><entry valign="middle">3</entry></row><row><entry align="center" valign="middle">3</entry><entry valign="middle" /><entry valign="middle">SP_Req</entry><entry valign="middle">4</entry></row><row><entry align="center" valign="middle">4</entry><entry valign="middle">1</entry><entry valign="middle">DA_Alloc</entry><entry valign="middle">3</entry></row><row><entry align="center" valign="middle">4</entry><entry valign="middle">2</entry><entry valign="middle">DA_Alloc</entry><entry valign="middle">4</entry></row><row><entry align="center" valign="middle">5</entry><entry valign="middle">1</entry><entry valign="middle">DA_Alloc</entry><entry valign="middle">5</entry></row><row><entry align="center" valign="middle">5</entry><entry valign="middle">2</entry><entry valign="middle">DA_Alloc</entry><entry valign="middle">3</entry></row><row><entry align="center" valign="middle">6</entry><entry valign="middle">--</entry><entry valign="middle">SP_Alloc</entry><entry valign="middle">5</entry></row><row><entry align="center" valign="middle">7</entry><entry valign="middle">1,2</entry><entry valign="middle">DA_Alloc</entry><entry valign="middle">8</entry></row><row><entry align="center" valign="middle">8</entry><entry valign="middle">2</entry><entry valign="middle">DA_Alloc</entry><entry valign="middle">4</entry></row><row><entry align="center" valign="middle">9</entry><entry valign="middle">--</entry><entry valign="middle">SP_Alloc</entry><entry valign="middle">8</entry></row></tbody></tgroup></table></tables>

0087The top-level state diagram for the link scheduling protocol is shown in <figref idref="f0003">FIG. 5</figref>. The diagram shows two independent processes <b>30</b> and <b>32</b> that are responsible for maintaining and modifying the time slot allocation database. On the left side is the state diagram for the process for maintaining and assigning semi-permanent (SP) time slots, i.e., process <b>30.</b> This process has priority over the assignments made by the process <b>32</b> on the right, which has responsibility for assigning the available (DA) time slots. Within process path <b>31</b>, the time slots that can be seized are as follows: free, DA allocated, and in process of being DA allocated. Similarly, within process path <b>33</b>, the time slots that can be seized are as follows: free, DA allocated and also need to be reallocated.

0088This database must be controlled as a locked database such that for any given time slot assignment state, only one of the two scheduling processes may modify that state at a given point in time. Once one of the processes begins to modify the state of a particular time slot assignment, the state is locked and the other process may not modify it until it is released.

0089At any time each time slot in the DB is in one of seven states as indicated in TABLE 1. Available time slots are said to be in the free state, i.e., they are not assigned to a link to one of its neighbor nodes either because a scheduling conflict has prevented assignment or because the time slot has recently become free and has not yet been scheduled.

0090As indicated, a time slot in the free state, may be scheduled either as an SP time slot or a DA time slot. A time slot that has been allocated as SP assigned may be modified only by the process that maintains SP time slots. The time slot may be deallocated by this process if network topology changes or if a more desirable topology is possible. Until such a time slot is returned to the free state, the process for maintaining and assigning the DA time slots cannot modify its state.

0091In addition, any time slot with a DB state indicating that it is in the process of being SP assigned cannot be allocated by the DA assignment process. This includes states indicating that SP request and reply messages have been sent. However, if the state of a time slot is DA allocated, then it may be reallocated by the DA assignment process. This might be done if the loading on the network indicated that a reallocation of the DA time slot is needed.

0092In contrast, the process allocating SP time slots has priority. In addition to assigning free slots, it may seize and reassign all time slots that have been DA assigned or are in the process of being DA assigned. This is done to provide a straightforward process of ensuring at least a single SP time slot assigned to each neighbor node during a frame of N_frame time slots. SP allocated time slots are returned to the free state only if the link is lost or if the topology control function determines that a particular link should no longer be in the list of the top N links to be established with neighbor nodes.

0093<figref idref="f0003">FIG. 5</figref> illustrates how this process works at the top level. The SP slot assignment process has greater flexibility in allocating time slots. It can seize more time slots for allocation than the DA process, and it can seize time slots that either have been DA allocated or are in the process of being DA allocated. The SP process may receive various events for processing including topology change events from the topology control function and protocol messages.

0094Such events might include loss of link to a neighbor, discovery of a new neighbor, reception of an SP allocation request message from a neighbor node, and the discovery that a topology change should occur to either add a link to a neighbor, break a link, or do both. The topology change event notification will carry data that will describe the topology change that needs to occur.

0095If the event described a loss of a link, then the only action that must be taken is to change the appropriate time slot state in the slot assignment DB to "free." If a link is to be added the process is more complex. In this case, the SP slot assignment process initiates protocol message exchanges with the new neighbor node and modifies the slot assignment DB. This ultimately results in the agreement between the two nodes on a time slot assignment for the SP slot assigned to this link. Only a single SP time slot is to be assigned to each link with a neighbor to simplify the protocol. Additional details of this protocol are described below.

0096The process of assigning DA time slots follows a similar procedure. The DA slot assignment process must calculate the DA time slot needs and compare them with the allocated time slots to determine if a new time slot reallocation is needed. If a reassignment of DA slots is initiated, it will also lead to a series of protocol message exchanges with neighbor nodes to agree on the reassigned time slots. The DA slot assignment process may reassign only time slots that are in the free state or not SP assigned. More about the protocol details and the process for determining when DA time slot reassignment is needed will be discussed below.

0097Allocating semi-permanent time slots to directional links will now be discussed. In the description of the approach for allocating N semi-permanent time slots assume that N is fixed and intelligently chosen with respect to the network size and environment. Also assume that N_frame = 2N-1. N_frame could also be set at any value higher than this to provide additional on-demand time slots if that is deemed to be useful for the particular network and traffic environment.

0098Several important functions are provided by the topology control function. The neighbor priority (NP) list is generated by the topology control function and is used to indicate the preferred PA neighbor nodes for the assignment of time slots.

0099If the length of the NP list is N or smaller, then the topology control function will generate topology change events to the SP slot assignment process to make it attempt to get time slot assignments to all of these neighbor nodes. If the length of the NP list is greater than N, then it will generate topology change events to the SP slot assignment process to obtain time slot assignments to each of the N highest priority nodes on the NP list.

0100The NP list is constantly changing due to network dynamics. When PA links go down, the node is removed from the NP list and the time slot(s) for that link are then subject to reallocation. This is initiated by the topology control function which sends the SP slot assignment process a link delete event. Thus, the SP time slot and any DA time slots allocated to that link become available for reallocation to another node on the PA list.

0101The first choice when slots become available is to allocate the slot(s) to additional PA neighbor nodes if that is possible given the current state of the NP list. If no additional neighbor nodes can be added, then the slot(s) can be reallocated on a DA basis.

0102<figref idref="f0003">FIG. 6</figref> shows a state diagram of the SP slot assignment process. In order to manage the protocol message processing, a link scheduling message DB is created as shown in TABLE 3. This maintains the state needed from prior protocol exchanges to be used when the next SP message arrives for processing. The idle process does event management in that it checks received events prior to allowing a state change to one of the other states.

0103These operations include checking received messages to determine if they are consistent with the current state of the DB. If a message is inconsistent with the DB, it is discarded. Certain timeouts may indicate that DB state needs to be reset. This process performs this function. <tables id="tabl0003" num="0003"><table frame="all"><title>TABLE 3</title><tgroup cols="7"><colspec colnum="1" colname="col1" colwidth="16mm" /><colspec colnum="2" colname="col2" colwidth="20mm" /><colspec colnum="3" colname="col3" colwidth="18mm" /><colspec colnum="4" colname="col4" colwidth="25mm" /><colspec colnum="5" colname="col5" colwidth="34mm" /><colspec colnum="6" colname="col6" colwidth="30mm" /><colspec colnum="7" colname="col7" colwidth="22mm" /><thead><row><entry valign="top"><b>Nbr_ID</b></entry><entry valign="top"><b>Link State</b></entry><entry valign="top"><b>Time out</b></entry><entry valign="top"><b>Time Slot List</b></entry><entry valign="top"><b>Selected Time Slot</b></entry><entry valign="top"><b>Selected Subslot</b></entry><entry valign="top"><b>Num_ tries</b></entry></row></thead><tbody><row rowsep="0"><entry align="center">1</entry><entry>SP_Alloc</entry><entry>--</entry><entry>--</entry><entry>2</entry><entry>1</entry><entry>--</entry></row><row rowsep="0"><entry align="center">1</entry><entry>SP_Alloc</entry><entry>--</entry><entry>--</entry><entry>2</entry><entry>2</entry><entry>--</entry></row><row><entry rowsep="0" align="center">1</entry><entry rowsep="0">DA_Alloc</entry><entry rowsep="0">--</entry><entry rowsep="0">--</entry><entry rowsep="0">5</entry><entry rowsep="0">1</entry><entry rowsep="0">--</entry></row><row><entry align="center">2</entry><entry>SP_Alloc</entry><entry>--</entry><entry>--</entry><entry>4</entry><entry>1</entry><entry>--</entry></row><row rowsep="0"><entry align="center">2</entry><entry>SP_Alloc</entry><entry>--</entry><entry>--</entry><entry>4</entry><entry>2</entry><entry>--</entry></row><row rowsep="0"><entry align="center">2</entry><entry>DA_Alloc</entry><entry>--</entry><entry>--</entry><entry>5</entry><entry>2</entry><entry>--</entry></row><row rowsep="0"><entry align="center">3</entry><entry>SP_Req</entry><entry>T2</entry><entry>Ls</entry><entry>--</entry><entry /><entry>1</entry></row><row rowsep="0"><entry align="center">4</entry><entry>SP_Alloc</entry><entry>--</entry><entry>--</entry><entry>6</entry><entry>1</entry><entry>--</entry></row><row><entry align="center">4</entry><entry>SP_Alloc</entry><entry>--</entry><entry>--</entry><entry>6</entry><entry>2</entry><entry>-</entry></row></tbody></tgroup></table></tables>

0104There are four basic message types required in the SP time slot assignment protocol as listed below in Table 4. The use of these are self-explanatory and consistent with the prior discussion. <tables id="tabl0004" num="0004"><table frame="all"><title>TABLE 4</title><tgroup cols="2"><colspec colnum="1" colname="col1" colwidth="27mm" /><colspec colnum="2" colname="col2" colwidth="72mm" /><thead><row><entry valign="top"><b>Message Type</b></entry><entry valign="top"><b>Message Function</b></entry></row></thead><tbody><row rowsep="0"><entry>REQ_SPTS</entry><entry>Request New SP Slot Allocation</entry></row><row rowsep="0"><entry>REPLY_SPTS</entry><entry>Reply to Received REQ_SPTS</entry></row><row rowsep="0"><entry>CONFIRM</entry><entry>Response to Received REPLY_SPTS</entry></row><row><entry>DELETE_TS</entry><entry>Message Indicating Deleted Time Slot Allocation</entry></row></tbody></tgroup></table></tables>

0105An example of SP time slot assignment is shown in <figref idref="f0004">FIG. 7</figref>. Nodes 1 and 2 both have 3 neighbors with the SP time slots allocations shown for each link. Therefore, they can add an additional link between themselves. The link scheduling protocol will find an acceptable time slot for the SP allocation. The corresponding protocol message exchange is shown in TABLE 5.

0106Node 1 initiates the exchange by sending a REQ_SPTS(L=(4,5,6,7)) with a list of at least N candidate time slots. This list may include all free and DA time slots. Node 1 is using slots 1, 2 and 3 for SP allocations to its neighbors so its list L contains the other time slots 4, 5, 6 and 7. When the request message is sent, the appropriate changes are made to the time slot and link scheduling message data structures. Node 2 is using time slots 4, 5 and 6 as SP allocations for its links to its 3 neighbors so it selects time slot 7 as the only one that will work for the new link. It sends this choice in the reply message.

0107When a reply message is sent, the appropriate changes are also made to the time slot and link scheduling message data structures. Finally, when a confirm is sent or received, the state of the appropriate time slots are changed to "SP allocated to link (1,2)."

0108Note also that if nodes 1 and 2 had already selected 4 neighbor nodes, it would still be possible for them to find common time slots with which to establish a link between them if they used the same time slots with at least two of their neighbors. <tables id="tabl0005" num="0005"><table frame="all"><title>TABLE 5</title><tgroup cols="3" colsep="0"><colspec colnum="1" colname="col1" colwidth="108mm" /><colspec colnum="2" colname="col2" colwidth="14mm" /><colspec colnum="3" colname="col3" colwidth="44mm" colsep="1" /><thead><row><entry valign="top">Node 1</entry><entry valign="top" /><entry valign="top">Node 2</entry></row></thead><tbody><row><entry>Receives Link Add Event From Its Topology Control For A Link From Node 1 to Node 2</entry><entry /><entry /></row></tbody></tgroup><tgroup cols="3" colsep="0"><colspec colnum="1" colname="col1" colwidth="108mm" /><colspec colnum="2" colname="col2" colwidth="14mm" /><colspec colnum="3" colname="col3" colwidth="44mm" colsep="1" /><thead><row rowsep="0"><entry valign="top">Send</entry><entry valign="top" /><entry valign="top">Msg Lost</entry></row></thead><tbody><row><entry>REQ_SPTS(L=(4,5,6,7) )</entry><entry>→</entry><entry /></row></tbody></tgroup><tgroup cols="3" colsep="0"><colspec colnum="1" colname="col1" colwidth="108mm" /><colspec colnum="2" colname="col2" colwidth="14mm" /><colspec colnum="3" colname="col3" colwidth="44mm" colsep="1" /><thead><row rowsep="0"><entry valign="top">Timeout and retry</entry><entry valign="top" /><entry valign="top" /></row></thead><tbody><row rowsep="0"><entry>Resend</entry><entry>→</entry><entry>Rcvd</entry></row><row rowsep="0"><entry>REQ_SPTS (L= ( 4, 5, 6, 7 ))</entry><entry /><entry>REQ_SPTS (L=(4, 5, 6, 7) )</entry></row><row rowsep="0"><entry>Rcvd REPLY_SPTS(Slot 7)</entry><entry>←</entry><entry>Send REPLY_SPTS(Slot 7)</entry></row><row rowsep="0"><entry>Send CONFIRM(Slot 7)</entry><entry>→</entry><entry>Rcvd CONFIRM(Slot 7)</entry></row><row><entry>Slot 7 Allocated to Link (1,2)</entry><entry /><entry>Slot 7 Allocated to Link (1,2)</entry></row></tbody></tgroup></table></tables>

0109Some initial pseudocode describing the processes required in <figref idref="f0003">FIG. 6</figref> has been developed. There are various events that may occur which must be processed by the SP slot assignment process <b>34</b>. Event management is done in the idle process as shown in TABLE 6. Four categories of events are shown: received message, check timeouts, link addition notification from topology control, and link failure or link deletion.

0110Received messages are first checked versus the link scheduling message DB to insure that the message is consistent with the current state of the DB. For example, if we sent a request to a neighbor, the next message expected is a reply. To simplify this distributed protocol, only one thread of SP protocol message exchanges is allowed at a time. This is enforced in the procedure by checking the DB to see if other SP message exchanges are ongoing prior to initiating a link add transition or prior to processing a REQ_SPTS message.

0111If a link addition cannot be initiated because another SP protocol thread is currently in process, the link addition will be postponed by backing off and rescheduling for a later time when the other process is expected to be completed. Allowing multiple attempts is done to handle potential conflict between several nodes attempting to add links simultaneously. This is not meant to deal with the problem of an unreliable RF link. This latter issue should be addressed by using a link protocol on the overhead channel that uses ARQ and retransmission to recover lost/errored messages.

0112Thus, the distributed scheduling protocol can assume that messages will not get lost. This allows simplification of the protocol. When topology control selects a neighbor node from the NP list to connect to as a new neighbor, it issues a topology change (link addition) event which (after consistency checks in the idle process) causes a transition to the link add state in the SP slot assignment process. <tables id="tabl0006" num="0006"><img file="EP1559211B1_D0004.tif" /></tables><tables id="tabl0007" num="0007"><img file="EP1559211B1_D0005.tif" /></tables>

0113Pseudocode for the link add process is shown in TABLE 7. This starts a process which requires coordination of the SP time slot assignment and protocol message exchanges between only the two neighbor nodes. The node requesting the link sends a REQ_SPTS message to the candidate neighbor node with the list of acceptable time slots for the link.

0114The list of candidate time slots must contain at least N time slots including at least one semi-permanent time slot SP. The list can also include possibly all of the N-1 available DA time slots. The available or on-demand time slots may be currently temporarily allocated for on-demand traffic.

0115This list will be priority-ordered to indicate the time slot preference that causes the least perturbation in the current available time slot assignments. In other words, the notation being used is that a time slot is not an SP time slot unless already allocated to a communication link. Any of the 2N-1 time slots may be an SP time slot. Thus, the list of N time slots sent are all either free time slots or an available DA time slot. These may be N-1 SP time slots but they are already allocated and are not on the list.

0116The REQ_SPTS message can be sent up to MAX_TRIES times to allow for unreliable links and conflicts with other assignments potentially occurring simultaneously. The timeout in the link scheduling message DB triggers the retries if there is no REPLY_SPTS message from the neighbor node in response to the REQ_SPTS message. Once the REQ_SPTS message is sent the process returns to the idle state where other events can be processed. <tables id="tabl0008" num="0008"><img file="EP1559211B1_D0006.tif" /></tables><tables id="tabl0009" num="0009"><img file="EP1559211B1_D0007.tif" /></tables>

0117The neighbor receiving a REQ_SPTS message will have its SP slot assignment process transition to the process <i>REQ_SPTS</i> state. The procedure for processing this message is shown in TABLE 8. This procedure takes the offered list of time slots, Ls, and selects its preferred time slot, Ns.

0118If the number of links to neighbor nodes, Num_links, is less than the limit N, the procedure selects the time slot it prefers from this list. Then a REPLY_SPTS reply message with this selection is sent. If the link cannot be accepted or if there is another ongoing SP slot assignment in process, a negative REPLY_SPTS reply message is sent.

0119The selected time slot will be selected from one of its N available time slots or one of its free time slots. An available time slot is either a "free" time slot or an available DA time slot. There will be at least N of these if we can add another link. Each node always manages its time slots so that there are N time slots available to assign as semi-permanent time slots (one to each of N neighbor nodes if that many neighbor nodes are available). If it accepts the link, then it will have at most N-1 other neighbor nodes with one semi-permanent time slot allocated per node. The procedure also makes the appropriate modifications to the state in the link scheduling message DB and the slot assignment DB. <tables id="tabl0010" num="0010"><img file="EP1559211B1_D0008.tif" /></tables>

0120A received REPLY_SPTS message is processed as shown in TABLE 9. The choice of time slot, Ns, received from the neighbor node is extracted from the message. We will also require the node to confirm this reply with either a positive or negative CONFIRM message that indicates that it will agree to use the allocated time slot. This three-way handshake eliminates uncertainty in the outcome of the scheduling process.

0121If the REPLY_SPTS message is a positive reply, then the choice of time slot, Ns, is examined to see if it is still an allowable assignment for a new SP time slot for the new link. If it is allowable, then the appropriate modifications to the state in the slot assignment and link scheduling message databases are made. Then a positive CONFIRM message is returned.

0122If the received REPLY_SPTS message was negative, then the slot assignment and link scheduling message databases are reset for this Nbr_ID. Otherwise, if the choice of Ns is no longer allowable, then the link scheduling message database is reset for this Nbr_ID. Then a negative CONFIRM message is sent to the neighbor node rejecting the link. <tables id="tabl0011" num="0011"><img file="EP1559211B1_D0009.tif" /></tables>

0123Table 10 shows the procedure for processing CONFIRM messages. If the CONFIRM is positive, the link is considered to be added to the set of neighbors. The number of links for the node, Num_links, is incremented. The assigned time slot, Ns, is marked SP_Alloc in the slot assignment DB, and the link message state in the link scheduling message DB is reset for index Nbr_ID. If the message was a negative CONFIRM, then the slot assignment and link scheduling message databases are reset for this Nbr_ID. <tables id="tabl0012" num="0012"><img file="EP1559211B1_D0010.tif" /></tables>

0124An allocated time slot may need to be deallocated for one of several reasons. If during the course of normal operation a link goes down or becomes unreliable, then the topology control function gets involved to address the unreliable link problem. Ultimately, it may generate a topology change (e.g., link deletion) event directing the SP slot assignment process to delete all slots assigned to the link.

0125The steps involved in this procedure are shown in TABLE 11. The link is de-allocated by sending a DELETE_TS message from the node requesting the de-allocation of all the time slots which are shared with the other node. In addition, the appropriate entries in the link scheduling message DB and the slot assignment DB are reset. <tables id="tabl0013" num="0013"><img file="EP1559211B1_D0011.tif" /></tables>

0126Table 12 shows the procedure for processing a received DELETE_TS message. The list of deallocated time slots, Ls, is extracted from the message. Then the appropriate state in the slot assignment DB and in the link scheduling message DB is reset. <tables id="tabl0014" num="0014"><img file="EP1559211B1_D0012.tif" /></tables>

0127In summary, the objective for the function allocating the semi-permanent time slots is to connect to as many neighbor nodes as possible up to N. If N neighbor nodes are obtained, then each is allocated a single semi-permanent time slot. Once a new link is established by this protocol, both nodes will commence operation in the newly allocated SP time slot.

0128This operation will test the new link to determine if reliable communication can be maintained using the allocated time slot. This insures that there is no unusual interference that occurs in this particular time slot. If the link is tested as unreliable, then the topology control function will be notified so that the time slot can be deallocated and used for other purposes.

0129Allocation of available (on-demand) time slots will now be discussed. The available time slots are to be allocated in a manner that is responsive to the fluctuating demands of network traffic. Again, assume that N is fixed and intelligently chosen with respect to the network size and environment. Also assume that <i>N_frame</i> = 2<i>N</i>-1.

0130To allow fine granularity in the allocation of available capacity, time slots will be divided into m_s sub-time slots. Assume for the rest of the following discussion that m_s = 2. This will be accomplished by defining a sub-time slot to be a specific time slot allocation that repeats every m_s^th (or second) frame.

0131A request for available time slots from one node to a neighbor node is allowed only if at least one semi-permanent time slot is allocated for the link between these two nodes. After a link is allocated at least one semi-permanent time slot, then a node may request a periodic allocation of a single time slot every m_s^th (or second) frame. The messages used for scheduling the available time slots can be sent over the PA link for scheduling time slots several frames in advance of when they are needed since the link has an allocation of at least one semi-permanent time slot per frame.

0132A key requirement for efficient allocation of available time slots is the measurement of the traffic requirements on each link. Two measures will be needed. First, the measured average traffic sent over link (i, k) (in units of the number of time slots per frame) will be denoted by T_ikse. This measure will include all traffic sent over one or more semi-permanent time slots per frame as well as any available time slots.

0133In addition, we also need to maintain a current measure of the queue state, Q_ik, for link (i, k). Larger values of Q_ik indicate the need for an immediate allocation of one or more available time slots. Occasional bursts of demand may produce increases in Q_ik, which should then trigger a request for additional time slots of on-demand capacity until the queue size decreases.

0134The total number of time slots (quantized to 1/2 of a time slot with m_s = 2) allocated on link (i, k) will be denoted by <maths id="math0004"><math display="inline"><msubsup><mi>N</mi><mi mathvariant="italic">ik</mi><mi mathvariant="italic">tot</mi></msubsup><mn mathvariant="italic">.</mn></math><img file="EP1559211B1_D0013.tif" /></maths> The time slot demand is defined as follows: <maths id="math0005" num="(4)"><math display="block"><msubsup><mi mathvariant="italic">T</mi><mi mathvariant="italic">ik</mi><mi mathvariant="italic">dem</mi></msubsup><mo>=</mo><mi>f</mi><mfenced><msubsup><mi mathvariant="italic">T</mi><mi mathvariant="italic">ik</mi><mi mathvariant="italic">se</mi></msubsup><msub><mi mathvariant="italic">Q</mi><mi mathvariant="italic">ik</mi></msub></mfenced><mo>,</mo></math><img file="EP1559211B1_D0014.tif" /></maths> which is a function of the measured traffic plus the estimated additional capacity needed that is indicated by the queue size. Then the number of time slots needed on this link, <maths id="math0006"><math display="inline"><msubsup><mi>T</mi><mi mathvariant="italic">ik</mi><mi mathvariant="italic">need</mi></msubsup><mo>,</mo></math><img file="EP1559211B1_D0015.tif" /></maths> is as follows: <maths id="math0007" num="(5)"><math display="block"><msubsup><mi mathvariant="italic">T</mi><mi mathvariant="italic">ik</mi><mi mathvariant="italic">need</mi></msubsup><mo>=</mo><mi>max</mi><mfenced><msubsup><mi mathvariant="italic">T</mi><mi mathvariant="italic">ik</mi><mi mathvariant="italic">dem</mi></msubsup><msubsup><mi mathvariant="italic">T</mi><mi mathvariant="italic">ki</mi><mi mathvariant="italic">dem</mi></msubsup></mfenced></math><img file="EP1559211B1_D0016.tif" /></maths>

0135The metric assigned to this link is as follows: <maths id="math0008" num="(6)"><math display="block"><msubsup><mi mathvariant="italic">M</mi><mi mathvariant="italic">ik</mi><mi mathvariant="italic">DA</mi></msubsup><mo>=</mo><msubsup><mi mathvariant="italic">T</mi><mi mathvariant="italic">ik</mi><mi mathvariant="italic">need</mi></msubsup><mo>-</mo><msubsup><mi mathvariant="italic">N</mi><mi mathvariant="italic">ik</mi><mi mathvariant="italic">tot</mi></msubsup><mo>+</mo><mi>B</mi><mo>,</mo></math><img file="EP1559211B1_D0017.tif" /></maths> which is a measure of the estimated number of additional time slots that should be allocated to this link through the DA slot allocation mechanism. B is a bias term that might be nominally set at about 1/4 to 1/2 of a time slot to allocated enough excess capacity to each link to avoid significant queuing. While we are illustrating the approach using the metric defined in (4), a variety of other forms of metric could also be used as the basis for allocating the DA time slots.

0136<figref idref="f0004">FIG. 8</figref> shows a state diagram of the DA slot assignment process <b>36</b>. The state diagram and the protocol exchanges are similar to those of the SP slot assignment process. In order to simplify the protocol message processing, only a single thread of DA time slot allocation can be in process at any time. The idle process does event management in that it checks received events prior to allowing a state change to one of the other states.

0137These operations include the following. Check received messages to determine if they are consistent with the current state of the DB. If a message is inconsistent with the DB, it is discarded. Certain timeouts may indicate that DB state needs to be reset. This process performs this function. It also determines if the DA slot assignment is optimal given the traffic load needs of the node. It may cause a transition to the add DA slot state if it determines if a new DA time slot must be added to a particular link.

0138There are four basic message types required in the DA time slot assignment protocol as listed below in TABLE 13. These are very similar to those used in the SP slot allocation. The use of these is self-explanatory and consistent with the prior discussion of the SP slot allocation process. <tables id="tabl0015" num="0015"><table frame="all"><title>TABLE 13</title><tgroup cols="2"><colspec colnum="1" colname="col1" colwidth="27mm" /><colspec colnum="2" colname="col2" colwidth="125mm" /><thead><row><entry valign="top"><b>Message Type</b></entry><entry valign="top"><b>Message Function</b></entry></row></thead><tbody><row rowsep="0"><entry>REQ_DATS</entry><entry>Request New DA Slot Assignment</entry></row><row rowsep="0"><entry>REPLY_DATS</entry><entry>Reply to Received REQ_DATS</entry></row><row rowsep="0"><entry>CONFIRM</entry><entry>Response to Received REPLY_DATS</entry></row><row rowsep="0"><entry>DELETE_TS</entry><entry>Message Indicating Deleted Time Slot Allocation</entry></row><row><entry>LINK METRIC</entry><entry>Message Broadcast to Neighbor Nodes with Link Metric for Each Link to a Neighbor Node</entry></row></tbody></tgroup></table></tables>

0139An example of DA time slot assignment is shown in <figref idref="f0005">FIG. 9</figref>. Node 1 wants to add an additional DA time slot allocation for its link (1,2). The corresponding protocol message exchange is shown in TABLE 5. Node 1 initiates the exchange by sending a REQ_DATS (L=(4.2, 5,6)) indicating that it can support allocations of all of slots 5 and 6 and sub-slot 4.2. This list may include all free and DA time slots, the later of which are less needed.

0140When the request message is sent, the appropriate changes are made to the time slot and link scheduling message data structures. Node 2 is using time slots 1, 3 and 6 as SP allocations for its links to its 3 neighbors and sub-slots 2.1 and 3.2 as DA allocations. It can select either sub-slot 4.2 or both sub-slots of slot 5. It chooses and sends this choice in the reply message.

0141When a reply message is sent the appropriate changes are also made to the time slot and link scheduling message data structures. Finally, when a confirm is sent or received, the state of the appropriate time slots are changed to "sub-slot 4.2 DA allocated to link (1,2)." <tables id="tabl0016" num="0016"><table frame="all"><title>TABLE 14</title><tgroup cols="3" colsep="0"><colspec colnum="1" colname="col1" colwidth="101mm" /><colspec colnum="2" colname="col2" colwidth="14mm" /><colspec colnum="3" colname="col3" colwidth="51mm" colsep="1" /><thead><row><entry valign="top">Node 1</entry><entry valign="top" /><entry valign="top">Node 2</entry></row></thead><tbody><row><entry>Determines That The Link From Node 1 to Node 2 Requires An Additional DA Time Slot</entry><entry /><entry /></row></tbody></tgroup><tgroup cols="3" colsep="0"><colspec colnum="1" colname="col1" colwidth="101mm" /><colspec colnum="2" colname="col2" colwidth="14mm" /><colspec colnum="3" colname="col3" colwidth="51mm" colsep="1" /><thead><row rowsep="0"><entry valign="top">Send</entry><entry valign="top" /><entry valign="top">Msg Lost</entry></row></thead><tbody><row><entry>REQ_DATS(L=(4.2,5,6))</entry><entry>→</entry><entry /></row></tbody></tgroup><tgroup cols="3" colsep="0"><colspec colnum="1" colname="col1" colwidth="101mm" /><colspec colnum="2" colname="col2" colwidth="14mm" /><colspec colnum="3" colname="col3" colwidth="51mm" colsep="1" /><thead><row rowsep="0"><entry valign="top">Timeout and retry</entry><entry valign="top" /><entry valign="top" /></row></thead><tbody><row rowsep="0"><entry>Resend</entry><entry>→</entry><entry>Rcvd</entry></row><row rowsep="0"><entry>REQ_DATS(L=(4.2,5,6))</entry><entry /><entry>REQ_DATS(L=(4.2,5,6))</entry></row><row rowsep="0"><entry>Rcvd REPLY_DATS(Slot 4.2)</entry><entry>←</entry><entry>Send REPLY_DATS(Slot 4.2)</entry></row><row rowsep="0"><entry>Send CONFIRM(Slot 4.2)</entry><entry>→</entry><entry>Rcvd CONFIRM(Slot 4.2)</entry></row><row><entry>Slot 4.2 DA Allocated to Link (1,2)</entry><entry /><entry>Slot 4.2 DA Allocated to Link (1,2)</entry></row></tbody></tgroup></table></tables>

0142The following approach is used at each network node to allocate the (N-1) available time slots for directional links to neighbor nodes. Using these measures each node will continuously maintain the link metric, <maths id="math0009"><math display="inline"><msubsup><mi>M</mi><mi mathvariant="italic">ik</mi><mi mathvariant="italic">DA</mi></msubsup><mo>,</mo></math><img file="EP1559211B1_D0018.tif" /></maths> for each of its links allocated a semi- permanent time slot. Each node will use this link metric to indicate the need for additional transmission time slots to each neighbor node. The largest values of <maths id="math0010"><math display="inline"><msubsup><mi>M</mi><mi mathvariant="italic">ik</mi><mi mathvariant="italic">DA</mi></msubsup></math><img file="EP1559211B1_D0019.tif" /></maths> indicate the links with the greatest need for additional on-demand time slot allocation. A positive value of <maths id="math0011"><math display="inline"><msubsup><mi>M</mi><mi mathvariant="italic">ik</mi><mi mathvariant="italic">DA</mi></msubsup></math><img file="EP1559211B1_D0020.tif" /></maths> indicates the number of additional time slots required, and a negative value of indicates the number of time slots that can be surrendered for reallocation.

0143As the metrics, <maths id="math0012"><math display="inline"><msubsup><mi>M</mi><mi mathvariant="italic">ik</mi><mi mathvariant="italic">DA</mi></msubsup><mo>,</mo></math><img file="EP1559211B1_D0021.tif" /></maths> are maintained, if the largest link metric indicates a need for an additional sub-slot allocation and if there are sub-slots available either as free slots or as excess DA allocation to other links (again indicated by a small metric), then the process transitions to the add DA slot state and the process of finding a DA sub-slot allocation is initiated.

0144As with the semi-permanent time slots, the node need only coordinate the selection of the DA time slot to be assigned for a directional link to a neighbor with that neighbor. This means that a neighbor will send a request to the neighbor for the time slot assignment over the directional link, and receive either a grant of the assignment or a denial of the request over the same link.

0145Some initial pseudocode describing the processes required in <figref idref="f0004">FIG. 8</figref> has been developed. There are various events that may occur which must be processed by the DA slot assignment process. Event management is done in the idle process as shown in TABLE 6.

0146Four categories of events are shown: 1) received message, 2) check timeouts, 3) recalculation of link metrics, and 4) DA time slot needs and DA time slot deletion. Received messages are first checked versus the link scheduling message DB to insure that the message is consistent with the current state of the DB. For example, if we sent a request to a neighbor, the next message expected is a reply.

0147To simplify this distributed protocol, only one thread of DA protocol message exchanges is allowed at a time. This is enforced in the procedure by checking the DB to see if other DA message exchanges are ongoing prior to initiating an add DA slot transition or prior to processing a REQ_DATS message. If an addition slot cannot be initiated because another DA protocol thread is currently in process, the addition slot will not be done.

0148It can be naturally rescheduled on the next opportunity for recalculation of link metrics and DA time slot needs. Link metrics will be recalculated periodically according to a preset schedule. A link which has a link metric greater than a certain threshold, Max_metric_threshold, is a candidate for obtaining a new DA sub-lot.

0149The link with the maximum metric that exceeds this threshold will be selected as the next link to which a new DA sub-slot is allocated. When a new DA sub-slot needs to be allocated and if it satisfies the above conditions, then a transition to the add DA slot state occurs in the DA slot assignment process. <tables id="tabl0017" num="0017"><img file="EP1559211B1_D0022.tif" /></tables><tables id="tabl0018" num="0018"><img file="EP1559211B1_D0023.tif" /></tables>

0150Pseudocode for the additional DA slot process is shown in TABLE 16. This starts a process which requires coordination of the time slot assignment and protocol message exchanges between only the two neighbor nodes. The node requesting the link sends a REQ_DATS message to the candidate neighbor node with the list of acceptable time slots for the link.

0151The list of candidate time slots must contain all free sub-slots and all DA sub-slots with a metric below a certain threshold, Min_metric_threshold. The DA time slots may be currently temporarily allocated for other DA traffic. This list will be priority-ordered to indicate the sub-slot preference that causes the least perturbation in the current on-demand time slot assignments. The priority ordering will be first the free time slots followed by the sub-slots with the smallest metrics progressing up to the largest metric less than the Min_metric_threshold.

0152In order to simplify this distributed protocol, only one thread of DA protocol message exchanges is allowed at a time. This is enforced in the idle procedure. The REQ_DATS message is only sent once, but it could be unsuccessful if the neighbor node is currently processing another DA protocol exchange. In this case, the node will eventually receive a negative REPLY_DATS message. The attempt to add the DA slot may be made again in this case if this link has the largest metric the next time the link metrics are evaluated. Once the REQ_DATS message is sent the process returns to the idle state where other events can be processed. <tables id="tabl0019" num="0019"><table frame="all"><title>TABLE 16</title><tgroup cols="1" colsep="0"><colspec colnum="1" colname="col1" colwidth="155mm" colsep="1" /><thead><row><entry valign="top"><b>Procedure for Addition of a New DA Subslot to the Link to Node Nbr_ID (Generate REQ_DATS Message)</b></entry></row></thead><tbody><row rowsep="0"><entry>Construct list Ls of time slots (subslots) to offer to Nbr_ID from Free time slots and</entry></row><row rowsep="0"><entry>DA subslots with excess capacity (Link_metric &lt; Min_metric_threshold)</entry></row><row rowsep="0"><entry>Append list Ls to REQ_SPTS message and send to Nbr_ID</entry></row><row rowsep="0"><entry>Setup timeout and Link Message state in Link Scheduling Message DB for index</entry></row><row><entry>Nbr_ID and in Slot Assignment DB</entry></row></tbody></tgroup></table></tables>

0153The neighbor receiving a REQ_DATS message will have its DA slot assignment process transition to the REQ_SPTS state. The procedure for processing this message is shown in TABLE 17. This procedure takes the offered list of sub-slots, Ls, and selects its preferred sub-slot, Ns. The sub-slot accepted is the first sub-slot on the list, ls, that is either marked free in the slot assignment DB or is DA allocated with a link metric less than Min_metric_threshold. Then a REPLY_DATS reply message with this selection is sent. If the link cannot be accepted or if there is another ongoing DA slot assignment in process, a negative REPLY_DATS reply message is sent. The procedure also makes the appropriate modifications to the state in the link scheduling message DB and the slot assignment DB. <tables id="tabl0020" num="0020"><img file="EP1559211B1_D0024.tif" /></tables><tables id="tabl0021" num="0021"><img file="EP1559211B1_D0025.tif" /></tables>

0154A received REPLY_DATS message is processed as shown in TABLE 18. The choice of sub-slot, Ns, received from the neighbor node is extracted from the message. We require the node to confirm this reply with either a positive or negative CONFIRM message that indicates that it will agree to use the allocated time slot. As indicated in the SP allocation process, this three-way handshake eliminates uncertainty in the outcome of the scheduling process.

0155If the REPLY_DATS message is a positive reply, then the choice of sub-slot, Ns, is examined to see if it is still an allowable assignment for a new DA sub-slot for the new link. If it is allowable, then the appropriate modifications to the state in the slot assignment and link scheduling message databases are made. Then a positive CONFIRM message is returned.

0156If the received REPLY_SPTS message was negative, then the slot assignment and link scheduling message databases are reset for this Nbr_ID. Otherwise, if the choice of Ns is no longer allowable, then the link scheduling message database is reset for this Nbr_ID. Then a negative CONFIRM message is sent to the neighbor node rejecting the link. <tables id="tabl0022" num="0022"><img file="EP1559211B1_D0026.tif" /></tables><tables id="tabl0023" num="0023"><img file="EP1559211B1_D0027.tif" /></tables>

0157TABLE 19 shows the procedure for processing CONFIRM messages. If the CONFIRM is positive, the selected sub-slot to be added to the allocation to the link to Nbr_ID. The assigned time slot, Ns, is marked DA_Alloc in the slot assignment DB, and the link message state in the link scheduling message DB is reset for index Nbr_ID. If the message was a negative CONFIRM, then the slot assignment and link scheduling message databases are reset for this sub-slot. <tables id="tabl0024" num="0024"><img file="EP1559211B1_D0028.tif" /></tables>

0158An allocated time slot may need to be deallocated for one of several reasons. If during the course of normal operation a link goes down or becomes unreliable, then the topology control function gets involved to address the unreliable link problem. Ultimately, it may generate a topology change (e.g., a link deletion) event directing the SP slot assignment process to delete all slots assigned to the link.

0159The steps involved in this procedure are shown in TABLE 11. The link is de-allocated by sending a DELETE_TS message from the node requesting the de-allocation of all the time slots which are shared with the other node with. In addition, the appropriate entries in the link scheduling message DB and the slot assignment DB are reset. <tables id="tabl0025" num="0025"><table frame="all"><title>TABLE 20</title><tgroup cols="1" colsep="0"><colspec colnum="1" colname="col1" colwidth="129mm" colsep="1" /><thead><row><entry valign="top"><b>Procedure for DA TS Delete to Node Nbr_ID (Generate DELETE_TS Message)</b></entry></row></thead><tbody><row rowsep="0"><entry>Construct message, DELETE_TS, containing the DA subslot, Ns, that is to be deleted</entry></row><row rowsep="0"><entry>and send to Nbr_ID</entry></row><row rowsep="0"><entry>Reset Link Scheduling Message DB for index Nbr_ID and Slot Assignment DB for subslot Ns</entry></row><row><entry><b>Return</b> to Idle state</entry></row></tbody></tgroup></table></tables>

0160Table 21 shows the procedure for processing a received DELETE_TS message. The subslot, Ls, to be deallocated is extracted from the message. Then the appropriate state in the slot assignment DB and in the link scheduling message DB is reset. <tables id="tabl0026" num="0026"><table frame="all"><title>TABLE 21</title><tgroup cols="1" colsep="0"><colspec colnum="1" colname="col1" colwidth="165mm" colsep="1" /><thead><row><entry valign="top"><b>Procedure for Processing DELETE_TS Message from Nbr_ID</b></entry></row></thead><tbody><row rowsep="0"><entry>Extract DA subslot, Ns, from the DELETE_TS message from Nbr_ID Reset the Slot Assignment DB (mark it as Free) for subslot Ns Reset Link Message state in Link Scheduling Message DB for subslot Ns</entry></row><row><entry><b>Return</b> to Idle state</entry></row></tbody></tgroup></table></tables>

0161The link scheduling algorithm is also applicable to multiple simultaneous beams generated by the phased array antenna 16. Assume the extension to a system with nodes each employing multiple antenna beams with separate receivers such as a multiple beam phased array (or other types of multiple, directional antennas). Furthermore, assume that all nodes do not all have to have the same number of beams, i.e., node k has B_k beams. This is equivalent to B_k parallel links possible at any time slot.

0162We are extending the previous discussion (which assumed a single steered beam) to allow the B_k beams to be time-shared among a set of neighbor nodes larger than B_k. Even though the nodes may each have different numbers of beams, all nodes must use a common time slot format and frame with a number of time slots per frame for each beam equal to N_frame-Consider an upper limit at any node k on the number of semi-permanently (SP) assigned time slots on any one of its B_k beams (and therefore the maximum number of allowable neighbor nodes per beam) to be denoted by N_beam. The value of N_beam is dependent only on the number of time slots per frame and not the number of beams. As in (3) we will specify that N_beam must satisfy the following equation: <maths id="math0013" num="(7)"><math display="block"><msub><mi>N</mi><mi mathvariant="italic">frame</mi></msub><mo>≥</mo><mn>2</mn><mo>⋅</mo><msub><mi>N</mi><mi mathvariant="italic">beam</mi></msub><mo>-</mo><mn>1</mn></math><img file="EP1559211B1_D0029.tif" /></maths>

0163Assume that all nodes in a network are connected by directional links, where node k has B_k beams with beam sharing by time hopping and pointing to its neighbor nodes. Further, assume the number of neighbors allowed per beam is equal to N_beam, the fixed limit on the allowable number of semi-permanent time slots allowed per beam (with one SP time slot allocated per neighbor).

0164If the fixed value of N_beam for each beam at each neighbor node satisfies (7), then all nodes can select a different semi-permanent time slot for each of these links and each of its beams by mutual agreement with the neighbor for that link without regard to what colors other nodes are selecting more than one hop away. This allows each node to select its N_beam semi-permanent time slots for each beam in a very direct fashion by communicating only with its neighbor node. By following this strategy, each node is able to support at least <maths id="math0014" num="(8)"><math display="block"><msub><mi>N</mi><mi>k</mi></msub><mo>=</mo><msub><mi>B</mi><mi>k</mi></msub><mo>⋅</mo><msub><mi>N</mi><mi mathvariant="italic">beam</mi></msub></math><img file="EP1559211B1_D0030.tif" /></maths> neighbors and each allocated a single SP time slot with no more than N_beam such time slots allocated per beam.

0165Verification that N_beam neighbors per beam can be supported as long as (7) is satisfied follows directly from the verification of the observation for the single beam case. Then if all B_k beams have their SP time slots scheduled in the same fashion, it is obvious that the number of neighbor nodes that can be supported is the product of the number of beams and the number of neighbors per beam resulting in (8).

0166An example of SP time slot assignment between two nodes with an unequal number of beams per node is shown in <figref idref="f0005">FIG. 10</figref>. In this example node 1 has 2 beams and node 2 has 3 beams. While the two nodes have different numbers of beams, both nodes must use the same frame structure. In this example N_frame = 5 time slots per frame. From (7) and (8), this allows node 1 to have a maximum of 6 neighbors and node 2 to have a maximum of 9 neighbors.

0167Initially both nodes have one less than the maximum number of neighbors they are allowed under the constraints of (7) and (8). The SP beam/time slots allocations are shown for each link. These nodes can add an additional link between themselves while still satisfying the constraints of (7) and (8). The link scheduling protocol will find an acceptable beam/time slot for the SP allocation for each node, and it operates in essentially the same way it did with the single beam case.

0168The corresponding protocol message exchange is shown in TABLE 22. Node 1 initiates the exchange by sending a REQ_SPTS(L=( 1, 2, 3)) with a list of at least N_beam candidate time slots. Note the 3 beam IDs are denoted by a, b and c, and the slot number is denoted by the subscript on the beam ID. Node 1 had to identify that it had used all 3 allowable SP time slots on beam a, but it had allocated only 2 of the 3 allowable SP time slots on its beam b.

0169Thus, it sent a list of the 3 SP time slots (available on beam b) to node 2. This list may include all free and DA time slots on this beam. When the request message is sent, the appropriate changes are made to the time slot and link scheduling message data structures. Node 2 has previously allocated SP all available SP time slots on beams a and b for its links to its 8 neighbors.

0170Thus, beam c is the only beam that can accept a new SP allocation. When it receives the REQ_SPTS(L=( 1, 2, 3)) from node 1, it selects beam/time slot c₃ as the only one that will work for the new link (having previously allocated c₁ and c₂ as SP time slots). It sends this choice in the reply message. When a reply message is sent the appropriate changes are also made to the beam/time slot and link scheduling message data structures. Finally, when a confirm is sent or received, the state of the appropriate time slots are changed to "SP allocated to link (1,2)." <tables id="tabl0027" num="0027"><table frame="all"><title>TABLE 22</title><tgroup cols="3" colsep="0"><colspec colnum="1" colname="col1" colwidth="97mm" /><colspec colnum="2" colname="col2" colwidth="14mm" /><colspec colnum="3" colname="col3" colwidth="55mm" colsep="1" /><thead><row><entry valign="top">Node 1</entry><entry valign="top" /><entry valign="top">Node 2</entry></row></thead><tbody><row><entry>Receives Link Add Event From Its Topology Control For A Link From Node 1 to Node 2</entry><entry /><entry /></row><row rowsep="0"><entry>Send REQ_SPTS(L=(1, 2, 3))</entry><entry>→</entry><entry>Rcvd Send REQ_SPTS(L=(1, 2, 3))</entry></row><row rowsep="0"><entry>Rcvd REPLY_SPTS(Slot 3)</entry><entry>←</entry><entry>Send REPLY_SPTS(Slot 3)</entry></row><row><entry>Send CONFIRM(Slot 3)</entry><entry>→</entry><entry>Rcvd CONFIRM(Slot 3)</entry></row><row><entry>Beam / Slot b₃ Allocated to Link (1,2)</entry><entry /><entry>Beam / Slot c₃ Allocated to Link (1,2)</entry></row></tbody></tgroup></table></tables>

0171The changes that are required to implement the multiple beam scheduling algorithm/protocol are straightforward and are as follows. Add the beam ID as a variable in the state of the time slot DB and the link scheduling message DB. Use (7) and (8) as the criteria for determining if it is possible to schedule a new SP time slot. We specify a value for the parameters N_frame and N_beam for the network.

0172To offer a new SP time slot to a potential neighbor, the algorithm must first find a beam for which the number of neighbors is less than N_beam. This beam can then be used to add the new neighbor. The REQ_SPTS message that the node sends to its neighbor will specify N_beam available time slots for that beam that are not currently SP allocated.

0173Having received an REQ_SPTS message the node must find one of its beams for which the number of neighbors is less than N_beam. This beam can then be used to add the new neighbor. Comparing the list of N_beam time slots in the received REQ_SPTS message with the N_beam time slots not currently allocated in the selected beam, at least one time slot can be found that is common to both lists. That time slot can be selected as the time slot to send in the REPLY_SPTS message. Once the originating node receives the REPLY_SPTS message, both nodes will have selected their beam and the common time slot allocation.

0174This example implicitly assumed that a single frequency band is used for each of the beams. In this case, a node could have several beams simultaneously communicating over the same band without interference. This interference-free operation may be difficult to support in practice. A similar formulation of the problem could be done with each beam operating in a different frequency band, i.e., beams a, b, and c in <figref idref="f0005">Figure 10</figref> each use a different frequency band. In terms of the scheduling algorithm, we would apply the same constraints on the allocation of SP time slots. However, in actually allocating the time slot/beam combinations we would need to find an allocation such that the two nodes are using the same beam (equivalent to using the same band) as well as the same time slot. This equivalent to making each beam/time slot combination different from the scheduling perspective. Thus, the number of available time slots is the number of beams multiplied by the frame size. In this case the constraint on assigning SP time slots to potential neighbors is given by <maths id="math0015" num="(9)"><math display="block"><mi>B</mi><mo>⋅</mo><msub><mi>N</mi><mi mathvariant="italic">frame</mi></msub><mo>≥</mo><mn>2</mn><mo>⋅</mo><mi>N</mi><mo>-</mo><mn>1</mn><mo>,</mo></math><img file="EP1559211B1_D0031.tif" /></maths> where B denotes the number of beams. This constraint on the number of neighbors is slightly more restrictive than that of (7) and (8) because of the requirement that nodes which share an SP time slot must also use the same beam/frequency channel as well as the same time slot. For the example N_frame = 5 and B = 3, then the constraint of (9) allows 8 neighbors for each node whereas the constraints of (7) and (8) will allow 9 neighbors for each node.

0175The example problem in <figref idref="f0005">Figure 10</figref> has 2 nodes each with 3 beams with each beam operating in a different frequency band, i.e., beams a, b, and c each use a different frequency band. Assume also that the frame size is 5. Both nodes have already committed 7 SP time slots to neighbor nodes and thus, from (9), they can each add an additional neighbor with an SP time slot allowing them to establish a link between them. The committed SP time slots are indicated in the figure, and the message exchanges required to establish the SP time slot assignment and the new link are indicated in Table 23. The message exchange is initiated by node 1 by sending a REQ_SPTS (L=(a₄, a₅, b₃, b₄, b₅, c₃, c₄, c₅)) message to node 2 which must include the 8 beam/time slot combinations it has not previously allocated as SP time slots. In this example, node 2 had already allocated 7 beam/time slot combinations that were not used by node 1 (which were in the list of 8 beam/time slot combinations received in the REQ_SPTS message). Thus, by (9) there must be at least one remaining beam/time slot combination that it can select for allocation (c₅). This is the SP beam/time slot combination allocated to the link between nodes 1 and 2 as show in both <figref idref="f0005">Figure 11</figref> and Table 23. <tables id="tabl0028" num="0028"><table frame="all"><title>TABLE 23</title><tgroup cols="3" colsep="0"><colspec colnum="1" colname="col1" colwidth="76mm" /><colspec colnum="2" colname="col2" colwidth="14mm" /><colspec colnum="3" colname="col3" colwidth="76mm" colsep="1" /><thead><row><entry valign="top">Node 1</entry><entry valign="top" /><entry valign="top">Node 2</entry></row></thead><tbody><row><entry>Receives Link Add Event From Its Topology Control For A Link From Node 1 to Node 2</entry><entry /><entry /></row><row rowsep="0"><entry>Send REQ_SPTS(L=(a₄, a₅, b₃, b₄, b₅, C₃, C₄, c₅))</entry><entry>→</entry><entry>Rcvd Send REQ_SPTS(L=(a₄, a₅, b₃, b₄, b₅, C₃, C₄, C₅))</entry></row><row rowsep="0"><entry>Rcvd REPLY_SPTS (Beam/Slot c₅)</entry><entry>←</entry><entry>Send REPLY_SPTS (Beam/Slot c₅)</entry></row><row rowsep="0"><entry>Send CONFIRM (Beam/Slot c₅)</entry><entry>→</entry><entry>Rcvd CONFIRM (Beam/Slot c₅)</entry></row><row><entry>Beam / Slot c₅ Allocated to Link (1,2)</entry><entry /><entry>Beam / Slot c₅ Allocated to Link (1,2)</entry></row></tbody></tgroup></table></tables>

0176The present invention thus provides a fully distributed link scheduling algorithm and protocol for phased array networks. The description of the algorithm/protocol details assumed the case of a single directional beam per node, which is time-shared and pointed toward neighbor nodes during the allocated time slot for that access. However, the approach can be used for an arbitrary number of steered beams per node.