Method and apparatus of robust neural temporal coding, learning and cell recruitments for memory using oscillation

BACKGROUND

1. Field

Certain aspects of the present disclosure generally relate to neural system engineering and, more particularly, to a method and apparatus of robust neural temporal coding, learning and cell recruitments for memory using oscillations.

2. Background

Neurons in a neural system can communicate information temporally using temporal codes in the form of timed spikes. Because of this, methods of coding and decoding and methods of learning such temporal information are of primary interest.

In particular, it is desired to distinguish temporal patterns and, in contrast to other temporal pattern methods, not merely coincidence of inputs or order of inputs. The present disclosure provides methods that are biologically-inspired/consistent but reduced in complexity and capable of coding, decoding, recognizing, and learning temporal spike signal patterns.

SUMMARY

Certain aspects of the present disclosure provide a method of merging a network of spiking neuron circuits with a rule for learning synaptic weights associated with the neuron circuits. The method generally includes providing synaptic inputs into a neuron circuit of the network, wherein each of the synaptic inputs is associated with a synaptic weight of the synaptic weights and a time delay, latching each of the synaptic inputs being weighted and delayed, upon a rise in an input of the neuron circuit comprising the synaptic inputs, and upon the input or upon the neuron circuit spiking based on the rise in the input, applying the learning rule on the latched synaptic inputs to determine a change in the synaptic weight associated with that synaptic input.

Certain aspects of the present disclosure provide an electrical circuit for merging a network of spiking neuron circuits with a rule for learning synaptic weights associated with the neuron circuits. The electrical circuit generally includes a first circuit configured to provide synaptic inputs into a neuron circuit of the network, wherein each of the synaptic inputs is associated with a synaptic weight and a time delay, a second circuit configured to latch each of the synaptic inputs being weighted and delayed, upon a rise in an input of the neuron circuit comprising the synaptic inputs, and a third circuit configured to apply, upon the input or upon the neuron circuit spiking based on the rise in the input, the learning rule on the latched synaptic inputs to determine a change in the synaptic weight associated with that synaptic input.

Certain aspects of the present disclosure provide an apparatus for merging a network of spiking neuron circuits with a rule for learning synaptic weights associated with the neuron circuits. The apparatus generally includes means for providing synaptic inputs into a neuron circuit of the network, wherein each of the synaptic inputs is associated with a synaptic weight and a time delay, means for latching each of the synaptic inputs being weighted and delayed, upon a rise in an input of the neuron circuit comprising the synaptic inputs, and means for applying, upon the input or upon the neuron circuit spiking based on the rise in the input, the learning rule on the latched synaptic inputs to determine a change in the synaptic weight associated with that synaptic input.

Certain aspects of the present disclosure provide a method of regulating a firing rate of a neuron circuit of a neural network. The method generally includes computing periodically the firing rate of the neuron circuit by counting a number of firings of the neuron circuit within a time period, determining whether the firing rate is below a lower bound or above an upper bound, and adjusting the firing rate by a step amount based on the determination.

Certain aspects of the present disclosure provide an electrical circuit for regulating a firing rate of a neuron circuit of a neural network. The electrical circuit generally includes a first circuit configured to compute periodically the firing rate of the neuron circuit by counting a number of firings of the neuron circuit within a time period, a second circuit configured to determine whether the firing rate is below a lower bound or above an upper bound, and a third circuit configured to adjust the firing rate by a step amount based on the determination.

Certain aspects of the present disclosure provide an apparatus regulating a firing rate of a neuron circuit of a neural network. The apparatus generally includes means for computing periodically the firing rate of the neuron circuit by counting a number of firings of the neuron circuit within a time period, means for determining whether the firing rate is below a lower bound or above an upper bound, and means for adjusting the firing rate by a step amount based on the determination.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above-recited features of the present disclosure can be understood in detail, a more particular description, briefly summarized above, may be had by reference to aspects, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only certain typical aspects of this disclosure and are therefore not to be considered limiting of its scope, for the description may admit to other equally effective aspects.

<figref idref="DRAWINGS">FIG. 1</figref> illustrates an example network of neurons in accordance with certain aspects of the present disclosure.

<figref idref="DRAWINGS">FIG. 2</figref> illustrates an example relative delay neuron model in accordance with certain aspects of the present disclosure.

<figref idref="DRAWINGS">FIG. 3</figref> illustrates an example realignment of relative delay inputs by dendritic model in accordance with certain aspects of the present disclosure.

<figref idref="DRAWINGS">FIG. 4</figref> illustrates an example temporal shift of learning curves in accordance with certain aspects of the present disclosure.

<figref idref="DRAWINGS">FIG. 5</figref> illustrates an example temporal shift and sensitivity damping of learning curves in accordance with certain aspects of the present disclosure.

<figref idref="DRAWINGS">FIG. 6</figref> illustrates example learning curves resulting from dynamic spiking Hebbian rule method in accordance with certain aspects of the present disclosure.

<figref idref="DRAWINGS">FIG. 7</figref> illustrates an example of robustness aspects of the relative delay neuron model in accordance with certain aspects of the present disclosure.

<figref idref="DRAWINGS">FIG. 8</figref> illustrates an example single oscillation as a basic catalyst for integrity coding in accordance with certain aspects of the present disclosure.

<figref idref="DRAWINGS">FIG. 9</figref> illustrates an example oscillation reference and integrity temporal coding in accordance with certain aspects of the present disclosure.

<figref idref="DRAWINGS">FIG. 10</figref> illustrates an example oscillation reference and integrity rate coding in accordance with certain aspects of the present disclosure.

<figref idref="DRAWINGS">FIG. 11</figref> illustrates an example of connectivity for recruitment of a particular cell for memory in accordance with certain aspects of the present disclosure.

<figref idref="DRAWINGS">FIG. 12</figref> illustrates an example temporal view of recruitment of a particular cell for memory in accordance with certain aspects of the present disclosure.

<figref idref="DRAWINGS">FIG. 13</figref> illustrates an example of using oscillation to separate a temporal pattern into highly resolvable sub-patterns in accordance with certain aspects of the present disclosure.

<figref idref="DRAWINGS">FIG. 14</figref> illustrates an example of using oscillations to associate patterns in accordance with certain aspects of the present disclosure.

<figref idref="DRAWINGS">FIG. 15</figref> illustrates example operations that may be performed at a network of neuron circuits in accordance with certain aspects of the present disclosure.

<figref idref="DRAWINGS">FIG. 15A</figref> illustrates example components capable of performing the operations illustrated in <figref idref="DRAWINGS">FIG. 15</figref>.

<figref idref="DRAWINGS">FIG. 16</figref> illustrates other example operations that may be performed at a neuron circuit of a neural network in accordance with certain aspects of the present disclosure.

<figref idref="DRAWINGS">FIG. 16A</figref> illustrates example components capable of performing the operations illustrated in <figref idref="DRAWINGS">FIG. 16</figref>.

DETAILED DESCRIPTION

Various aspects of the disclosure are described more fully hereinafter with reference to the accompanying drawings. This disclosure may, however, be embodied in many different forms and should not be construed as limited to any specific structure or function presented throughout this disclosure. Rather, these aspects are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. Based on the teachings herein one skilled in the art should appreciate that the scope of the disclosure is intended to cover any aspect of the disclosure disclosed herein, whether implemented independently of or combined with any other aspect of the disclosure. For example, an apparatus may be implemented or a method may be practiced using any number of the aspects set forth herein. In addition, the scope of the disclosure is intended to cover such an apparatus or method which is practiced using other structure, functionality, or structure and functionality in addition to or other than the various aspects of the disclosure set forth herein. It should be understood that any aspect of the disclosure disclosed herein may be embodied by one or more elements of a claim.

The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects.

Although particular aspects are described herein, many variations and permutations of these aspects fall within the scope of the disclosure. Although some benefits and advantages of the preferred aspects are mentioned, the scope of the disclosure is not intended to be limited to particular benefits, uses or objectives. Rather, aspects of the disclosure are intended to be broadly applicable to different technologies, system configurations, networks and protocols, some of which are illustrated by way of example in the figures and in the following description of the preferred aspects. The detailed description and drawings are merely illustrative of the disclosure rather than limiting, the scope of the disclosure being defined by the appended claims and equivalents thereof.

An Example Neural System

<figref idref="DRAWINGS">FIG. 1</figref> illustrates an example neural system <b>100</b> with multiple levels of neurons in accordance with certain aspects of the present disclosure. The neural system <b>100</b> may comprise a level of neurons <b>102</b> connected to another level of neurons <b>106</b> though a network of synaptic connections <b>104</b>. For simplicity, only two levels of neurons are illustrated in <figref idref="DRAWINGS">FIG. 1</figref>, although fewer or more levels of neurons may exist in a typical neural system.

As illustrated in <figref idref="DRAWINGS">FIG. 1</figref>, each neuron in the level <b>102</b> may receive an input signal <b>108</b> that may be generated by a plurality of neurons of a previous level (not shown in <figref idref="DRAWINGS">FIG. 1</figref>). The signal <b>108</b> may represent an input current of the level <b>102</b> neuron. This current may be accumulated on the neuron membrane to charge a membrane potential. When the membrane potential reaches its threshold value, the neuron may fire and generate an output spike to be transferred to the next level of neurons (e.g., the level <b>106</b>).

The transfer of spikes from one level of neurons to another may be achieved through the network of synaptic connections (or simply “synapses”) <b>104</b>, as illustrated in <figref idref="DRAWINGS">FIG. 1</figref>. The synapses <b>104</b> may receive output signals (i.e., spikes) from the level <b>102</b> neurons, scale those signals according to adjustable synaptic weights w₁^(i,i+1), . . . , w_P^(i,i+1)(where P is a total number of synaptic connections between the neurons of levels <b>102</b> and <b>106</b>), and combine the scaled signals as an input signal of each neuron in the level <b>106</b>. Every neuron in the level <b>106</b> may generate output spikes <b>110</b> based on the corresponding combined input signal. The output spikes <b>110</b> may be then transferred to another level of neurons using another network of synaptic connections (not shown in <figref idref="DRAWINGS">FIG. 1</figref>).

The neural system <b>100</b> may be emulated by an electrical circuit and utilized in a large range of applications, such as image and pattern recognition, machine learning, motor control, and alike. Each neuron in the neural system <b>100</b> may be implemented as a neuron circuit. The neuron membrane charged to the threshold value initiating the output spike may be implemented, for example, as a capacitor that integrates an electrical current flowing through it.

In an aspect, the capacitor may be eliminated as the electrical current integrating device of the neuron circuit, and a smaller memristor element may be used in its place. This approach may be applied in neuron circuits, as well as in various other applications where bulky capacitors are utilized as electrical current integrators. In addition, each of the synapses <b>104</b> may be implemented based on a memristor element, wherein synaptic weight changes may relate to changes of the memristor resistance. With nanometer feature-sized memristors, the area of neuron circuit and synapses may be substantially reduced, which may make implementation of a very large-scale neural system hardware implementation practical.

Since neuron circuits of the neural system <b>100</b> may communicate information temporally using so-called temporal codes in the form of timed spikes, the coding and decoding methods as well as methods of learning such temporal information are of primary interest. Certain aspects of the present disclosure support methods for distinguishing temporal patterns and, in contrast to other “temporal pattern” methods, not merely coincidence of inputs or order of inputs. Moreover, the present disclosure propose practical methods that are biologically-inspired/consistent but reduced in complexity and capable of coding, decoding, recognizing, and learning temporal spike signal patterns. In this disclosure, extensions are proposed to a scalable temporal neural model for robustness, confidence or integrity coding, and recruitment of cells for efficient temporal pattern memory. The proposed approach is biologically inspired by experimental evidence of oscillations or rhythms and synchrony, and at the same time, motivated to reduce modeling complexity.

Relative Delay and Dendritic Delay Lines

A method is proposed in the present disclosure in which a neuron's behavior may be determined based on a relative delay between inputs at synapses, a relative delay between inputs at synapses and a reference input, or both. The basic aspect of the temporal coding model is illustrated in <figref idref="DRAWINGS">FIG. 2</figref>. The spatial-temporal pattern of spikes output from neurons <b>202</b>, <b>204</b>, <b>206</b>, <b>208</b> (i.e., input neurons) may constitute spike inputs for a neuron <b>210</b>. Each input neuron may connect to the neuron <b>210</b> via one or more synapses on one or more dendrite of the neuron <b>210</b>. Each synapse may have an associated delay that characterizes the time delay that a spike from the input neuron undergoes before reaching the soma of neuron <b>210</b>, as illustrated in <figref idref="DRAWINGS">FIG. 2</figref> for synapses <b>212</b> connecting the neuron <b>204</b> with the neuron <b>210</b>. As illustrated in <figref idref="DRAWINGS">FIG. 2</figref>, the inputs may undergo scaling before delay and integration. Alternatively, the inputs may undergo delay before scaling for reduced processing in large-scale networks.

Using this method, a neuron may recognize spatial-temporal patterns in outputs of input neurons (spatial in the sense of input neurons, temporal in the sense of relative spike timing or temporal differences between spikes). This is illustrated in <figref idref="DRAWINGS">FIG. 3</figref> where input neurons <b>302</b>, <b>304</b>, <b>306</b>, <b>308</b> may spike at different times. However, as a result of delays in the dendrite(s), signals from the input neurons may reach a soma of output neuron x aligned in time. An input to the output neuron x at time t may thus be expressed as a function of delayed outputs of the input neurons, i.e.:

<maths id="MATH-US-00001" num="00001"><math overflow="scroll"><mtable><mtr><mtd><mrow><mrow><mrow><msub><mi>u</mi><mi>x</mi></msub><mo>⁡</mo><mrow><mo>(</mo><mi>t</mi><mo>)</mo></mrow></mrow><mo>=</mo><mrow><munder><mo>∑</mo><mi>j</mi></munder><mo>⁢</mo><mrow><msub><mi>w</mi><mi>j</mi></msub><mo>⁢</mo><mrow><msub><mi>v</mi><msub><mi>i</mi><mi>j</mi></msub></msub><mo>⁡</mo><mrow><mo>(</mo><mrow><mi>t</mi><mo>-</mo><mrow><msub><mi>n</mi><mi>j</mi></msub><mo>⁢</mo><mi>Δ</mi><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><mi>t</mi></mrow></mrow><mo>)</mo></mrow></mrow></mrow></mrow></mrow><mo>,</mo></mrow></mtd><mtd><mrow><mo>(</mo><mn>1</mn><mo>)</mo></mrow></mtd></mtr></mtable></math></maths><img file="US9053428B2_D0001.tif" /><br /> where j is the synapse index, i_jis the input neuron to which synapse j connects, n_jis the synaptic delay in units of Δt, v_i(t) is the output of input neuron i, and w_jis a synaptic weight. In equation (1), the synaptic delay represents a delay by which a total delay is abstracted. That total delay may be due to one or more (combination of) actual synaptic delays due to a digital-to-analog delay (i.e., a time for conversion from action potential (AP) to post-synaptic potential (PSP)), a dentritic delay (i.e., a passive travel time for the PSP to reach a soma), or other delays (e.g., axonal delays or network architecture delays due to paths through different layers or number of neurons).

Actual timings of firing of the neurons <b>302</b>, <b>304</b>, <b>306</b>, <b>308</b> are labeled as <b>310</b> in <figref idref="DRAWINGS">FIG. 3</figref>. Because of particular delays corresponding to synapses (i.e., denoted by multiples of time delay resolution Δt), the input timings after delays may or may not align once the relative delays are accounted for (labeled as <b>312</b> in <figref idref="DRAWINGS">FIG. 3</figref>). It can be observed from <figref idref="DRAWINGS">FIG. 3</figref> that some synapses are associated with delays that are too long (e.g., a synapse with delay <b>314</b>) or short (e.g., a synapse with delay <b>316</b>) to coincide with delays of other synapses. In an aspect, these short and long delays may be discarded from or not added to a recognized pattern, while delays that result in coincident delayed spikes may be retained or added to correspond to a recognized pattern.

In the preferred aspect of the present disclosure, discrete time delays of integer milliseconds may be utilized (i.e., delays in multiples of time delay resolution Δt=1 ms). However, in general, any discrete or continuous resolution may be used. In the discrete model, the delay may be represented by the integer n_xi, where x is the input neuron (e.g., the neuron <b>302</b> in <figref idref="DRAWINGS">FIG. 3</figref>), and i is the synapse index for that input neuron since there may be one or more synapses to each input.

In the following, it will be shown how to learn spatial temporal patterns in a robust manner.

Spike-Based Hebbian Learning Method

A robust reduced method for temporal coding and learning is first proposed in the present disclosure. Then, this method may be utilized as a basis for a method of further robustness using oscillations and rhythms. In an aspect of the present disclosure, these two methods can be used together for cell recruitment for memory.

Hebbian learning rules typically operate based on rate-coding or other windowed neuron models, adjusting synapse weights based on firing output over a trial time window. However, spike-based models may be used to reproduce precise timing behavior of neurons, which may fire with particular delays not necessarily coincident with particular inputs responsible for causing the firing. Methods used in the reduced model allow reconciling these differences.

In general, Hebbian learning refers to learning that associates (wires together) an input with an output when they occur simultaneously. However, a variety of such rules and related variations can be considered relevant for the methods being proposed in the present disclosure because of particular temporal aspects. With rate-coding, one might consider two neurons that are generally firing during a time window for association according to Hebbian principles. However, in the proposed methodology, the relative timing of individual spikes may be considered. Moreover, because a neuron may fire with some delay after inputs (causality), “simultaneity” may not be necessarily the optimal associative condition when considering behavior at the individual spike level. This may be important for the reasons discussed below.

Learning synaptic weights can be typically referred to as synaptic plasticity. For the Spike-Timing-Dependent Plasticity (STDP), synaptic weight adjustments in learning can be typically described in terms of a time difference between when a pre-synaptic input spike occurs and when a post-synaptic neuron fires, referenced by ΔT. Here, the convention can be used that a positive value means that the pre-synaptic input fired after the post-synaptic neuron. The learning can be expressed as a curve describing the amount and direction of weight adjustment across a range of time difference values. It should be noted that a standard STDP curve has a discontinuity at the origin.

However, an important aspect of learning curves may be shift. Examples of shifted curves are illustrated in <figref idref="DRAWINGS">FIG. 4</figref> and <figref idref="DRAWINGS">FIG. 5</figref>. <figref idref="DRAWINGS">FIG. 4</figref> illustrates an example <b>400</b> of temporal shift of learning curves in accordance with certain aspects of the present disclosure. <figref idref="DRAWINGS">FIG. 5</figref> illustrates an example <b>500</b> of temporal shift and sensitivity damping of learning curves in accordance with certain aspects of the present disclosure. It should be noted that the shift may be combined with other aspects, such as reinforcement or de-emphasis variations or other shape aspects.

Such shifting of the weight learning curve can be motivated for a variety of reasons. First, it may be desired to accommodate neuron depolarization delays (time constant of the soma, for example). In other words, if firing is delayed as in biologically motivated dynamic models of neurons, it may not be necessary to reinforce synaptic weights for extraneous inputs that may happen to arrive after the sufficiently depolarizing inputs but before the neuron fires. This may be prevented by shifting the curve toward the left, as illustrated in <figref idref="DRAWINGS">FIG. 5</figref>, which may hold the neuron from sliding forward toward the extraneous input times. Second, it may be desired to control (limit) a sliding effect that may occur if a neuron learns a part of a pattern and starts precession toward an earlier part of the pattern because of a nominal learning curve shape (i.e., reinforcing synapses with earlier and earlier input arrivals thus causing firing to occur earlier and earlier). This may be prevented by shifting the curve to the right reinforcing a narrow window of non-causal inputs, which may hold the neuron from sliding backward.

Typical Hebbian plasticity rules applied in the field of theoretical neuroscience, such as the Oja rule or the BCM rule (Bienenstock-Cooper-Munro rule) and their variations have weight regulation effects, which may stabilize the learning resulting from these rules. For example, the Oja's rule may provide weight change (as a vector) given by: <br />Δ<i>w=v</i>·(<i>u−αvw</i>)·τ,  (2)<br /> where v is a neuron's output and u is a neuron's input, τ is a time constant controlling weight adaptation (learning) rate and α is a parameter that controls normalization. It can be noticed that at the steady state u=αvw. Therefore, for coinciding input and output, the weights may be normalized to the value of 1/α. This may have an effect of regulation or so-called homeostasis. It is proposed in the present disclosure that neuronal regulation or homeostasis (e.g., maintaining a long-term firing rate constant) is an important component. Thus, when using STDP or curves such as those illustrated in <figref idref="DRAWINGS">FIGS. 4-5</figref> (as opposed to the Oja or BCM rules), the addition of neuronal regulation or homeostasis may be important.

Regarding homeostasis, it may be recommended that rather than targeting a particular firing rate, neurons may be allowed to operate in a firing rate range. Thus, it is proposed in the present disclosure that thresholds (or inputs) are scaled only if the rate falls beyond an upper or a lower range. This may provide stability as well as flexibility to different pattern densities. Thus, it is proposed that such adjustment is slow, i.e., occurring in the order of at least multiple pattern exposures and applied in steps.

However, it can be observed that the STDP-like effects (curves) may be reproduced without actually applying such learning curves directly (i.e., a reduced complexity model). Rather, using such Hebbian rules as the Oja rule, when combined with a dynamic spiking neuronal model such as the Izhikevich's simple model, it may be possible to observe the temporal learning curve effects discussed above (whether similar to the experimentally observed STDP curves or the variations discussed).

An example of learning curves resulting from dynamic spiking Hebbian rule method is illustrated in <figref idref="DRAWINGS">FIG. 6</figref>. It should be noted that while there are similarities with the biologically observed (idealized) STDP curves, there may exist differences not inconsistent with raw biological observed data where the actual data points are somewhat scattered near zero. This may also represent motivation to consider such different curves as described above. In an aspect, weights may be initially set high to encourage excitability and accelerate learning and the distribution of delay response. Observing a mean curve <b>602</b> and a mean of positive changes curve <b>604</b> illustrated in <figref idref="DRAWINGS">FIG. 6</figref> may provide insight that the optimal temporal plasticity curve may not be exactly as typically rendered. It can be observed a relatively flat tail of curve <b>604</b> as limited negative impact for non-causal (leftward) delays.

However, obtaining these effects without applying the learning curve directly may require a critical component, namely that inputs are latched. The latching may be critical because such learning rules as the Oja or BCM rules may typically assume inputs and outputs in terms of firing rate, whereas a dynamic spiking model may spike after some delay from the inputs. One way to accomplish this may be to latch inputs when the total input increases and maintain the latch until firing. Then, the latch contents and firing may be utilized according to the learning rule.

Aspects of the above can be seen in the following reduced model of temporal plasticity. The diagram illustrated in <figref idref="DRAWINGS">FIG. 6</figref> overlays a scatter plot of weight changes <b>606</b> with the mean weight change <b>602</b>, the mean positive weight change <b>604</b> and a mean negative weight change <b>608</b> depending on the delay between firing and the input (at the synapse). The latch operation may be described mathematically as:

<maths id="MATH-US-00002" num="00002"><math overflow="scroll"><mtable><mtr><mtd><mrow><mrow><mrow><mi>iff</mi><mo>⁢</mo><mfrac><mrow><mo>ⅆ</mo><mrow><mi>u</mi><mo>⁡</mo><mrow><mo>(</mo><mi>t</mi><mo>)</mo></mrow></mrow></mrow><mrow><mo>ⅆ</mo><mi>t</mi></mrow></mfrac></mrow><mo>&gt;</mo><mn>0</mn></mrow><mo>,</mo><mrow><mrow><mi>then</mi><mo>⁢</mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo>⁢</mo><msub><mi>u</mi><mi>latch</mi></msub></mrow><mo>=</mo><mrow><mrow><mi>u</mi><mo>⁡</mo><mrow><mo>(</mo><mi>t</mi><mo>)</mo></mrow></mrow><mo>.</mo></mrow></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mn>3</mn><mo>)</mo></mrow></mtd></mtr></mtable></math></maths><img file="US9053428B2_D0002.tif" />

When applying the Oja learning rule (or the BCM or other rule), instead of using the current values of inputs at the time of firing u(t), the latched version of inputs u_latchmay be used. This may have several advantages. First, it may not be required to store time stamps and compute time delays in order to apply the learning curve. Moreover, a small memory (latch) may be utilized. This may work because the input may increase before a neuron fires again (e.g., in the dynamic spiking model). Furthermore, variations on this latch condition may be used. For example, the largest total input since the last firing may be utilized, wherein it is being referred to the input post dendritic delay.

Learning Stability

Whether applying a Hebbian rule or STDP-like effects, it is proposed in the present disclosure that weights should be allowed or even designed to polarize (e.g., bipolar tendency to zero or one upon stable learning of a pattern). In other words, it is proposed that a learning rule should polarize weights on learning neurons and depolarize weights for non-learning neurons (neurons reserved for other memories or losing a competition to code a given pattern).

The reason for this is that the bipolar state (weights tending to zero or one) resulting from application of learning rules (STDP, Hebbian or otherwise) may have stability advantages when learning multiple patterns or sub-patterns. This may relate to the learning rule nature (e.g., additive or multiplicative nature). In an aspect, a neuron may be exposed to a pattern that it then learns according to the proposal and thus reaches a bipolar weight state. Subsequent exposure of this neuron with such bipolar weights (having learned that prior stimulus) to a new stimulus (a different temporal pattern) may provide less chance of disturbance of the weights. Thus, it may be less likely for the neuron to unlearn the prior pattern than if the learning rule left the weights distributed between zero and one (not bipolar).

Technically, this may occur because for the weights at or near zero, being multiplied by a learning factor to reinforce them counter to the prior pattern, the change may be minimal due to the weight being at or near zero. In addition, for the weights near one, being multiplied by a learning factor less than one to deemphasize them counter to the prior pattern, the change may be minimal due to the weight being at or near one. On the other hand, naive synapses, with weights in the middle range (or depolarized), may be much more likely to be recruited for a new pattern. In general, it is thus proposed that whatever method is used to adjust weights, that (a) weights should polarize on competitive winning (learning a given pattern), (b) depolarize otherwise (neurons not allocated to learn the given pattern) and (c) the learning rule should be designed such that polarizing weights may not be easily de-polarized.

Improving Robustness and Confidence

Certain aspects of the present disclosure support a method of efficiently measuring the confidence or robustness of a pattern match and a way to translate that into a temporal code. <figref idref="DRAWINGS">FIG. 7</figref> illustrates an example <b>700</b> for three cases of coincident inputs to a neuron's soma of varying magnitudes as a result of synaptic weights, dendritic delays, and combining when exposed to the same pattern.

It should be noted that in order to exceed a firing threshold, the combined coincident input may generally need to exceed a threshold (relative to resting potential). The contribution of weights, threshold, and number of synapses is depicted in the further description. In a case <b>702</b> illustrated in <figref idref="DRAWINGS">FIG. 7</figref>, there may be too few coincident synapses (or too few weights, or a threshold may <b>708</b> be too high). In a case <b>704</b>, the opposite may occur. Only in a case <b>706</b> the match may be perfect. The case <b>704</b> may be considered being either loose (redundant) or robust depending on perspective, context or noise level. Similarly, the case <b>706</b> may be considered perfect (precise or efficient) or sensitive (brittle, non-robust).

It should be also noted that for a given firing, the total contribution of a single input neuron may be determined by the total of synapses with coincident delay (not merely relative to one another but relative to the combination with other inputs) and their weights. If the total coincident input across input neurons for those synapses is below the threshold <b>708</b>, then firing may not occur. On the other hand, if the total coincident input across input neurons for those synapses is above the threshold <b>708</b>, then the firing may occur. This may be problematic, as it can be observed from <figref idref="DRAWINGS">FIG. 7</figref>. If, as depicted as the case <b>704</b>, some parts of the pattern may not be necessary for firing, such as an input pattern from a neuron <b>710</b>. Thus, the pattern match confidence may be low.

Solution to Robustness

It is proposed in the present disclosure a combination of one or more aspects to solve these aforementioned problems related to robustness. First, neuronal regulation or homeostasis may be used to control or normalize total contribution of contributing inputs, in terms of number, weight or otherwise. Thus, a neuron's input level may be adjusted for the target pattern to correspond to the case <b>706</b> from <figref idref="DRAWINGS">FIG. 7</figref> (i.e., sufficiently at or above the threshold <b>708</b>, but not too far over to result in firing without the correct pattern). In fact, the input level may be adjusted by scaling the weights. Also, these weights may be adjusted to give the desired robustness (excess input). This can be a building block for the following description of a confidence or integrity coding method.

It may be desired that an output neuron x matching a temporal spike pattern of input neurons <b>802</b>, <b>804</b>, <b>806</b>, <b>808</b> illustrated in <figref idref="DRAWINGS">FIG. 8</figref>. It can be noticed that if an oscillation <b>810</b> is introduced in a membrane potential or a firing threshold, depending on the phase during which the output neuron x considers the inputs may determine how precise the input may need to be to match the pattern. In an aspect, the most precision may be required at a trough <b>812</b> of the oscillation <b>810</b>. However, at <b>814</b> or even <b>816</b>, less precision may be required to fire the neuron. For example, at <b>814</b>, a spike from the neuron <b>802</b> may be missing entirely, and, at <b>816</b>, a spike from the neuron <b>806</b> may not be required.

Next, a reference firing may be included, as illustrated in <figref idref="DRAWINGS">FIG. 9</figref>. An oscillation <b>902</b> in combination with a reference neuron firing <b>904</b> (e.g. at a trough of the sinusoid <b>902</b>) may be used to convert confidence into a temporal code. As illustrated in <figref idref="DRAWINGS">FIG. 9</figref>, the closer the pattern recognizing neuron fires to the reference (trough) <b>904</b>, the better the match. Therefore, if an output of the reference neuron <b>904</b> and an output of a matcher neuron <b>906</b> are fed into a neuron <b>908</b>, then a temporally coded output of the neuron <b>908</b> may be used as a confidence measure of detecting a spiking pattern <b>910</b>.

The key aspects of the pattern <b>910</b> may be in the spike timing of input neurons <b>912</b>, <b>914</b>, <b>916</b>, <b>918</b>. In addition, the reference neuron <b>904</b> may fire on a particular phase based on the oscillation <b>902</b>. The confidence in the match may be evaluated by the neuron <b>906</b>, but submitting the output of neuron <b>906</b> and the reference neuron <b>904</b> to the neuron <b>908</b> that can learn (or be configured) with delays corresponding to the alignment. For example, one possible configuration can be such that if the output of neuron <b>906</b> aligns with the oscillation trough, then the neuron <b>908</b> may fire, and otherwise it may not. This example shows that in general any correspondence to oscillation phase may be determined and temporally coded.

It should be also noted that if the candidate x (or another variant y or z) fires during an up-wave (or down-wave) of the oscillation <b>902</b>, then the pattern <b>910</b> may not be exactly matching. In an aspect, by matching the temporal difference to the reference time, the neuron <b>908</b> may be easily configured (or learned) to temporally code that poorer quality. It can be noticed that the temporal coding neuron model is utilized as the basis for all of these neurons.

Essentially, confidence may be a function of oscillation phase, membrane time-constant, and the number and weights of coincident inputs. Hence, oscillation may be used to (a) increase or decrease the sensitivity to the number of inputs, (b) increase or decrease the sensitivity to coincidence of inputs, or (c) both.

Probabilistic Confidence and Rate Coding

It should be noted that the confidence may be coded as a rate code by using a bank of neurons having a range of sensitivities to pattern matching accuracy or confidence. The combined spiking of the neurons may act as an aggregate spike count or rate code of the confidence (i.e., more neurons firing means more confidence). For this purpose, outputs of the bank of varying-precision neurons may be fed to a rate coding confidence neuron <b>1002</b>, as illustrated in <figref idref="DRAWINGS">FIG. 10</figref>. <figref idref="DRAWINGS">FIG. 10</figref> illustrates two pattern cases <b>1004</b> and <b>1006</b>, where the first pattern case <b>1004</b> may have an input matching the pattern so all precision neurons fire and the confidence may be rate-coded into a high rate spike pattern <b>1008</b> from the neuron <b>1002</b>. In the second case, the pattern <b>1006</b> may not match as well, so only a subset of neurons may fire and the neuron <b>1002</b> may rate codes to a slower rate, as illustrated by a spike pattern <b>1010</b>.

In an aspect of the present disclosure, the neuron <b>1002</b> may fire a train of spikes in a number or rate that is a function of the number of inputs. This may also be combined with the aforementioned method of temporal-coding in various combinations (stages or pieces of network architecture) to achieve desired high-level robustness effects.

Recruiting Cells for Memories

It is further proposed in the present disclosure that the aforementioned concepts of integrity or confidence and precision are particularly relevant for memory aspects. <figref idref="DRAWINGS">FIG. 11</figref> illustrates a network <b>1102</b> of a large number of interconnected neurons, which may be exposed to a particular input that is desired to be remembered by the network <b>1102</b>. An important question to answer is whether it is necessary for the network <b>1102</b> to remember (code) the output of every single neuron in order for it to be able to recognize this input the next time the network is exposed to this particular input pattern. Another important question to answer is whether it is necessary for the network to remember (code) the output of all of its highest-layer (output) neurons.

It is suggested in the present disclosure that such a global or large scale memory is not only unnecessary but also inefficient. Instead, a method is proposed by which a few (even one) neuron's output may be sufficient to remember the input pattern. It is also disclosed how such a cell or cells recruited by a network system and a memory can be learned.

A method is proposed in the present disclosure to identify a key neuron or neurons for a memory by using, in part, the above proposed confidence method. It is shown how applying an oscillation can identify which neurons are particularly tuned to exact pattern(s) by either the temporal confidence code or probabilistic/rate code. Thus, it may be possible to identify and recruit this cell or cells for a particular memory. This particular cell or cells may be then connected (weights reinforced) to the memory cell inputs to be learned. With a memory cell bank and lateral inhibition, highly efficient storage of many patterns may thus be achieved.

This can be explained in the context of <figref idref="DRAWINGS">FIG. 11</figref> where the network layer or layers <b>1102</b> are abstracted. Inputs <b>1104</b> may be fed to the network <b>1102</b>, and neurons <b>1106</b>, <b>1108</b>, <b>1110</b> may represent few neurons in the network <b>1102</b>. It may be possible to determine which is the most precise temporal coding match for a given input, and assign that as a dominant (or only) input for a memory cell (e.g., the neuron <b>1106</b>). In a sense, the neuron <b>1106</b> can be called the “memory neuron”, but the m cell <b>1112</b> is referred to as the memory cell because it may code the coincidence of oscillation reference <b>1114</b> and the output of neuron <b>1106</b>.

This selection process may be also performed with the Hebbian learning rule. Accordingly, coincident input and output may be wired together so that a memory neuron learns the coincidence. In this case, many network neurons may be initially connected to one or more memory cells, and then the correspondence with the reference may be learned by adapting weights. For example, in <figref idref="DRAWINGS">FIG. 11</figref>, a weight of synapse connecting the neuron <b>1106</b> and the cell <b>1112</b> may represent a strong weight. It should be noted that a temporal pattern model may not be required for the memory neuron because the temporal coding may occur in the network layers. This can be explained in the context of <figref idref="DRAWINGS">FIG. 12</figref>.

In <figref idref="DRAWINGS">FIG. 12</figref>, it is apparent that a neuron <b>1202</b> most precisely codes the input pattern because it fires with the least offset from the oscillation trough. If a memory cell with minimal temporal coding delay range (i.e. mainly a coincidence coder) is used (i.e., a memory cell <b>1204</b>), then the memory cell <b>1204</b> may be trained to fire for the most coincident input, which would be inputs <b>1202</b> and <b>1206</b>. Thus, a mechanism may be developed in the present disclosure for remembering an input pattern with minimal resources (neurons).

Robustness over Long Time Frames

Furthermore, it is proposed in the present disclosure that by feeding an input to different parts of a network subject to different oscillation frequencies or offsets (example shown in <figref idref="DRAWINGS">FIG. 13</figref>), these parts of a temporal pattern of input may be separated (isolated) for robust pattern matching within those network parts and then the results may be recombined. In a highly efficient network, this may even occur by passing the pattern matching back and forth between two network parts as the oscillation peaks in one or the other.

A network may also be configured with only one oscillation and merely sample parts of a pattern but thereby separate the pattern into clearly separated parts in order to “clear” individual neuron states between sections and improve coding/recognition fidelity.

Temporally Correlated Memories

In an aspect, two temporally coded symbolic memories can be considered, which are desired to be connected (associated) to each other. In an aspect, oscillation (rhythms) may be used to re-align neuronal assemblies to obtain any desired overlap that can be encoded temporally. To understand how to do this, the temporal-confidence coding building block described above can be considered.

<figref idref="DRAWINGS">FIG. 14</figref> illustrates a spatial-temporal input pattern <b>1402</b> with rectangles indicating portions of the pattern recognized by particular neurons (in the absence of oscillation, i.e., high fidelity). Now, two oscillations <b>1404</b>-<b>1406</b> may be added, each rhythm applied to two circuits—one circuit comprising neurons <b>1408</b> and another circuit comprising neurons <b>1410</b>. If these two sets of neurons are considered to be two temporal confidence codes, these neurons may be brought into alignment for a secondary coincidence coding by (phase) shifting or scaling (changing the frequency) of the oscillation. It should be noted that with oscillation up-phases, the neurons may fire earlier because of the less stringent conditions. By bringing the two into a resolvable time window, then their association may be coded.

<figref idref="DRAWINGS">FIG. 15</figref> illustrates example operations <b>1500</b> that may be performed at a network of spiking neuron circuits for merging the network with a rule for learning synaptic weights associated with the neuron circuits in accordance with certain aspects of the present disclosure. At <b>1502</b>, synaptic inputs may be provided into a neuron circuit of the network, wherein each of the synaptic inputs may be associated with a synaptic weight and a time delay. At <b>1504</b>, each of the synaptic inputs being weighted and delayed may be latched upon a rise in an input of the neuron circuit comprising the synaptic inputs. At <b>1506</b>, upon the input or upon the neuron circuit spiking based on the rise in the input, the learning rule may be applied on the latched synaptic inputs to determine a change in the synaptic weight associated with that synaptic input.

In an aspect, that weighed and delayed synaptic inputs may be latched when the input of neuron circuit is at a largest value since the neuron circuit fired last time. According to certain aspects of the present disclosure, the learning rule may correspond to one of real-valued Hebbian learning rules, such as the Oja learning rule. Further, the applied learning rule may polarize the synaptic weight associated with that synaptic input. Also, the learning rule may be associated with a shifted STDP learning curve to compensate for a delay from a defined level of depolarization of the synaptic inputs to spiking of the neuron circuit.

In an aspect, the time delay may be equal to one or more multiples of a time delay resolution. The input of neuron circuit may comprise a sum of the synaptic inputs, wherein each of the summed synaptic inputs may be associated with a synapse characterized by the weight and the time delay (e.g., as defined by equation (1)).

In one aspect, the neuron circuit and the synaptic inputs may be associated with a dynamic spiking neuron model. In another aspect, the neuron circuit and the synaptic inputs may be associated with a leaky-integrate-and-fire neuron model.

In an aspect of the present disclosure, as illustrated in <figref idref="DRAWINGS">FIG. 9</figref>, a difference in time between firing of the neuron circuit and firing of a reference neuron circuit of the network may be utilized to temporally code an output of another neuron circuit of the network. The temporally coded output may comprise information about a confidence that a spiking pattern of the synaptic inputs matches a defined pattern, while outputs of the neuron circuit and the reference neuron circuit may be fed into the other neuron circuit to generate the temporally coded output. In another aspect, as illustrated in <figref idref="DRAWINGS">FIG. 10</figref>, an output of the neuron circuit may be provided into another neuron circuit of the network to generate an output of the other neuron circuit. Then, a firing rate of the output of other neuron circuit may indicate a confidence that a spiking pattern of the synaptic inputs into the neuron circuit matches a defined pattern.

In one aspect of the present disclosure, as illustrated in <figref idref="DRAWINGS">FIGS. 11-12</figref>, one of the neuron circuits may be selected as a memory cell to memorize a spiking pattern fed into the network, while oscillation may be applied at an input of a reference neuron circuit of the network. The selection may be based on that neuron circuit responding to the spiking pattern closest to a trough of the oscillation among a set of the neuron circuits.

<figref idref="DRAWINGS">FIG. 16</figref> illustrates example operations <b>1600</b> that may be performed at a neuron circuit of a neural network in accordance with certain aspects of the present disclosure. At <b>1602</b>, a firing rate of the neuron circuit may be computed periodically by counting a number of firings of the neuron circuit within a time period. At <b>1604</b>, it may be determined whether the firing rate is below a lower bound or above an upper bound. At <b>1606</b>, the firing rate may be adjusted by a step amount based on the determination.

In an aspect of the present disclosure, adjusting the firing rate may comprise boosting the firing rate, if the computed firing rate is below the lower bound. In another aspect, adjusting the firing rate may comprise dampening the firing rate, if the computed firing rate is above the upper bound.

According to certain aspects of the present disclosure, a common multiplier may be applied to all synaptic inputs of the neuron circuit to regulate the firing rate. In an aspect, adjusting the firing rate by the step amount may be achieved by adjusting the applied multiplier.

The various operations of methods described above may be performed by any suitable means capable of performing the corresponding functions. The means may include various hardware and/or software component(s) and/or module(s), including, but not limited to a circuit, an application specific integrate circuit (ASIC), or processor. Generally, where there are operations illustrated in Figures, those operations may have corresponding counterpart means-plus-function components with similar numbering. For example, operations <b>1500</b> and <b>1600</b> illustrated in <figref idref="DRAWINGS">FIG. 15</figref> and <figref idref="DRAWINGS">FIG. 16</figref> correspond to components <b>1500</b>A and <b>1600</b>A illustrated in <figref idref="DRAWINGS">FIG. 15A</figref> and <figref idref="DRAWINGS">FIG. 16A</figref>.

As used herein, the term “determining” encompasses a wide variety of actions. For example, “determining” may include calculating, computing, processing, deriving, investigating, looking up (e.g., looking up in a table, a database or another data structure), ascertaining and the like. Also, “determining” may include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory) and the like. Also, “determining” may include resolving, selecting, choosing, establishing and the like.

As used herein, a phrase referring to “at least one of” a list of items refers to any combination of those items, including single members. As an example, “at least one of: a, b, or c” is intended to cover: a, b, c, a-b, a-c, b-c, and a-b-c.

The various operations of methods described above may be performed by any suitable means capable of performing the operations, such as various hardware and/or software component(s), circuits, and/or module(s). Generally, any operations illustrated in the Figures may be performed by corresponding functional means capable of performing the operations.

The various illustrative logical blocks, modules and circuits described in connection with the present disclosure may be implemented or performed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array signal (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any commercially available processor, controller, microcontroller or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

The steps of a method or algorithm described in connection with the present disclosure may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in any form of storage medium that is known in the art. Some examples of storage media that may be used include random access memory (RAM), read only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM and so forth. A software module may comprise a single instruction, or many instructions, and may be distributed over several different code segments, among different programs, and across multiple storage media. A storage medium may be coupled to a processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor.

The methods disclosed herein comprise one or more steps or actions for achieving the described method. The method steps and/or actions may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and/or use of specific steps and/or actions may be modified without departing from the scope of the claims.

The functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media include both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage medium may be any available medium that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared (IR), radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray® disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Thus, in some aspects computer-readable media may comprise non-transitory computer-readable media (e.g., tangible media). In addition, for other aspects computer-readable media may comprise transitory computer-readable media (e.g., a signal). Combinations of the above should also be included within the scope of computer-readable media.

Thus, certain aspects may comprise a computer program product for performing the operations presented herein. For example, such a computer program product may comprise a computer readable medium having instructions stored (and/or encoded) thereon, the instructions being executable by one or more processors to perform the operations described herein. For certain aspects, the computer program product may include packaging material.

Software or instructions may also be transmitted over a transmission medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of transmission medium.

Further, it should be appreciated that modules and/or other appropriate means for performing the methods and techniques described herein can be downloaded and/or otherwise obtained by a user terminal and/or base station as applicable. For example, such a device can be coupled to a server to facilitate the transfer of means for performing the methods described herein. Alternatively, various methods described herein can be provided via storage means (e.g., RAM, ROM, a physical storage medium such as a compact disc (CD) or floppy disk, etc.), such that a user terminal and/or base station can obtain the various methods upon coupling or providing the storage means to the device. Moreover, any other suitable technique for providing the methods and techniques described herein to a device can be utilized.

It is to be understood that the claims are not limited to the precise configuration and components illustrated above. Various modifications, changes and variations may be made in the arrangement, operation and details of the methods and apparatus described above without departing from the scope of the claims.

While the foregoing is directed to aspects of the present disclosure, other and further aspects of the disclosure may be devised without departing from the basic scope thereof, and the scope thereof is determined by the claims that follow.