I am unable to make an edit, when I click the edit icon, it shows a history page with the incomplete post and am unable to write anything.
I have added the Memory layer and made connections from the Input layer to the memory along with recurring connections within the Memory layer to facilitate sustained activity. The self connections and the connections from the Input are excitatory. I get a huge burst of spikes from the Input to the Memory layer and get sustained activity even if I turn off the stimulus and also recurrent connections. A constant 25k spikes get added to the memory layer and am unable to fathom how that occurs!
I am using a conductance based equation for the Memory layer
g_leak = 10 * nS
w_e = 0.05 * nS
Cm = 200 * b2.pF
Vtm = -71 * mV
lif_eqs = '''
dv/dt = -(g_leak * (v - El) - (g_exc * (v - E_exc)) - (g_inh * (v - E_inh)))/Cm : volt (unless refractory)
dg_exc/dt = -g_exc/taue: siemens
dg_inh/dt = -g_inh/taui: siemens
'''
mem_neurons = 50
memory_layer = b2.NeuronGroup(mem_neurons,
exc_lif_eqs ,
threshold = 'v > Vtm' ,
reset = 'v = El' ,
refractory = 5 * ms ,
method = 'euler' ,
name = 'memory_layer')
mem_circle = memory_layer[0 : 25]
mem_circle.v = 'El + rand() * (Vtm)'
mem_circle.g_exc = 'rand() * w_e'
# Memory cross
mem_cross = memory_layer[25 : ]
mem_cross.v = 'El + rand() * (Vtm)'
mem_cross.g_exc = 'rand() * w_e'
Since the behavior is supposed to be excitatory per the reference paper, on the syanpse connection I update the weights like so
# Input circle to memory circle
Syn_neu_circle_mem = b2.Synapses(neu_circle, mem_circle, 'w : siemens' , on_pre = 'g_exc += w')
# Input cross to memory cross
Syn_neu_cross_mem = b2.Synapses(neu_cross , mem_cross , 'w : siemens' , on_pre = 'g_exc += w')
I run the simulation initially for 3 seconds and then turn off the stimulus and run again for 3 seconds and get the following results
With Stimulus
Poisson Circle Spikes: 3016
Poisson Cross Spikes: 2246
Poisson Triangle Spikes: 2506
Neuron Circle Spikes: 169
Neuron Cross Spikes: 50
Neuron Triangle Spikes: 63
Memory Circle Spikes: 24647
Memory Cross Spikes: 24708
After turning off stimulus
Poisson Circle Spikes: 3016
Poisson Cross Spikes: 2246
Poisson Triangle Spikes: 2506
Neuron Circle Spikes: 169
Neuron Cross Spikes: 50
Neuron Triangle Spikes: 63
Memory Circle Spikes: 49647
Memory Cross Spikes: 49708
I donât understand how so many spikes get propagated from the Input layer and where does the constant 25000 new spikes come from without any sign of decaying. This happens despite disabling the self recurrent connection which is supposed to promote sustained activity. Is there any way to tweak parameters uto limit the massive spike bursting activity?
I think thereâs an issue with your spike threshold Vtm at -71mV. This is very low and even below El which you also use as a reset. This means: without any input, the membrane potential goes towards El and will therefore cross the threshold. Then, as soon as a neuron spikes, its membrane potential will be reset above the threshold and it will immediately (after the refractory period) spike again. A typical threshold value would be rather on the order of -50mV. Also, two minor comments:
- The initialization here does not look correct:
This will initializemem_circle.v = 'El + rand() * (Vtm)'vto a value betweenElandEl + Vtm. What you typically want is to initialize it to a value betweenElandVtm(assuming a corrected threshold). So youâd rather writemem_circle.v = 'El + rand() * (Vtm - El)' - Really a minor efficiency point, but if you are using the same initializations for
vandg_excinmem_circleandmem_cross, you can directly writememory_layer.v = ...and initialize things for all the neurons.
Apologies that the edit functionality is still not working. Iâll open a new topic on #site-feedback for this to not âpolluteâ the topic here any further.
@mstimberg: The threshold was set low as with the regular value of -65 mV there were no spikes generated even with a very high firing rate of 255 Hz.
In the initialization I missed the El and have since corrected that value and thank you for suggesting the common initializations for v and g_exc and have incorporated your advise in my code. Are there any parameters that can be tweaked or is the input image not strong to propagate spikes? Is there any other way to try in terms of synapse connections or neuron equations?
Iâll post the screenshot for the edit functionality.
@mstimberg: I am unable to find that incomplete post of mine and so I tried making an edit in this post and thankfully I was able to do so. I guess you can mark it as one-off event and close it and we can look into if it happens again? 
You can certainly tweak the threshold a bit to adapt firing rates, but you cannot have a threshold that is below the value you use for the reset, otherwise you will get a neuron that fires with the maximal possible rate (as determined by the refractory period) continuously. In general, the El value should also be below the threshold, unless you want your neurons to be active without any input.
If your neurons do not spike despite a high input firing rate then there is either something wrong with the connection from the input population or your weights are too low. As @kernfel suggested earlier, maybe use a StateMonitor to record the activity from the neurons in your memory layer to see what is going on?
Great, good to hear. Iâll put a note in the other topic.
1 Like
@mstimberg: I recorded the values using a StateMonitor and most of them reach upto -66 mV and the threshold is set at -65*mV. Some reach -65.8 mV.
Memory state volt: [[-65.87065871 -65.9081045 -65.94507742 ... -70. -70.
-70. ]
[-66.98951389 -67.02202246 -67.05405374 ... -70. -70.
-70. ]
[-65.76001058 -65.81392251 -65.86696343 ... -70. -70.
-70. ]
...
[-66.16898763 -66.20320231 -66.23699153 ... -70. -70.
-70. ]
[-69.93669021 -69.96679429 -69.99613622 ... -70. -70.
-70. ]
[-66.14873069 -66.18042726 -66.21176369 ... -70. -70.
-70. ]] mV
The max limit of the weights in the memory layer is w_e = 0.9 * nS and the layers weights are initialized in this manner
Syn_mem_circle_circle.w = 'rand() * w_e'
Syn_mem_cross_cross.w = 'rand() * w_e'
Syn_neu_circle_mem.w = 'rand() * w_e'
Syn_neu_cross_mem.w = 'rand() * w_e'
From your comment about a possible wrong connection from the input I was thinking if I should try to generate the spikes from a NeuronGroup having a simplified LIF equation and store the spike times and indices and pass them on to the Input neurongroup via SpikeGeneratorGroup?
Iâll try some of the things above and see how it goes and thank you once again for offering assistance in this tough problem.
0.9nS should be reasonably strong, but of course it depends on the number of synapses and how often they spike. I think you said 255 Hz for the spike rate (which is pretty high), but how many synapses do you have? As a side note: I saw in some of your earlier code that you use all-to-all connectivity between the input PoissonGroup and the next layer. This means that all cells get exactly the same input, possibly not what you want. Apart from the strength and number of inputs, the reversal potential of your synapses is also important. If for example you have E_exc set at a value like -65mV, this would mean even the strongest possible input would only drive your membrane potential towards -65mV. A common value for E_exc is 0mV. Hope that helps!
1 Like
@mstimberg: E_exc indeed has been set to 0 mV as I want it to be an excitatory synapse and E_inh is set to -70 mV for inhibitory synapse.
To get the number of synpases can it be done using N_incoming and N_outgoing. In the synpase connection scheme : all-to-all I could not understand what you mean by all cells getting the same input? Since every cell is connected to each other would it not transmit each cells input to every other cell thereby all the cells receiving inputs from multiple cells?
I tried with a simple 1-1 scheme but that lead to no spikes in any of the input subgroups. I am also looking for some papers on connectivity and going through Brian2 examples to see how to connect only neurons which are spiking or receive strong stimuli. I donât want to sound cliched but really appreciate all your assitance offered by dedicating your time for these posts, it really helps a lot.
Ok, E_exc at 0mV is perfect. E_inh at -70mV is probably a bit too high (the inhibitory synapses will not have any effect if the membrane potential is around rest), but this is nothing to worry about right now.
Yes, there are a number of ways to get the number of synapses, you could have a look at N_incoming_post to get the number of synapses incoming for each neuron (see the documentation). But if you are not using probabilistic connections, the number of connections should be clear from the way you connect them. If you have e.g. 20 neurons connecting to 50 neurons with all-to-all connectivity, then you have 20 incoming synapses per target neuron for a total of 1000 synapses.
What I meant is: you have X neurons based on PoissonGroup, i.e. spiking randomly with a given rate. If you connect them in an all-to-all fashion, that means that each neuron in the target group gets spikes from all the X neurons. So, yes, all cells gets input from several neurons, but all cells get input from the same neurons, so the summed input is exactly identical. This is what you want if the weights of these connections are the ones you want to learn/adapt. But if your PoissonGroup neurons + the synapses of their outgoing connections are only meant as a description of random stimulation for individual cells, then a 1-to-1 connection would make more sense.
Make sure to have a look at the Synapses tutorial and the visualization function that is used in there to better understand the setup of connectivity patterns. In general, you might also want to install the brian2tools package, its plotting functions (in particular the brian_plot function) can be very useful to do quick&easy plots.
1 Like
Yes @mstimberg the idea is to learn the pattern and recognize it when presented with stimulus by the target group. This was the idea behind having an all-to-all connectivity. If my understanding is correct that is the aim presented in the reference paper. Do you think the summed input may not be strong enough to drive the target neurons to spike?
From your example, I have 50 source neurons connecting to 75 target neurons for a total of 50 incoming synapses for each target of 75 neurons and this totals to 3750 synapses. These should be sufficient to drive some spikes (?) but still at a loss on what parameters to tweak further. I am using the visualization function and thank you for the brian2tools package and will be installing them. I am looking at some brian2 tutorials and code examples on github but I am unable to get examples where sources of input include real world datasets like sample image array used in this simulation.
Which parameters want adjusting depends a lot on what youâre trying to do with the model, in the end. For example, if youâre interested in making it biologically plausible (to whatever extent), then that constrains parameter values in different ways than if you want to treat the model as a spiking machine learning tool. Ultimately, every parameter (including structural choices) has some effect on the modelâs behaviour, and is thus worthy of consideration; part of the art of modelling is figuring out which parts of a model are important in terms of not just the model behaviour, but also your goals.
We can help you with that to some degree, particularly with the more technical aspects of the work, but ultimately, you are the expert on the model youâre building - and so it is up to you to gain an intuition of how it behaves under various circumstances, which parameters or input regimes it is particularly sensitive to, etc.; this is not something we can do for you.
To gain this intuition, you may need some practice - change parameters one at a time, observe the changes, and take notes along the way. To observe the model, youâll want to do lots of plotting - record the membrane voltages and plot (some of) them; record the spike trains and plot them; make summary statistics of the whole population (and possibly plot them) if itâs too large to observe as a whole; and more. The more you learn about how your model behaves â besides understanding the maths behind it, of course! â , the better youâll be able to intuit how to make it do what you want it to do.
Thank you @kernfel, I am aware of what I can expect and what not. I am not asking for a fully written solution neither I have written that way in my posts. I was asking an opinion on if a certain parameter may or may not have an effect which is different from a direct solution. I have been experimenting and tinkering with a lot of parameters and going over resources before coming or posting in this forum or any for that matter. I appreciate your efforts in helping out with some and would certainly appreciate if you donât assume and come to conclusion about my intent if that is what you had in mind.
I may be new to computational modelling but I have experience in modelling in other aspects of CS and software engineering and know where to draw the line on expectations from internet forums. So really appreciate your post and adviceâŚcheers 
1 Like
I appreciate that. I reacted mainly to your last set of questions, which are difficult to answer well without making assumptions about what youâre trying to achieve:
This, I would say, is an empirical question, answered by plotting the postsynaptic voltages;
The implied question about sufficiency very much depends upon baseline assumptions; a specifically engineered model could drive spikes at any input level (including without input at all, as youâve already seen), and what level should be enough is a modelling decision. The parameters youâd touch for that, in particular, are the synaptic weights, the bias current (if any), the membrane time constant (or leak conductance, equivalently), the resting potential, the firing threshold, the synaptic time constants and reversal potentials ⌠what Iâm trying to point out is that, very likely, you know these better than we do, and youâve probably tried to tune several of those already.
Iâm not trying to brush you off, of course. I apologise if it came across that way, and I certainly hope youâll keep sharing your progress â I guess what I should have written in the first place is: These are difficult questions that, at least for my part, require more insight into the project than youâve provided us with, including possibly hands-on work on the model itself.