@adam, a while ago, I stumbled on a similar problem with a bit of complication, that my simulations are pretty long (up to a few days in model time 
). Of course, no one memory can hold even spike times for a whole run, say nothing about time series of voltages or whatever one needs from simulations. The simulator must be stopped every while, intermediate results should be saved, and all buffers must be cleared. So for this kind of data, we need a suitable file to hold chunks of data and keep track of each fragment. I’ve tried Pandas and raw hd5 for this, but both weren’t up to the job. Specifically, hd5 is pretty fragile and doesn’t have enough redundancy to restore data structure when the simulation crashes with bugs and a hd-file doesn’t close properly. I lost 3 days run on this unfortunate ‘feature’ of hd5. I finally figured out that it isn’t a highly complex task and wrote my one python module to hold this kind of data.
But this isn’t the end. Sorry for the long post. My models have lots of parameters, so I thought it may be handy to save parameters for each simulation along with very general statistics of the result, some graphs (in the form of pictures), and my comment on the run. This may be an excellent job for Pandas frame or hd5, but after some tests, I landed to sqlight and general SQL queries. Again, I wrote a wrapper to have a tree-like structure to hold parameters. It got complicated here because I wanted to provide some help-line for each parameter and change parameters through CLI arguments (you see, I’m lazy)! So what I’m using now looks like that:
1. Default configuration.
stkdefault = """
/{
Some general words about the model
The model was coded by me
}
/network/{
/geom/{
/shape = 'H' ; H-hexagonal or S-square
/x = 7 ; number of columns
/y = 16 ; number of raws
}
/gj/{
/geom/{
/p = 0.3 ; gap junction probability
/unit = 'n' ; if n - probability of connection per neuron. it 'c' - probability of connection per pear of neurons within maxd distance
/maxd = 1.5 ; gap junction max distance
/dir = None ; gap junction direction (random if None)
}
/r = 700. ; gap junction resistance
}
/syn/{
/gsyn = None \; synaptic conductance:
\; if positive - set for everyone,
\; if negative - set the portion of minimal synaptic conductance for each neuron,
; if None set to 1/3 of the minimal synaptic conductance for each neuron.
/rand = False ; If ture - gsyn randomly init between 0 and /syn/gsyn value
/prenorm = True \; If true the total synaptic conductance will be set to /syn/gsyn value for each neuron in the **beginning**!
; It is active only in model building. For dynamical normalization see /network/btdp
/geom/{
/pmax = None \; peak of synaptic probability for exp or exp2 distribution.
; If None closest LGN cells are connected to each rGC
/sig = None ; sigma for exponential distribution. Works only if /network/syn/geom/pmax is not None
/sig2 = None ; sigma for gaussian distribution. Works only if /network/syn/geom/pmax is not None and /sig is None
/o2o = False ; one-to-one creates only one connection for each rGC to a closest LGN cell.
/a2a = False ; all-to-all connections: each rGC connects all dLGN cells. It is the best configuration for btdp
}
/NMDA2AMPA = 2.25 ; ratio between peak of NMDA current to peak of AMPA current
/AMPA/{
/tau1 = 1. ; AMPA rising time constant
/tau2 = 2.2 ; AMPA falling time contant From Chen & Regehr 2000 Table 1 P10-P14
}
/NMDA/{
/tau1 = 1. ; NMDA rising time constant
/tau2 = 150. ; NMDA falling time contant From Chen & Regehr 2000 Table 1 P10-P14
}
/ppr_u0 = 0.3 \; sets presynaptic single spike depression
; it set to obtain paired-pulse ratio 0.73 From Chen & Regehr 2000 Table 1 P10-P14
/toneExc = None ; Conductance for tonic excitation (None to block)
/toneInh = None ; Conductance for GABA_B receptors (None to block)
}
......
}
"""
#building model parameter tree and process a CLI arguments
mth = methods(stkdefault, 'mth', locals(), argvs = args )
#and using them
syn_equ="""
da1/dt = -a1/{/AMPA/tua1}/ms :1
da2/dt = -a2/{/AMPA/tua2}/ms :1
dm1/dt = -m1/{/NMDA/tua1}/ms :1
dm2/dt = -m2/{/NMDA/tua2}/ms :1
""".format(**mth["/network/syn"])
2. Help generation.
python mymodel.py --help
Some general words about the model
The model was coded by me
Usage: mymodel.py [options] [parameters]
Any model parameter can be altered by /parameter_name=parameter_value in command line
Parameters:
/network/geom/shape = 'H'
: H-hexagonal or S-square
/network/geom/x = 7
: number of columns
/network/geom/y = 16
: number of raws
/network/gj/geom/p = 0.3
: gap junction probability
/network/gj/geom/unit = 'n'
: if n - probability of connection per neuron. it 'c' - probability of connection per
: pear of neurons within maxd distance
/network/gj/geom/maxd = 1.5
: gap junction max distance
/network/gj/geom/dir = None
: gap junction direction (random if None)
/network/gj/r = 700.
: gap junction resistance
/network/syn/gsyn = None
: synaptic conductance: if positive - set for everyone, if negative - set the
: portion of minimal synaptic conductance for each neuron, if None set to 1/3 of the
: minimal synaptic conductance for each neuron.
/network/syn/rand = False
: If ture - gsyn randomly init between 0 and /syn/gsyn value
/network/syn/prenorm = True
: If true the total synaptic conductance will be set to /syn/gsyn value for each neuron
: in the **beginning**! It is active only in model building. For dynamical
: normalization see /network/btdp
/network/syn/geom/pmax = None
: peak of synaptic probability for exp or exp2 distribution. If None closest LGN cells
: are connected to each rGC
/network/syn/geom/sig = None
: sigma for exponential distribution. Works only if /network/syn/geom/pmax is not None
/network/syn/geom/sig2 = None
: sigma for gaussian distribution. Works only if /network/syn/geom/pmax is not None and
: /sig is None
/network/syn/geom/o2o = False
: one-to-one creates only one connection for each rGC to a closest LGN cell.
/network/syn/geom/a2a = False
: all-to-all connections: each rGC connects all dLGN cells. It is the best configuration
: for btdp
/network/syn/NMDA2AMPA = 2.25
: ratio between peak of NMDA current to peak of AMPA current
/network/syn/AMPA/tau1 = 1.
: AMPA rising time constant
/network/syn/AMPA/tau2 = 2.2
: AMPA falling time contant From Chen & Regehr 2000 Table 1 P10-P14
/network/syn/NMDA/tau1 = 1.
: NMDA rising time constant
/network/syn/NMDA/tau2 = 150.
: NMDA falling time contant From Chen & Regehr 2000 Table 1 P10-P14
/network/syn/ppr_u0 = 0.3
: sets presynaptic single spike depression it set to obtain paired-pulse ratio 0.73
: From Chen & Regehr 2000 Table 1 P10-P14
/network/syn/toneExc = None
: Conductance for tonic excitation (None to block)
/network/syn/toneInh = None
: Conductance for GABA_B receptors (None to block)
3. Changes in parameters.
Try model with different AMPA time constant as simple as
python mymodel.py /network/syn/AMPA/tau2=30.5
3. All parameters are saved in a single database
even with some preview
and diff between model parameters
Of course, one can put lightweight statistics into a “parameter file” and analyze it later. I usually compute mean firing rates or R2 synchronization index at the end of the run, so I can also see the diff in results with the diff in parameters. Moreover, I can collect parameters/data through the database and export it to Pandas/NumPy for analyzes.
4. BUT
Both codes for parameter manipulation and for storing big chunks of data are combined in one simulation toolkit simtoolkit. Unfortunately, all of this isn’t in great shape. It needs lots of development and helps with bug fixing. However, whenever I’m trying to finish documentation and create a GitHub page for this project, I’m sinking into words and postponing the GitHub repository. At this point, it works only in my hands, and I’m not sure if it will be helpful to anyone else.
I must spend a couple of days cleaning code and documenting this project at some point, but if you find it interesting to consider as an alternative, I’ll commit to doing this work asap.
RTH
!
.
It would be great to support this somehow… The general architecture of the exporter is to deal with each variable individually (and you can ask