Multiple run in standalone mode

[Jul94]

My case is a bit different from this one but also involves multiple runs in the standalone mode. The code is a slightly modified version of the Brian2 and Python3 port of the paper of Diehl&Cook2015, “Unsupervised learning of digit recognition using spike-timing-dependent plasticity”, which has been mentioned a few times in this forum (see this and this). However, I think a general outline of the code will suffice:

set_device('cpp standalone', build_on_run=False)

# Define auxiliary functions

# Load dataset

# Set parameters and equations

# Create network and connections

# Run the simulation
net = Network()
for obj_list in [neuron_groups, input_groups, connections, rate_monitors, 
spike_monitors, spike_counters]:
    for key in obj_list:
        net.add(obj_list[key])
...
# loop until the whole dataset has been covered
while j < (num_examples):
  # several run() calls for resting and presentation times

device.build(compile=True, run=True, debug=True, clean=False)

# Save results

# Plot results

device.delete(code=False)

So the key idea here is that we have a (large) network and run it in a loop, but not because we are trying different parameters/network configurations, but because we are presenting the different input patterns to the network.

When trying to run the simulation in standalone mode, I noticed the execution speed gradually slowed down as the simulation progressed. I understand that is due to what @mstimberg explained in a different thread:

@mstimberg

The problem is the compilation stage. What standalone basically does to support multiple runs, is that it generates the whole network code over and over again for each run (one reason is that external constants are hardcoded into the code, so to take this into account we have to write the code again). For complex networks this means that there is a potentially huge number of code files that need to be compiled. This not only takes a long time, but depending on the compiler and OS it can also simply run into a “command line too long issue”. Finally, on linux we run make -j , which means that it uses independent parallel processes to compile all the source files. If there are many of them, this can take a lot of memory and make things crash. I would therefore consider changing the devices.cpp_standalone.extra_make_args_unix preference to something like ['-j', '8'] (for 8 processes, adapt this to the number of processors on your machine). I thought we had already changed this, but apparently not…

Finally, there’s a much faster technique where you only compile a single code and then change parameters “under the hood”, but this approach is not yet exposed by Brian. If you are interested in that approach you should have a look at this gist .

I have yet to try the approach from that gist (at first I thought it’s not exactly the same situation, but it might work nonetheless), but on a more general note, I am wondering what is the best workflow in that scenario. That is, what is the best approach when you want to speed up a simulation where you feed an entire dataset to a large and fairly complex network (without any other change of parameters)? Any ideas?

Note: This is my first attempt at implementing a complex simulation in Brian, so I have still quite a few things to try and learn. For instance, one of my next steps will be to use a proper event-based dataset (e.g. N-MNIST) instead of standard MNIST, I think that might simplify some things about the original structure of the code. But of course any other ideas and advice is really appreciated!