Is there a way to know the progress of compilation?

Support
1

Description of problem

Hi, all. This topic is a continue of How to compute the time-average values on the fly? - #9 by Jasmine9691.

Now my model has over 160 equations, hundreds of neurons and synapses. It takes me really a LONG time to compile using no matter cython or cpp_standalone. One hour has passed but the compilation has not been completed. I wonder

  1. whether there is a way to know the progress of compilation, so that I can know how much time I have to wait for.
  2. whether there is a way to accelerate the compilation, e.g., something like make -j8.

Thanks in advance!

2

I compiled step by step, and found this could be faster (1-3 minutes).

For example, my full code is

A = NeuronGroup(10, eqs_A)
B = NeuronGroup(20, eqs_B)
S = Synapses(A, B, on_pre=eqs_S)
...
net = Network(A,B,S,...)
net.run(10*ms)

Compiling this code should have been much too time-consuming.

I first compiled only A and commented others:

A = NeuronGroup(10, eqs_A)
#B = NeuronGroup(20, eqs_B)
#S = Synapses(A, B, on_pre=eqs_S)
#...
net = Network(A)
net.run(10*ms)

This compilation took < 10 second.

Then I uncommented B and added it to the net:

A = NeuronGroup(10, eqs_A)
B = NeuronGroup(20, eqs_B)
#S = Synapses(A, B, on_pre=eqs_S)
#...
net = Network(A, B)
net.run(10*ms)

This compilation also took < 10 second.

Similarly, I uncommented other objects and compiled them step by step. And the total compilation took me only 1-2 minutes. During the process, some bugs (e.g., some identifiers cannot be resolved) were also quickly detected and resolved. I guess they could also have slowed down the compilation. To validate this, I clear_cache('cython') after all the bugs were fixed, and found that the compilation took only 3 minutes.

3

Hi, for Cython, I don’t see a way to get feedback on the compilation progress without changing Brian’s package source code manually. For C++ standalone, you can give extra arguments to the make command (note that it already passes -j on Linux/macOS by default). You’ll also need to switch on debug mode since otherwise all output is suppressed. E.g. you can use:

set_device('cpp_standalone', debug=True)
prefs.devices.cpp_standalone.extra_make_args_unix += ['--debug=bj']

But compilation – especially with Cython – will always be slow if you have many objects. Given that you state that you have “160 equations, hundreds of neurons and synapses”, I am pretty sure that you can drastically reduce this number by putting together groups of neurons that share the same basic equations, and only differ in parameters. You can always use Subgroups (i.e. neurons = NeuronGroup(...); group1 = neurons[0:10]; group2 = neurons[10:20] etc.). This will drastically reduce the time for compilation, since every NeuronGroup results in between 1 and 3 files that each need to be compiled individually, and similar for Synapses.

1 Like
4

Hi @mstimberg . Thanks for you kind suggestions. I tried subgroups with same equations and parameters but different synaptic connections. It indeed speeded up the compilation and a bit simulation. Also, I could intialize all the subgroups together, which saved dozens lines of my code.

1 Like