Running simulations in parallel

Support
1

Hi!

What’s the recommended way to run simulations in parallel? Let’s say I only want change some initial conditions and input stimuli (for which I have explicit spike times).

Thanks,

Sebastian

2

Hi,

I have created a script that allows you to submit multiple simulations to your system and even specify the number of threads you want to reserve. I have a 12 thread cpu and here I have set it to utilize 6 of them (you can adjust this as you wish). So I can run 6 simulations in parallel, and when one of them finishes running, the next one in the row will take its place. Check this out, it’s actually quite simple:

import os
from numpy import arange
from time import time
from multiprocessing import Pool


#### Define functions #########################################################
freq_range = arange(0.5, 1.6, 0.1, dtype=float)
def sim_params():
    """Specify all combinations of simulation paramaters"""
    params = []
    for input_ec in [True]:
        for input_ca3 in [True]:
            for noise in [True, False]:
                for freq_ec in freq_range*82:
                    for freq_ca3 in freq_range*120:
                        _params = ' '.join(
                            [f'INPUT_EC={input_ec}',
                             f'INPUT_CA3={input_ca3}',
                             f'NOISE={noise}',
                             f'FREQ_EC={freq_ec:.1f}*Hz',
                             f'FREQ_CA3={freq_ca3:.1f}*Hz',
                             'N_PYR=10000',
                             'T_SIM=500*ms'])
                        params.append(_params)
    return params


def run_process(process):
    """os call to run a single simulation"""
    os.system(f'python {process}')


def submit_processes(scriptname=None, args=None, threads=1):
    to_submit = [f'{scriptname} {arg}' for arg in args]
    pool = Pool(processes=threads)
    pool.map(run_process, to_submit)


#### Submit jobs ##############################################################
t_start = time()
script = 'simulations.py'
cmd_params = sim_params()
n_threads = 6
print(f'NUMBER OF SIMULATIONS: {len(cmd_params)}')
print(f'MAX NUMBER OF THREADS: {n_threads}\n')
submit_processes(scriptname=script, args=cmd_params, threads=n_threads)
t_end = time()
print(f"\nOverall time: {t_end-t_start:.2f} s")

You can create your own sim_params() function according to your needs.
In the top of ‘simulations.py’ (where I have my Brian code), after the imports I have this piece of code:

params_list = []
for i in sys.argv[1:]:
    var, value = i.split('=')
    exec(f"{var} = {value}")
    params_list.append(f"{var}={value}")

params = ' '.join(params_list)
sim_id = params.replace("*", "")
print(f"Simulation { {sim_id} } submitted.")

I have no idea if this is the optimal way but for me it works like a charm. Also in my case, I see huge performance improvement when I set:

brian2.prefs.codegen.target = 'numpy'

I hope I helped :slight_smile:
Good luck.

2 Likes
3

Hi!

I’ve found brian2/examples/multiprocessing at master · brian-team/brian2 · GitHub which gives some examples on using Python’s multiprocessing. Seems to be what I’ve been looking for.

Cheers,

Sebastian

2 Likes