Description of problem Hi, I have a couple of issues that appear to be related, stemming from the use of multiprocess: parallelizing simulations with multiprocess.Pool produces a lot of warning messages, but it doesn’t kill the process and the code runs to completion when calling via “python my_s…

Without having the time to delve into this further: I saw Cannot allocate memory in your error message, so can you make sure that you are not simply running out of memory and/or disk space when you run the simulations in parallel?

Aside from the traceback, the actual errors seem to be of the form File “/home/macke/rgao97/.conda/envs/automind_slurm/lib/python3.8/logging/handlers.py”, line 170, in doRollover os.remove(dfn) FileNotFoundError: [Errno 2] No such file or directory: ‘/tmp/brian_debug_m6zje3k0.log.1’ which points t…

It’s tricky to debug these sorts of issues when it only occurs on a particular cluster, etc. It looks like the issue is something to do with the logging system, so one option would be to turn logging to a file off and see if that fixes the problem. You need to update the preference logging.file_log…

hey @dan and @kernfel , thanks for your replies! I tried a bunch of things this week to debug this, no luck, but I will document it all here in case it’s helpful / if someone else stumbles onto the same problem. tl;dr: it seems like multiprocess is really the problem, because running the job with th…

yeah wow that did it: I was allocating 10GB across 64 processes, which was apparently not enough, 5GB per process solved it. thank you @mstimberg !!!

Simulation with multiprocess returns weird errors and hangs on slurm cluster

Support

mstimberg Split this topic 23 April 2021 08:55 7

A post was split to a new topic: Problem running benchmarks from Tikidji-Hamburyan et al 2017

Topic		Replies	Views
Multiprocessing in standalone mode Support	5	1125	15 February 2021
Python crash or random computer reboot Support	3	273	15 March 2024
Error while running parallel standalone simulation in Brian2? Support performance , parallel	4	35	1 September 2025
Error while using brian2 parallelization on Linux cluster Support parallel , compiler	3	87	19 November 2024
Fatal error: 'stdio.h' file not found, problem with StateMonitor of threshold adaptation Support compilation , clang , cython , macos	2	729	15 November 2022

Simulation with multiprocess returns weird errors and hangs on slurm cluster

Related topics