Description of problem
I have a functioning standalone simulator but it does not work with joblib because my simulator is implemented in an object oriented way. I am trying to figure out how to make it work without completely changing the implementation. I am aware of this example and I basically need the object oriented version of it.
Minimal code to reproduce problem
I can’t provide minimal code but this pseudo code should illustrate the issue:
The simulator runs like this:
simulator = Simulator()
parameters = [1, 2, 3]
result = simulator.run(parameters)
Internally, Simulator()
creates and compiles the network and simulator.run(parameters)
parses the parameters and uses b2.device.run(run_args=run_args)
to run the simulation. This works generally well but causes problems with joblib. As I understand it, the fundamental issue is that each job needs its own compiled version of the code. So Simulator()
needs to create not one but n_job
simulators. Deciding how many jobs there should be at simulator instantiation time is no problem. Simulator
has a self.make_network()
method and I will simply call it n_job
times. But the above example uses os.getpid()
, and I will only know the pid
when I pass simulator.run() to joblib, so I cannot use it at instantiation time. Is there anything else I can use to make the jobs not interfere with each other? Any pointers/ideas would be appreciated.