Delete function usage in Brian2.8

Support
1

Hi @mstimberg , I have the following code using the delete function but the number of files still keep on increasing. Majority of the files are in ‘cpp_standalone0/static_arrays’. Could it be that I didn’t use it correctly and Is there a way to check this further?

    for stimulus_idx, (img, label) in enumerate(tqdm_ray.tqdm(data_in)):
        switch_stimulus = False
        stimulus_strength = input_param.scale_img # starting strength
        self.repetitions = 0 # num of repetitions for one sample
        while switch_stimulus == False:
            scaled_img = img*stimulus_strength
            arguments = {
                self.input_neurons.rate: scaled_img*brian2.Hz, 
                self.input_neurons.label: label, 
                self.input_neurons.stimulus_idx: stimulus_idx, 
                self.input_neurons.stimulus_strength:stimulus_strength, 
            }
            for ih in range(self.N_hidden):
                arguments.update({
                    self.exci_neurons[ih].stimulus_idx: stimulus_idx,
                    self.exci_neurons[ih].stimulus_strength: stimulus_strength,
                    self.exci_neurons[ih].label: label,
                })
                
            if label!=-1: # skip the fake stimulus
                arguments.update({self.exci_neurons[ih].vt: vt[ih][:]})
                
                if self.task == 'train': 
                    for ih in range(len(self.S_input2e)):
                        arguments.update({
                            self.S_input2e[ih].w: S_input2e_w[ih][:], 
                        })
                    for i in range(len(self.S_efe)):
                        for j in range(len(self.S_efe[i])):
                            arguments.update({
                                self.S_efe[i][j].w: S_efe_w[i][j][:], 
                            })
                    for i in range(len(self.S_efe_TopDown)):
                        for j in range(len(self.S_efe_TopDown[i])):
                            arguments.update({
                                self.S_efe_TopDown[i][j].w: S_efe_TopDown_w[i][j][:], 
                            })
                    for ih in range(len(self.S_e2e_exci)):
                        arguments.update({
                            self.S_e2e_exci[ih].w: S_e2e_exci_w[ih][:], 
                        })
                    for ih in range(len(self.S_e2e_exci_TopDown)):
                        arguments.update({
                            self.S_e2e_exci_TopDown[ih].w: S_e2e_exci_TopDown_w[ih][:], 
                        })
                        
            brian2.device.run(run_args=arguments, with_output=False) # process the sample
                            
            for ih in range(self.N_hidden):
                vt[ih] = self.exci_neurons[ih].vt[:]
                    
                
        # delete large arraies from the last stimulus to avoid too many files when running last number of samples
        brian2.device.delete(code=False, directory=False)

Please see the attachment in following rely for the list of files in “static_array”.

2

Here I attached the complete list of the static_array files:
static_arrays.txt (642.5 KB)

3

Hi @DavidKing2020. Just to make sure since this is the behaviour is expected for older Brian versions: you are running this with Brian 2.8.0 (or directly from GitHub master)?

4

I mean you mention 2.8 in your subject line, but please double-check :blush:

5

Hi @mstimberg , I indeed use brian2.8.

 Python 3.11.9 | packaged by conda-forge | (main, Apr 19 2024, 18:36:13) [GCC 12.3.0] on linux
 Type "help", "copyright", "credits" or "license" for more information.
 >>> import brian2
 >>> brian2.__version__
 '2.8.0'
>>>
6

Thanks for confirming.

I know that your code is not complete, but in the snippet you posted, the device.delete call is outside the while switch_stimulus == False loop, which means that if this loop is executed repeatedly then the delete call will only delete the files from the last iteration. Maybe this explains why you have init_....dat files that won’t get deleted?

Also note that the .lock files are never deleted, so their number will still increase. This is mostly due to the fileflock library we are using and should hopefully not matter much, since these files are empty. But if they bother you: these .lock files can always be safely deleted when no simulation is running.

1 Like
7

Thanks @mstimberg . This should be the case. My constraint is that there’s a limit on the number of files (2M) allowed to be stored on the disk and lock file dominate the list in this case. In addition to moving the delete into the while loop, is there a way to manually delete the corresponding lock file from the finished stimulus?

8

I see, we should look into this. We don’t have anything built-in to delete lock files, but you can use standard Python tools, e.g. something like

from pathlib import Path
...
for p in (Path(brian2.device.project_dir)/"static_arrays").glob("*.lock"):
    p.unlink()
9

Thanks. This should solve the problem

1 Like