The Cython code generator fails on gcc 12.2.0 but works flawlessly on 10.5.0

Support
1

Description of problem

Pretty complex script which was run a few million times failed after update.

Minimal code to reproduce problem

have tried to localize the problem but it seems related to c/c++ compilation

What you have aready tried

I switched back to gcc 10.5.0 and the problem went away. The problem seems related to OpenMP.

Full traceback of error (if relevant)

WARNING    Removing unsupported flag '-O3' from compiler flags. [brian2.codegen.cpp_prefs]
gcc: error trying to exec 'cc1plus': execvp: No such file or directory
WARNING    Removing unsupported flag '-ffast-math' from compiler flags. [brian2.codegen.cpp_prefs]
gcc: error trying to exec 'cc1plus': execvp: No such file or directory
WARNING    Removing unsupported flag '-fno-finite-math-only' from compiler flags. [brian2.codegen.cpp_prefs]
gcc: error trying to exec 'cc1plus': execvp: No such file or directory
WARNING    Removing unsupported flag '-march=native' from compiler flags. [brian2.codegen.cpp_prefs]
gcc: error trying to exec 'cc1plus': execvp: No such file or directory
WARNING    Removing unsupported flag '-std=c++11' from compiler flags. [brian2.codegen.cpp_prefs]
gcc: error trying to exec 'cc1plus': execvp: No such file or directory
gcc: error trying to exec 'cc1plus': execvp: No such file or directory
WARNING    Removing unsupported flag '-w' from compiler flags. [brian2.codegen.cpp_prefs]
gcc: error trying to exec 'cc1plus': execvp: No such file or directory
( a few millions of these lines :)
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/SMHS/home/rath/.local/lib/python3.10/site-packages/setuptools/_distutils/unixccompiler.py", line 185, in _compile
    self.spawn(compiler_so + cc_args + [src, '-o', obj] + extra_postargs)
  File "/SMHS/home/rath/.local/lib/python3.10/site-packages/setuptools/_distutils/ccompiler.py", line 1041, in spawn
    spawn(cmd, dry_run=self.dry_run, **kwargs)
  File "/SMHS/home/rath/.local/lib/python3.10/site-packages/setuptools/_distutils/spawn.py", line 70, in spawn
    raise DistutilsExecError(
distutils.errors.DistutilsExecError: command '/usr/bin/gcc' failed with exit code 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/SMHS/home/rath/.local/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/lustre/groups/colonneselab/v03-SimpleThReCx-20230815/GAinterface.py", line 293, in evaluator
    mth = main(mth,logger=mainlogger)
  File "/lustre/groups/colonneselab/v03-SimpleThReCx-20230815/SimpleCxThRx.py", line 567, in main
    br2obj[con].connect('i==j')
  File "/SMHS/home/rath/.local/lib/python3.10/site-packages/brian2/core/base.py", line 335, in device_override_decorated_function
    return func(*args, **kwds)
  File "/SMHS/home/rath/.local/lib/python3.10/site-packages/brian2/synapses/synapses.py", line 1593, in connect
    self._add_synapses_generator(
  File "/SMHS/home/rath/.local/lib/python3.10/site-packages/brian2/synapses/synapses.py", line 2108, in _add_synapses_generator
    codeobj = create_runner_codeobj(
  File "/SMHS/home/rath/.local/lib/python3.10/site-packages/brian2/codegen/codeobject.py", line 471, in create_runner_codeobj
    return device.code_object(
  File "/SMHS/home/rath/.local/lib/python3.10/site-packages/brian2/devices/device.py", line 368, in code_object
    codeobj.compile()
  File "/SMHS/home/rath/.local/lib/python3.10/site-packages/brian2/codegen/codeobject.py", line 112, in compile
    self.compiled_code[block] = self.compile_block(block)
  File "/SMHS/home/rath/.local/lib/python3.10/site-packages/brian2/codegen/runtime/cython_rt/cython_rt.py", line 181, in compile_block
    return cython_extension_manager.create_extension(
  File "/SMHS/home/rath/.local/lib/python3.10/site-packages/brian2/codegen/runtime/cython_rt/extension_manager.py", line 143, in create_extension
    module = self._load_module(
 File "/SMHS/home/rath/.local/lib/python3.10/site-packages/brian2/codegen/runtime/cython_rt/extension_manager.py", line 314, in _load_module
    build_extension.run()
  File "/SMHS/home/rath/.local/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
    self.build_extensions()
  File "/SMHS/home/rath/.local/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
    self._build_extensions_serial()
  File "/SMHS/home/rath/.local/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
    self.build_extension(ext)
  File "/SMHS/home/rath/.local/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 548, in build_extension
    objects = self.compiler.compile(
  File "/SMHS/home/rath/.local/lib/python3.10/site-packages/setuptools/_distutils/ccompiler.py", line 600, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/SMHS/home/rath/.local/lib/python3.10/site-packages/setuptools/_distutils/unixccompiler.py", line 187, in _compile
    raise CompileError(msg)
distutils.errors.CompileError: command '/usr/bin/gcc' failed with exit code 1
"""
.....

``
2

Hi, thanks for the report. I am not sure whether this is actually an issue in Brian or Cython, or rather an installation issue with the newer gcc version. For example, are you sure you actually installed it with C++ support? If yes, does the cc1plus binary exist (should be in something like /usr/lib/gcc/x86_64-linux-gnu/12)?

I’ll try to check with a gcc 12 installation later, but happy to hear that you could fix it by downgrading at least.

It shouldn’t be OpenMP-related, since we don’t use OpenMP with Cython :slight_smile:

3

Hi @mstimberg , Wellcom back from looong vacation. Hope it was nice.

The answer is YES. I checked that both C and C++ are there and function. Also, I compiled python from the source and configure detects gc++ without any problem.

4

Thanks, yes, I had a nice (and long :wink: ) vacation!

This will be a bit tricky to debug, since it most likely depends on the details of your setup. Since the problem seems to be in setuptools’ compilation procedure, potentially an update of setuptools could fix the issue? How did you install gcc 12?

5

It may not be a bug, it seems. All of this happened when our HPC guys rebuilt the cluster. They switched from RH to RockyLinux and used cloud image for the node operating system. I later ran into a different problem with spikeinterface and this issue related to system libraries that are “lightweight” in (heavy) Rocky Linux. So I think the version of gcc 12 on the node just has some issues with the same libraries. We can close this thread and forget about this problem.

6

Thanks for the follow-up, let us know if you run into a similar issue again and I will try to investigate in more detail.