Problem with Cython

adam · 24 August 2021 19:47

I’m also working on a mac running BigSur at the moment, and get Cython related errors when i try
brian2.test()
under the section
Running tests for target cython
I get about 90% F as in failed.

I tried marcel’s suggestion of uninstalling and installing the Brian2==2.4.2.post0.dev149 development version. Unfortunately, this didn’t fix the cython issue for me, and seemed to generate a single extra error under the numpy codegen section. This is what the error summary looked like:

**226 failed** , 24 passed, 8 skipped, 414 deselected in 380.99s (0:06:20)
ERROR: 2/4 test suite(s) did not complete successfully (see above).

let me know if there’s any other test outputs that would be useful and whether it would be useful for me to file this as an issue on github

mstimberg · 6 September 2021 13:22

Hi @SIbanez , hi @adam ,
it would be great if we could fix this issue before the release…
Since I wrote my comment, there have been various other fixes, so just to make sure that it isn’t fixed already, could you please update to the latest package from the test server:

pip install --upgrade --pre -i https://test.pypi.org/simple/ brian2

Then run the test suite again. To make this quicker (if Cython compilation fails, a lot of tests will fail), you can run:

>>> import brian2
>>> brian2.test('cython', test_codegen_independent=False, additional_args=['-x'])

This should stop after the first error. Please post the full error output here. Also, are you using a Conda environment? And if yes, do you have clang installed via conda packages (e.g. check with conda list| grep clang)?

Best,
Marcel

adam · 6 September 2021 14:12

Hi Marcel.

I created a new conda environment, and verified it has clang and did not have any existing brain2 installations. Then i ran the most recent pip installl command you posted.
Looks like it found the following version to install
Successfully installed brian2-2.4.2.post0.dev317
(adding a quick screenshot to make sure i didn’t miss anything)

I then opened python in the terminal and ran the cython test command and it failed. Some snippets of the error include:

clang-10: warning: -Wl,-export_dynamic: 'linker' input unused [-Wunused-command-line-argument]

 error: "Something's broken.  UCHAR_MAX should be defined in limits.h."

"Python's source code assumes C's unsigned char is an 8-bit type."

"Python.h requires that stdio.h define NULL."

error: unknown type name 'ldiv_t'

error: no member named 'acosf' in the global namespace; did you mean 'acos'?

but i’ll post the full error message in the next post

adam · 6 September 2021 14:16

The error log is longer than the character limit here, so i’ve uploaded it as a .txt
cython_error_awillats.txt (71.4 KB)

tail end of error log...

--------------------------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------------------------
clang-10: warning: -Wl,-export_dynamic: 'linker' input unused [-Wunused-command-line-argument]
clang-10: warning: -Wl,-export_dynamic: 'linker' input unused [-Wunused-command-line-argument]
clang-10: warning: -Wl,-export_dynamic: 'linker' input unused [-Wunused-command-line-argument]
clang-10: warning: -Wl,-export_dynamic: 'linker' input unused [-Wunused-command-line-argument]
clang-10: warning: -Wl,-export_dynamic: 'linker' input unused [-Wunused-command-line-argument]
clang-10: warning: -Wl,-export_dynamic: 'linker' input unused [-Wunused-command-line-argument]
In file included from /Users/adam/Library/Caches/cython/brian_extensions/_cython_magic_ab96af5f433b846928be415abd688381.cpp:46:
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/include/python3.8/Python.h:14:2: error: "Something's broken.  UCHAR_MAX should be defined in limits.h."
#error "Something's broken.  UCHAR_MAX should be defined in limits.h."
 ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/include/python3.8/Python.h:18:2: error: "Python's source code assumes C's unsigned char is an 8-bit type."
#error "Python's source code assumes C's unsigned char is an 8-bit type."
 ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/include/python3.8/Python.h:27:5: error: "Python.h requires that stdio.h define NULL."
#   error "Python.h requires that stdio.h define NULL."
    ^
In file included from /Users/adam/Library/Caches/cython/brian_extensions/_cython_magic_ab96af5f433b846928be415abd688381.cpp:46:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/include/python3.8/Python.h:30:
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/string.h:73:64: error: use of undeclared identifier 'strchr'
char* __libcpp_strchr(const char* __s, int __c) {return (char*)strchr(__s, __c);}
                                                               ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/string.h:80:75: error: use of undeclared identifier 'strpbrk'
char* __libcpp_strpbrk(const char* __s1, const char* __s2) {return (char*)strpbrk(__s1, __s2);}
                                                                          ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/string.h:87:65: error: use of undeclared identifier 'strrchr'; did you mean 'strchr'?
char* __libcpp_strrchr(const char* __s, int __c) {return (char*)strrchr(__s, __c);}
                                                                ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/string.h:75:13: note: 'strchr' declared here
const char* strchr(const char* __s, int __c) {return __libcpp_strchr(__s, __c);}
            ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/string.h:94:49: error: unknown type name 'size_t'
void* __libcpp_memchr(const void* __s, int __c, size_t __n) {return (void*)memchr(__s, __c, __n);}
                                                ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/string.h:96:46: error: unknown type name 'size_t'
const void* memchr(const void* __s, int __c, size_t __n) {return __libcpp_memchr(__s, __c, __n);}
                                             ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/string.h:98:46: error: unknown type name 'size_t'
      void* memchr(      void* __s, int __c, size_t __n) {return __libcpp_memchr(__s, __c, __n);}
                                             ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/string.h:101:74: error: use of undeclared identifier 'strstr'; did you mean 'strchr'?
char* __libcpp_strstr(const char* __s1, const char* __s2) {return (char*)strstr(__s1, __s2);}
                                                                         ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/string.h:77:13: note: 'strchr' declared here
      char* strchr(      char* __s, int __c) {return __libcpp_strchr(__s, __c);}
            ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/string.h:101:74: error: no matching function for call to 'strchr'
char* __libcpp_strstr(const char* __s1, const char* __s2) {return (char*)strstr(__s1, __s2);}
                                                                         ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/string.h:77:13: note: candidate disabled: <no message provided>
      char* strchr(      char* __s, int __c) {return __libcpp_strchr(__s, __c);}
            ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/string.h:101:81: error: cannot initialize a parameter of type 'char *' with an lvalue of type 'const char *'
char* __libcpp_strstr(const char* __s1, const char* __s2) {return (char*)strstr(__s1, __s2);}
                                                                                ^~~~
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/string.h:77:32: note: passing argument to parameter '__s' here
      char* strchr(      char* __s, int __c) {return __libcpp_strchr(__s, __c);}
                               ^
In file included from /Users/adam/Library/Caches/cython/brian_extensions/_cython_magic_ab96af5f433b846928be415abd688381.cpp:46:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/include/python3.8/Python.h:34:
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/stdlib.h:142:34: error: unknown type name 'ldiv_t'
inline _LIBCPP_INLINE_VISIBILITY ldiv_t div(long __x, long __y) _NOEXCEPT {
                                 ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/stdlib.h:143:12: error: no member named 'ldiv' in the global namespace
  return ::ldiv(__x, __y);
         ~~^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/stdlib.h:146:34: error: unknown type name 'lldiv_t'
inline _LIBCPP_INLINE_VISIBILITY lldiv_t div(long long __x,
                                 ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/stdlib.h:148:12: error: no member named 'lldiv' in the global namespace
  return ::lldiv(__x, __y);
         ~~^
In file included from /Users/adam/Library/Caches/cython/brian_extensions/_cython_magic_ab96af5f433b846928be415abd688381.cpp:46:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/include/python3.8/Python.h:63:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/include/python3.8/pyport.h:212:
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/math.h:771:93: error: no member named 'acosf' in the global namespace; did you mean 'acos'?
inline _LIBCPP_INLINE_VISIBILITY float       acos(float __lcpp_x) _NOEXCEPT       {return ::acosf(__lcpp_x);}
                                                                                          ~~^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/math.h:771:46: note: 'acos' declared here
inline _LIBCPP_INLINE_VISIBILITY float       acos(float __lcpp_x) _NOEXCEPT       {return ::acosf(__lcpp_x);}
                                             ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/math.h:772:93: error: no member named 'acosl' in the global namespace; did you mean 'acos'?
inline _LIBCPP_INLINE_VISIBILITY long double acos(long double __lcpp_x) _NOEXCEPT {return ::acosl(__lcpp_x);}
                                                                                          ~~^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/math.h:771:46: note: 'acos' declared here
inline _LIBCPP_INLINE_VISIBILITY float       acos(float __lcpp_x) _NOEXCEPT       {return ::acosf(__lcpp_x);}
                                             ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/math.h:778:38: error: call to 'acos' is ambiguous
acos(_A1 __lcpp_x) _NOEXCEPT {return ::acos((double)__lcpp_x);}
                                     ^~~~~~
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/math.h:771:46: note: candidate function
inline _LIBCPP_INLINE_VISIBILITY float       acos(float __lcpp_x) _NOEXCEPT       {return ::acosf(__lcpp_x);}
                                             ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/math.h:772:46: note: candidate function
inline _LIBCPP_INLINE_VISIBILITY long double acos(long double __lcpp_x) _NOEXCEPT {return ::acosl(__lcpp_x);}
                                             ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
=========================================================================================== short test summary info ============================================================================================
FAILED ../../opt/anaconda3/envs/neuroenv_braindev/lib/python3.8/site-packages/brian2/tests/test_base.py::test_active_flag - brian2.core.base.BrianObjectException: Error encountered with object named "neuro...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
1 failed, 414 deselected in 5.85s
ERROR: 1/1 test suite(s) did not complete successfully (see above).
False

mstimberg · 6 September 2021 14:39

Many thanks for the info. I did some googling, it might be not so much about Brian but due to the use of clang from conda. Could you please try the following:

Create a simple C++ test file, e.g.:

#include <iostream>
int main()
{
    std::cout << "Hello, world!\n";
}

Try to compile this file from within the activated conda environment, using $CXX testfile.cpp (the CXX variable should be set when the environment is activated).
If this fails with an error message about missing iostream, could you run $CXX --gcc-toolchain=$CONDA_PREFIX testfile.cpp and see whether this works?

adam · 6 September 2021 14:51

interesting.
g++ testfile.cpp compiles just fine, the executable works as expected!

if I echo $CXX i get x86_64-apple-darwin13.4.0-clang++

but $CXX testfile.cpp fails with several errors:

**error:** **use of undeclared identifier 'wcschr'**
[...]

but nothing explicitly mentioning iostream as far as I can see

I also tried $CXX --gcc-toolchain=$CONDA_PREFIX testfile.cpp but I got a clang warning that it didn’t use the gcc-toolchain argument.

I’m going to dig around stackoverflow to see if there’s something else I can do to get clang working on mac

mstimberg · 6 September 2021 15:00

Yes, I think the problem is specific to the clang compiler. I got the suggestion about the gcc-toolchain argument from this stackoverflow question: c++ - clang++ fails to compile hello world - Stack Overflow
It is curious that your clang doesn’t use this argument, it is listed in the command line docs: Clang command line argument reference — Clang 13 documentation .

adam · 6 September 2021 15:16

odd.
yeah here’s the command: clang++ --gcc-toolchain=$CONDA_PREFIX testfile.cpp

and error msg ...

clang-10: warning: argument unused during compilation: '--gcc-toolchain=/Users/adam/opt/anaconda3/envs/neuroenv_braindev' [-Wunused-command-line-argument]
In file included from testfile.cpp:1:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/iostream:37:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/ios:214:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/iosfwd:95:
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/wchar.h:137:77: error: use of undeclared identifier 'wcschr'
wchar_t* __libcpp_wcschr(const wchar_t* __s, wchar_t __c) {return (wchar_t*)wcschr(__s, __c);}
                                                                            ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/wchar.h:144:87: error: use of undeclared identifier 'wcspbrk'
wchar_t* __libcpp_wcspbrk(const wchar_t* __s1, const wchar_t* __s2) {return (wchar_t*)wcspbrk(__s1, __s2);}
                                                                                      ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/wchar.h:151:78: error: use of undeclared identifier 'wcsrchr'; did you mean 'wcschr'?
wchar_t* __libcpp_wcsrchr(const wchar_t* __s, wchar_t __c) {return (wchar_t*)wcsrchr(__s, __c);}
                                                                             ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/wchar.h:139:16: note: 'wcschr' declared here
const wchar_t* wcschr(const wchar_t* __s, wchar_t __c) {return __libcpp_wcschr(__s, __c);}
               ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/wchar.h:158:86: error: use of undeclared identifier 'wcsstr'; did you mean 'wcschr'?
wchar_t* __libcpp_wcsstr(const wchar_t* __s1, const wchar_t* __s2) {return (wchar_t*)wcsstr(__s1, __s2);}
                                                                                     ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/wchar.h:141:16: note: 'wcschr' declared here
      wchar_t* wcschr(      wchar_t* __s, wchar_t __c) {return __libcpp_wcschr(__s, __c);}
               ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/wchar.h:158:86: error: no matching function for call to 'wcschr'
wchar_t* __libcpp_wcsstr(const wchar_t* __s1, const wchar_t* __s2) {return (wchar_t*)wcsstr(__s1, __s2);}
                                                                                     ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/wchar.h:141:16: note: candidate disabled: <no message provided>
      wchar_t* wcschr(      wchar_t* __s, wchar_t __c) {return __libcpp_wcschr(__s, __c);}
               ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/wchar.h:158:93: error: cannot initialize a parameter of type 'wchar_t *' with an lvalue
      of type 'const wchar_t *'
wchar_t* __libcpp_wcsstr(const wchar_t* __s1, const wchar_t* __s2) {return (wchar_t*)wcsstr(__s1, __s2);}
                                                                                            ^~~~
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/wchar.h:141:38: note: passing argument to parameter '__s' here
      wchar_t* wcschr(      wchar_t* __s, wchar_t __c) {return __libcpp_wcschr(__s, __c);}
                                     ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/wchar.h:165:60: error: unknown type name 'size_t'
wchar_t* __libcpp_wmemchr(const wchar_t* __s, wchar_t __c, size_t __n) {return (wchar_t*)wmemchr(__s, __c, __n);}
                                                           ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/wchar.h:167:57: error: unknown type name 'size_t'
const wchar_t* wmemchr(const wchar_t* __s, wchar_t __c, size_t __n) {return __libcpp_wmemchr(__s, __c, __n);}
                                                        ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/wchar.h:169:57: error: unknown type name 'size_t'
      wchar_t* wmemchr(      wchar_t* __s, wchar_t __c, size_t __n) {return __libcpp_wmemchr(__s, __c, __n);}
                                                        ^
In file included from testfile.cpp:1:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/iostream:37:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/ios:214:
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/iosfwd:219:14: error: use of undeclared identifier 'mbstate_t'
typedef fpos<mbstate_t>    streampos;
             ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/iosfwd:220:14: error: use of undeclared identifier 'mbstate_t'
typedef fpos<mbstate_t>    wstreampos;
             ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/iosfwd:225:14: error: use of undeclared identifier 'mbstate_t'
typedef fpos<mbstate_t>    u16streampos;
             ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/iosfwd:226:14: error: use of undeclared identifier 'mbstate_t'
typedef fpos<mbstate_t>    u32streampos;
             ^
In file included from testfile.cpp:1:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/iostream:37:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/ios:215:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/__locale:15:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/string:511:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/string_view:179:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/__string:57:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/algorithm:649:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/initializer_list:46:
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/cstddef:49:9: error: no member named 'ptrdiff_t' in the global namespace
using ::ptrdiff_t;
      ~~^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/cstddef:50:9: error: no member named 'size_t' in the global namespace
using ::size_t;
      ~~^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/cstddef:53:9: error: no member named 'max_align_t' in the global namespace
using ::max_align_t;
      ~~^
In file included from testfile.cpp:1:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/iostream:37:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/ios:215:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/__locale:15:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/string:511:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/string_view:179:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/__string:57:
In file included from /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/algorithm:649:
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/initializer_list:61:5: error: unknown type name 'size_t'
    size_t    __size_;
    ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/initializer_list:65:38: error: unknown type name 'size_t'
    initializer_list(const _Ep* __b, size_t __s) _NOEXCEPT
                                     ^
/Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin/../include/c++/v1/initializer_list:73:13: error: unknown type name 'size_t'
    typedef size_t    size_type;
            ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.

perhaps this is a silly suggestion, I don’t know much about compilers, but given that g++ works (at least for hello world) is there any way to get Brian2 to use g++ rather than clang++?
Would overriding CXX (something like export CXX=g++) make sense?

adam · 6 September 2021 15:24

Hmm, some more potentially useful information:

g++ --version

Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Apple clang version 12.0.5 (clang-1205.0.22.11)
Target: x86_64-apple-darwin20.4.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

clang++ --version

clang version 10.0.0 
Target: x86_64-apple-darwin20.4.0
Thread model: posix
InstalledDir: /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin

$CXX --version

clang version 10.0.0 
Target: x86_64-apple-darwin13.4.0
Thread model: posix
InstalledDir: /Users/adam/opt/anaconda3/envs/neuroenv_braindev/bin

Notably g++, which was successful is using a more up to date version of clang installed in a conda-independent location (/Library/Developer/CommandLineTools/usr/bin).

To me that suggests I need to reinstall / update the clang that gets used within the conda environment (?)

mstimberg · 6 September 2021 15:29

Hmm, yes your g++ is actually just an alias for clang-12… It might be that the clang version that Anaconda provides is simply too old. There’s a newer version in conda-forge, so you could try something like conda update -c conda-forge clangxx and see where this gets you…

adam · 6 September 2021 15:37

neither
conda update -c conda-forge clangxx
nor
conda install -c anaconda clangxx
worked. The first returned “all requested packages already installed”, and the second installed clangxx-10.0.0

with the following message:

The following packages will be SUPERSEDED by a higher-priority channel:

  ca-certificates    conda-forge::ca-certificates-2021.5.3~ --> anaconda::ca-certificates-2020.10.14-0
  certifi            conda-forge::certifi-2021.5.30-py38h5~ --> anaconda::certifi-2020.6.20-py38_0
  clangxx                                         pkgs/main --> anaconda

looking here:
https://anaconda.org/anaconda/clangxx
it does look like the latest clang available through conda -c anaconda for mac is 10.0 but the -c conda-forge should have 12.0.1 available …

adam · 6 September 2021 15:42

going to try
conda install -c conda-forge clangxx=12.0.1
but my suspicion is that maybe some other dependencies of clangxx or things which depend on clangxx are holding back the version (?)

mstimberg · 6 September 2021 15:45

Not sure, these things can unfortunately quickly become a mess… BTW, I didn’t reply to this point earlier, but setting CXX to your global gxx (i.e. clang-12) should work with Brian, we let Python’s standard mechanisms search for the compiler, and it should take into account the CXX variable.

adam · 6 September 2021 16:59

I was able to set export CXX='g++' for the conda environment using the approach outlined here
https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#macos-and-linux

that means $CXX testfile.cpp works (--gcc-toolchain still gets ignored), but the cython test within brian still fails. Are there any other pieces i’d need to modify to get brian/Cython to use the new CXX=g++ ?

(conda is taking a very long time to try to install clangxx=12.0.1. if that doesn’t work i’ll probably start another environment from scratch and dig deeper into how conda installs clang)

mstimberg · 6 September 2021 17:04

I’ll try to look into this in more detail tomorrow, there might be some other variables that are checked first (e.g. CPP or GCC, …) – maybe try export | grep clang to see which ones are still set to conda’s clang?

adam · 6 September 2021 18:33

Minor breakthrough
using export | grep clang I was able to identify some other environment variables.

CC=x86_64-apple-darwin13.4.0-clang
CLANG=/Users/adam/opt/anaconda3/envs/neuroenv/bin/x86_64-apple-darwin13.4.0-clang
CLANGXX=/Users/adam/opt/anaconda3/envs/neuroenv/bin/x86_64-apple-darwin13.4.0-clang++
CXX=x86_64-apple-darwin13.4.0-clang++

So i updated $CONDA_PREFIX/etc/conda/activate.d/env_vars.sh to replace these with gcc and g++ and now the brian2 tests (include Cython related ones) pass!

successful test report

 brian2.test()
Running tests in /Users/adam/opt/anaconda3/envs/neuroenv_braindev/lib/python3.8/site-packages/brian2 for targets numpy, cython (excluding long tests)
Running Brian version 2.4.2.post0.dev317 from '/Users/adam/opt/anaconda3/envs/neuroenv_braindev/lib/python3.8/site-packages/brian2'
Testing codegen-independent code
Resetting to default preferences

Running doctests
..........................................                                                                                                                                          [100%]
42 passed in 23.06s
Running tests that do not use code generation
................................................................................................................................................................................... [ 49%]
................................................................................................................................................................................... [ 98%]
......                                                                                                                                                                              [100%]
364 passed, 321 deselected in 238.90s (0:03:58)
Running tests for target numpy:
....s...............................ssssss......................................................................................................................................... [ 66%]
............................................................................................                                                                                        [100%]
264 passed, 7 skipped, 414 deselected in 329.04s (0:05:29)
Running tests for target cython:
....s..............................s......s...s......s...............................................s...........s................................................................. [ 66%]
..............................................................ss............................                                                                                        [100%]
262 passed, 9 skipped, 414 deselected in 316.08s (0:05:16)
OK: 4/4 test suite(s) did complete successfully.
True

I’ve written a quick markdown guide that retraces the steps I used:

override_CXX_in_conda_env.md

i’m going to go back through and see

whether setting CC, CLANG were necessary
whether this means I can now use cython as a codegen target within brian

adam · 6 September 2021 20:15

update:
setting CC='gcc' and CXX='g++' seem to be sufficient (looks like modifying CLANG and CLANGXX isn’t necessary for the basic tests to pass) but i’ve left these in my guide for now.

I did a quick benchmark using one of the intro tutorial scripts and was able to demonstrate a speedup now that I have cython working!

SIbanez · 8 September 2021 12:18

Hello Marcel and Adam,
I have reproduced what Adam did and all the tests are running fine for me too, for the Brian version 2.4.2.post0.dev323 (after using
pip install --upgrade --pre -i Simple index brian2
as Marcel suggested).
Thank you SO much for this help!
Sara

Tioz90 · 25 October 2021 07:15

I was also having the same issue on MacOS 12.0.1 and brian 2.4.2 but your solution seems to have worked!

mstimberg · 5 November 2021 09:36

Just for the record: We just merged Helpful compile error on Unix by mstimberg · Pull Request #1367 · brian-team/brian2 · GitHub and while it does not directly fix the issue, at least now the “Cannot use Cython…” warning also includes the following note:

Certain compiler configurations (e.g. clang in a conda environment on OS X) are known to be problematic. Note that you can switch the compiler by setting the ‘CC’ and ‘CXX’ environment variables. For example, you may want to try ‘CC=gcc’ and ‘CXX=g++’.