Opened 2 years ago

Closed 2 years ago

#30675 closed defect (fixed)

Follow up to #30596: missing symbol in linking on gentoo with gcc10

Reported by: François Bissey Owned by:
Priority: blocker Milestone: sage-9.2
Component: cython Keywords: bitset
Cc: Travis Scrimshaw Merged in:
Authors: François Bissey Reviewers: Jonathan Kliem
Report Upstream: N/A Work issues:
Branch: 2cf0ebe (Commits, GitHub, GitLab) Commit: 2cf0ebea650b1d6af55bd04f148a02d52146b299
Dependencies: Stopgaps:

Status badges

Description

As per https://trac.sagemath.org/ticket/30596#comment:4, #30596 leads to some missing symbols in linking on Gentoo with gcc-10.

The first consequence of it, is failing to build the documentation:

Traceback (most recent call last):
  File "sage_setup/docbuild/__main__.py", line 1, in <module>
    from sage_setup.docbuild import main
  File "/dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_8/sage_setup/docbuild/__init__.py", line 58, in <module>
    import sage.all
  File "/dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_8/build/lib/sage/all.py", line 129, in <module>
    from sage.data_structures.all import *
  File "/dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_8/build/lib/sage/data_structures/all.py", line 3, in <module>
    from .bitset import Bitset, FrozenBitset
ImportError: /dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_8/build/lib/sage/data_structures/bitset.cpython-38-x86_64-linux-gnu.so: undefined symbol: _bitset_issubset

Only 3 symbols from bitset_intrinsics.h are missing

ldd -r /dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_8/build/lib/sage/data_structures/bitset.cpython-38-x86_64-linux-gnu.so | grep _bit
undefined symbol: _bitset_issubset      (/dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_8/build/lib/sage/data_structures/bitset.cpython-38-x86_64-linux-gnu.so)
undefined symbol: _bitset_isempty       (/dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_8/build/lib/sage/data_structures/bitset.cpython-38-x86_64-linux-gnu.so)
undefined symbol: _bitset_eq    (/dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_8/build/lib/sage/data_structures/bitset.cpython-38-x86_64-linux-gnu.so)

Declaring those three inline functions as static fix the problem. This probably implies that the inline keyword semantic from C99+ is involved.

Change History (17)

comment:1 Changed 2 years ago by François Bissey

Authors: François Bissey
Branch: u/fbissey/bitset_static
Cc: Travis Scrimshaw added
Commit: 7edd1b6fa9bf9d7cc95e012ae415ffa2d594a375
Dependencies: 30596#30596
Status: newneeds_review

@tscrim would you be so good as to check that it doesn't create problems for you?


New commits:

887e1f3outsource functions that can be optimized by intrinsics
bcfa7f1add file
49a95f3use same type declarations as in bitset.pxi
7edd1b6make problematic inline functions static.

comment:2 Changed 2 years ago by François Bissey

My changes are only the last commit of course.

comment:3 Changed 2 years ago by gh-kliem

Ok, I think I understand now.

This helped me:

http://gudok.xyz/inline/

It seems that there are three solutions to this:

  • declare the function as static, as you did
  • add a file bitset_intrinsics.c and declare the functions in there extern inline ...; I don't know how well this plays with cython
  • go for C++ and should be fine

comment:4 Changed 2 years ago by François Bissey

Yes, that would sum it up. And I think that option 2 would be problematic but I wouldn't know before trying for real. I hadn't thought about going full C++ (because I hate C++) but that would be a viable option.

comment:5 Changed 2 years ago by gh-kliem

But then all of the functions should be made static, shouldn't they.

This problem will also show up, when you compile a debug version, because then nothing will be inlined and nothing will work anymore.

comment:6 Changed 2 years ago by gh-kliem

At the moment, I don't see a need of going C++. It might arise at some point however, in particular because C++ supports templates. So far, I'm handling the templates via fused types in cython, that also works.

comment:7 in reply to:  5 Changed 2 years ago by François Bissey

Replying to gh-kliem:

But then all of the functions should be made static, shouldn't they.

This problem will also show up, when you compile a debug version, because then nothing will be inlined and nothing will work anymore.

I must say, I am still wondering why just those 3. But if we are both in agreement that they should all be made static then I'll do that. I hadn't thought of doing a debug version but yes it should be more revealing.

comment:8 Changed 2 years ago by git

Commit: 7edd1b6fa9bf9d7cc95e012ae415ffa2d594a375df4f462d4c5be00230d3f13e2c9d69b54c7abd91

Branch pushed to git repo; I updated commit sha1. New commits:

df4f462make all inline functions in bitset static.

comment:9 Changed 2 years ago by gh-kliem

Reviewers: Jonathan Kliem
Status: needs_reviewpositive_review

Ok. Thanks for fixing this.

comment:10 Changed 2 years ago by Volker Braun

Status: positive_reviewneeds_work

Merge conflict

comment:11 Changed 2 years ago by git

Commit: df4f462d4c5be00230d3f13e2c9d69b54c7abd912cf0ebea650b1d6af55bd04f148a02d52146b299

Branch pushed to git repo; I updated commit sha1. New commits:

2cf0ebeMerge branch 'develop' into bitset_static

comment:12 Changed 2 years ago by François Bissey

Dependencies: #30596
Status: needs_workpositive_review

Rebased and put back to positive review.

comment:13 Changed 2 years ago by Steven Trogdon

This issue also appears on Gentoo with gcc9. An incremental build of vanilla from 9.2.beta13 -> 9.2.beta14 without this branch gives

[dochtml] Traceback (most recent call last):
[dochtml]   File "/usr/lib/python3.8/runpy.py", line 185, in _run_module_as_main
[dochtml]     mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
[dochtml]   File "/usr/lib/python3.8/runpy.py", line 144, in _get_module_details
[dochtml]     return _get_module_details(pkg_main_name, error)
[dochtml]   File "/usr/lib/python3.8/runpy.py", line 111, in _get_module_details
[dochtml]     __import__(pkg_name)
[dochtml]   File "/local/sage-git/sage/local/lib/python3.8/site-packages/sage_setup/docbuild/__init__.py", line 58, in <module>
[dochtml]     import sage.all
[dochtml]   File "/local/sage-git/sage/local/lib/python3.8/site-packages/sage/all.py", line 129, in <module>
[dochtml]     from sage.data_structures.all import *
[dochtml]   File "/local/sage-git/sage/local/lib/python3.8/site-packages/sage/data_structures/all.py", line 3, in <module>
[dochtml]     from .bitset import Bitset, FrozenBitset
[dochtml] ImportError: /local/sage-git/sage/local/lib/python3.8/site-packages/sage/data_structures/bitset.cpython-38-x86_64-linux-gnu.so: undefined symbol: _bitset_intersection
make[3]: *** [Makefile:1877: doc-html] Error 1
make[2]: *** [Makefile:1767: all-start] Error 2

and

$ ldd -r /local/sage-git/sage/local/lib/python3.8/site-packages/sage/data_structures/bitset.cpython-38-x86_64-linux-gnu.so | grep _bit
undefined symbol: _bitset_intersection  (/local/sage-git/sage/local/lib/python3.8/site-packages/sage/data_structures/bitset.cpython-38-x86_64-linux-gnu.so)
undefined symbol: _bitset_first_in_limb_nonzero (/local/sage-git/sage/local/lib/python3.8/site-packages/sage/data_structures/bitset.cpython-38-x86_64-linux-gnu.so)
undefined symbol: _bitset_first_in_limb (/local/sage-git/sage/local/lib/python3.8/site-packages/sage/data_structures/bitset.cpython-38-x86_64-linux-gnu.so)
undefined symbol: _bitset_issubset      (/local/sage-git/sage/local/lib/python3.8/site-packages/sage/data_structures/bitset.cpython-38-x86_64-linux-gnu.so)
undefined symbol: _bitset_difference    (/local/sage-git/sage/local/lib/python3.8/site-packages/sage/data_structures/bitset.cpython-38-x86_64-linux-gnu.so)
undefined symbol: _bitset_isempty       (/local/sage-git/sage/local/lib/python3.8/site-packages/sage/data_structures/bitset.cpython-38-x86_64-linux-gnu.so)
undefined symbol: _bitset_eq    (/local/sage-git/sage/local/lib/python3.8/site-packages/sage/data_structures/bitset.cpython-38-x86_64-linux-gnu.so)
undefined symbol: _bitset_symmetric_difference  (/local/sage-git/sage/local/lib/python3.8/site-packages/sage/data_structures/bitset.cpython-38-x86_64-linux-gnu.so)
undefined symbol: _bitset_len   (/local/sage-git/sage/local/lib/python3.8/site-packages/sage/data_structures/bitset.cpython-38-x86_64-linux-gnu.so)
undefined symbol: _bitset_union (/local/sage-git/sage/local/lib/python3.8/site-packages/sage/data_structures/bitset.cpython-38-x86_64-linux-gnu.so)

comment:14 Changed 2 years ago by Steven Trogdon

This branch does allow the html-docs to build.

comment:15 Changed 2 years ago by Matthias Köppe

Priority: criticalblocker

comment:16 Changed 2 years ago by Dima Pasechnik

Keywords: bitset added

comment:17 Changed 2 years ago by Volker Braun

Branch: u/fbissey/bitset_static2cf0ebea650b1d6af55bd04f148a02d52146b299
Resolution: fixed
Status: positive_reviewclosed
Note: See TracTickets for help on using tickets.