#30534 closed enhancement (invalid)
Repackage pynac as a pip-installable package
Reported by: | mkoeppe | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | sage-duplicate/invalid/wontfix |
Component: | packages: standard | Keywords: | pynac, sd110, sd111 |
Cc: | rws, slelievre, gh-tobiasdiez, was, fbissey, tscrim, gh-kliem, vdelecroix, embray | Merged in: | |
Authors: | Reviewers: | François Bissey | |
Report Upstream: | Reported upstream. No feedback yet. | Work issues: | |
Branch: | Commit: | ||
Dependencies: | #30446, #31064 | Stopgaps: |
Description (last modified by )
Pynac has a compile-time dependency on the Python library but is not installed using Python package infrastructure. This is problematic because Python users cannot install it using standard Python tools - for example for testing different Python versions.
It would make sense for src/sage/libs/pynac/pynac.pxd
and pynac_wrap.h
to be shipped with this Python package, introducing a dependency on Cython. (The only thing blocking this is from sage.libs.gmp.types cimport mpz_t, mpq_t, mpz_ptr, mpq_ptr
- but that can be fixed by some minimal cut&paste.)
(The other parts of src/sage/libs/pynac
, constant.pxd
, constant.pyx
, pynac.pyx
use various imports from sage and must remain in sage.libs.pynac
)
Unfortunately, there is an unrelated project that has claimed the name "Pynac" on pypi.org: https://pypi.org/project/Pynac
Attachments (1)
Change History (77)
comment:1 Changed 2 years ago by
Cc: | rws slelievre added |
---|---|
Description: | modified (diff) |
Keywords: | pynac added |
comment:2 Changed 2 years ago by
Description: | modified (diff) |
---|
comment:3 Changed 2 years ago by
Description: | modified (diff) |
---|
comment:4 Changed 2 years ago by
Unfortunately, there is an unrelated project that has claimed the name "Pynac" on pypi.org:
Collision between
- Pynac = "Py-Dynac" = Python bindings for the Dynac particle tracking code
- Pynac = "Py-GiNaC" = Python derivative of the C++ library GiNaC
Possible solutions:
- rename these projects to Py-Dynac and Py-GiNaC, or Pydynac and Pyginac
- no renaming but use the name pyginac on PyPI for "Py-GiNaC"
Note:
- "Py-Dynac" got the pynac name on PyPI
- "Py-GiNaC" got the pynac.org domain name
Regardless of how we handle getting "Py-GiNaC" on PyPI, a disambiguation note in both "Pynac" projects would be nice, with links to each other.
I will propose changes via pull requests or other to add such a note in relevant places.
comment:6 Changed 2 years ago by
Cc: | gh-tobiasdiez added |
---|
comment:7 Changed 2 years ago by
Cc: | was added |
---|---|
Keywords: | sd110 added |
We briefly discussed it at sd110.
From William Stein (CoCalc) to Everyone: (1:37 PM)
py-ginac is pretty good.
comment:8 Changed 2 years ago by
Cc: | fbissey added |
---|
comment:10 Changed 2 years ago by
Replying to chapoton:
There is already some
py_ginac
somewhere :
Thanks for the pointer. This appears to be an active project. From https://sourceforge.net/projects/pyginac.moebinv.p/:
"This is a refresh and maintained version of the previous stalled project https://sourceforge.net/projects/pyginac/ It works with the current version of GiNaC."
The source (git) is at https://sourceforge.net/p/moebinv/pyginac/code/ci/master/tree/
It provides the Python packages cginac
and ginac
.
comment:11 Changed 2 years ago by
It calls itself "pyGiNaC" but it does not define a distribution of this name. It must be built using scons and is not pip-installable - so it also does not claim a project name on PyPI.
comment:12 Changed 2 years ago by
Report Upstream: | N/A → Reported upstream. No feedback yet. |
---|
comment:13 Changed 2 years ago by
Keywords: | sd111 added |
---|
comment:14 Changed 2 years ago by
Milestone: | sage-9.3 → sage-9.4 |
---|
Setting new milestone based on a cursory review of ticket status, priority, and last modification date.
comment:16 Changed 2 years ago by
Dependencies: | → #30446 |
---|
comment:17 Changed 2 years ago by
Branch: | → u/mkoeppe/repackage_pynac_as_a_pip_installable_package |
---|
comment:18 Changed 2 years ago by
Commit: | → 44e4dd88ade4ef49f9dde054b3d75f4b57a4956a |
---|
Branch pushed to git repo; I updated commit sha1. New commits:
44e4dd8 | Fixup
|
comment:19 Changed 2 years ago by
Authors: | → Matthias Koeppe, ... |
---|
The new package installs correctly. sage.libs.pynac
needs more work.
comment:20 Changed 2 years ago by
Cc: | tscrim added |
---|
comment:21 Changed 2 years ago by
Cc: | gh-kliem vdelecroix added |
---|
I think I'll need some help here from people who know about import
and cimport
in Cython...
comment:23 Changed 2 years ago by
I am getting runtime linker errors like the following on this branch:
[dochtml] import sage.libs.pynac.pynac # initialize pynac before .ring [dochtml] ImportError: dlopen(/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-darwin.so, 2): Symbol not found: __ZN5GiNaC12library_initD1Ev [dochtml] Referenced from: /Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-darwin.so [dochtml] Expected in: flat namespace [dochtml] in /Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-darwin.so
comment:24 Changed 2 years ago by
Cc: | embray added |
---|
That is beyond my knowledge as it has something to do with how the packages are linked. Perhaps looking at how p_group_cohomology does its setup might have a clue?
Erik, do you know about these things or know a good person to ask?
comment:25 Changed 2 years ago by
The double leading underscore is very strange. If you remove one it becomes simply
$ c++filt -n _ZN5GiNaC12library_initD1Ev GiNaC::library_init::~library_init()
but with two, it is nothing. Some naming or interface is not set right.
comment:26 follow-up: 45 Changed 2 years ago by
First of all let me remark that I wasn't even able to build it at all until I manually removed all files in build/pkgs/sagelib/src/build/cythonized/sage/libs/pynac/
.
Until then there was still the old pynac_wrap.h
dangling around and it did not clean itself.
comment:27 Changed 2 years ago by
And something isn't right about pynac/setup.py
:
(sage-sh) kliem@cofio:sage$ python Python 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from ginac.pynac import * Traceback (most recent call last): File "<stdin>", line 1, in <module> ModuleNotFoundError: No module named 'ginac.pynac' >>> from ginac.libpynac import * Segmentation fault
comment:28 Changed 2 years ago by
I even get segementation fault in ipython when it tries to load the modules of ginac.
comment:29 follow-up: 38 Changed 2 years ago by
This part in numeric.cpp
leads to a segmentation fault when importing the module:
3321 #ifdef PYNAC_HAVE_LIBGIAC 3322 giac::gen* numeric::to_giacgen(giac::context* cptr) const 3323 { 3324 if (t == LONG) 3325 return new giac::gen(v._long); 3326 +---- 7 lines: if (t == MPZ) {------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3333 +---- 10 lines: if (t == MPQ) {------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3343 else 3344 return nullptr; 3345 } 3346 #endif
giac might not be linked correctly, but I don't know. Also ./bootstrap
is missing from setup.py
. Just running ./configure
isn't working.
comment:30 follow-up: 35 Changed 2 years ago by
Also I would guess that you need to include the header files as depends=...
.
What is definitely missing is gmptypes
. Also it seems weird that you name the module ginac.libpynac
but then you cimport from ginac.pynac import *
in pynac.pyx
. This isn't defined.
Anyway, if I know just ./bootstrap; make
pynac
and then include the folder ginac
, then I'm back to your orignal problem.
First I got:
ImportError: /home/mi/kliem/Applications/pynac/ginac/pynac.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZTVN5GiNaC9containerISt6vectorEE
This I made go away by declaring things static in pynac_wrap.h
. Then I get the original error about library_init
.
The class library_init
is defined in ex.h
. However, the methods are then defined in utils.cpp
.
But even if I change this, the error doesn't go ayay. By now, I don't understand anything anymore.
comment:31 Changed 2 years ago by
I noticed while building the package that giac is automagically detected and used. Current sage package explicitly disable it. I remember chatting with Ralph when he made it available and that it was currently broken. So, first thing will be to disable it.
comment:32 Changed 2 years ago by
See https://github.com/pynac/pynac/pull/372 "configure: Change default to --with-giac=no"
comment:33 Changed 2 years ago by
I pulled on your python_package branch, I guess I should have used the attached tarball.
comment:34 Changed 2 years ago by
I've just the branch into python_package
and will also prepare a new tarball
comment:35 Changed 2 years ago by
Replying to gh-kliem:
Also it seems weird that you name the module
ginac.libpynac
but then you cimportfrom ginac.pynac import *
inpynac.pyx
. This isn't defined.
Thanks, renamed now
comment:36 Changed 2 years ago by
Commit: | 44e4dd88ade4ef49f9dde054b3d75f4b57a4956a → 72da71ac5f262afae58655ceccc2da30954e2175 |
---|
comment:37 Changed 2 years ago by
OK, so pulling from your branch and building with my system packages and tools on gentoo with python setup.py install --prefix=$someprefix
I cannot see anything obviously wrong with what is installed. ldd -r pynac.cpython-39-x86_64-linux-gnu.so
just report what I am expecting to be missing, that is symbols from python itself.
I am not currently setup for compiling on OS X, which may have trickier issue at linking.
comment:38 Changed 2 years ago by
Replying to gh-kliem:
Also
./bootstrap
is missing fromsetup.py
. Just running./configure
isn't working.
I've now added some instructions to the README
comment:39 Changed 2 years ago by
https://github.com/kliem/pynac/tree/python_package
I did some experiments. Not all changes in this branch are necessary to get things to work. In particular, I did many experiments trying to avoid this segmentation fault, but I couldn't find something. It's ipython specific. Sage just complains about linkage.
As reported above I get a weird segmentation fault, whenever I import ginac.pynac
.
It goes away magically, if I import ginac.gmptypes
beforehand. I don't know why.
Getting sage to start still does not work, because
ImportError: /srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/libs/pynac/pynac.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN5GiNaC8py_funcsE
I don't know why it would be undefined. It is sage specific. In an Ipython session it is fine even if I add something that uses it to `ginac/pynac.pyx'
+cdef py_gcd(n, k):
+ return n*k
+
+
+def init_function_table():
+ py_funcs.py_gcd = &py_gcd
+
If I pip install this, I can run this function just fine in ipython:
(sage-sh) kliem@cofio:pynac$ ipython Python 3.7.3 (default, Jul 25 2020, 13:03:44) Type 'copyright', 'credits' or 'license' for more information IPython 7.13.0 -- An enhanced Interactive Python. Type '?' for help. In [1]: from ginac import gmptypes In [2]: from ginac.pynac import * In [3]: init_function_table()
What is even more strange about this segementation fault is that in sage on develop branch I can run import os
beforehand to fix this. This does not work in ipython.
comment:40 follow-up: 44 Changed 2 years ago by
Trying to build this branch from vanilla sage, pynac building fails miserably, unlike my simple build from from your git fork. I could import pynac.ginac when build from your git repo, but here I get
Error compiling Cython file: ------------------------------------------------------------ ... # (at your option) any later version. # http://www.gnu.org/licenses/ #***************************************************************************** from cpython cimport * from ginac.pynac cimport * ^ ------------------------------------------------------------ ginac/pynac.pyx:18:0: 'ginac/pynac.pxd' not found Error compiling Cython file: ------------------------------------------------------------ ... defined by GiNaC. This allows us to prevent collisions with C++ level special functions when a user asks to construct a symbolic function with the same name. """ global GINAC_FN_SERIAL GINAC_FN_SERIAL = g_registered_functions().size() ^ ------------------------------------------------------------ ginac/pynac.pyx:30:22: undeclared name not builtin: g_registered_functions Compiling ginac/pynac.pyx because it changed. [1/1] Cythonizing ginac/pynac.pyx
comment:41 Changed 2 years ago by
Commit: | 72da71ac5f262afae58655ceccc2da30954e2175 → f24d6466798713a884877b0fa4afb1c6b20d29fb |
---|
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
cea4cd5 | ci-cygwin-standard.yml: More stages, continue-on-error: true
|
cf31b79 | Fixup after cherry-pick
|
073124c | Merge branch 't/31084/makefile__add__ptest__targets_that_do_not_depend_on_the_docbuild' into t/31064/ci_cygwin__yml__adjust_to_new_script_packages__bootstrap___prereq
|
8769bd6 | .github/workflows/ci-cygwin-*.yml: Separate docbuild and ptest
|
3826222 | Fix up stage iv
|
598b0c2 | local-cygwin-choco: Do not pass through PATH, use full pathname of choco instead
|
d65299c | Merge branch 't/29124/script-packages-prereq-toolchain-bootstrap' into t/31064/ci_cygwin__yml__adjust_to_new_script_packages__bootstrap___prereq
|
a5e4051 | Merge branch 't/30944/tox__improve_local_sudo_ubuntu_standard' into t/31064/ci_cygwin__yml__adjust_to_new_script_packages__bootstrap___prereq
|
42f4458 | Merge tag '9.3.beta7' into t/31064/ci_cygwin__yml__adjust_to_new_script_packages__bootstrap___prereq
|
f24d646 | Merge #31064
|
comment:42 Changed 2 years ago by
Dependencies: | #30446 → #30446, #31064 |
---|
comment:43 Changed 2 years ago by
I have pushed a few more changes to the pynac branch and have set up testing against this ticket - running at https://github.com/mkoeppe/pynac/runs/2010476345
comment:44 follow-up: 46 Changed 2 years ago by
Yes, same error here. ginac/pynax.pdx
is missing from the configure file now.
After adding it, the build runs fine. Sage doens't start with the same stupid error:
ImportError: /srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/libs/pynac/pynac.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN5GiNaC8py_funcsE
The segmentation fault in ipython went away for an ImportError
:
ImportError: /srv/public/kliem/sage/local/lib/python3.7/site-packages/ginac/pynac.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN4giac25i_lex_is_strictly_greaterERKNS_7index_mES2_
Replying to fbissey:
Trying to build this branch from vanilla sage, pynac building fails miserably, unlike my simple build from from your git fork. I could import pynac.ginac when build from your git repo, but here I get
Error compiling Cython file: ------------------------------------------------------------ ... # (at your option) any later version. # http://www.gnu.org/licenses/ #***************************************************************************** from cpython cimport * from ginac.pynac cimport * ^ ------------------------------------------------------------ ginac/pynac.pyx:18:0: 'ginac/pynac.pxd' not found Error compiling Cython file: ------------------------------------------------------------ ... defined by GiNaC. This allows us to prevent collisions with C++ level special functions when a user asks to construct a symbolic function with the same name. """ global GINAC_FN_SERIAL GINAC_FN_SERIAL = g_registered_functions().size() ^ ------------------------------------------------------------ ginac/pynac.pyx:30:22: undeclared name not builtin: g_registered_functions Compiling ginac/pynac.pyx because it changed. [1/1] Cythonizing ginac/pynac.pyx
comment:45 follow-up: 47 Changed 2 years ago by
Replying to gh-kliem:
First of all let me remark that I wasn't even able to build it at all until I manually removed all files in
build/pkgs/sagelib/src/build/cythonized/sage/libs/pynac/
.Until then there was still the old
pynac_wrap.h
dangling around and it did not clean itself.
Has anyone experienced this? This seems to be a bug in our build system?
comment:46 follow-up: 52 Changed 2 years ago by
Replying to gh-kliem:
Yes, same error here.
ginac/pynax.pdx
is missing from the configure file now.
pynac.pxd
is generated by setup.py
from pynac.pxd.in
via the configure
script.
Installation of the new package can be seen to succeed in https://github.com/mkoeppe/pynac/runs/2010476345
comment:47 Changed 2 years ago by
Replying to gh-kliem:
Replying to gh-kliem:
First of all let me remark that I wasn't even able to build it at all until I manually removed all files in
build/pkgs/sagelib/src/build/cythonized/sage/libs/pynac/
.Until then there was still the old
pynac_wrap.h
dangling around and it did not clean itself.Has anyone experienced this? This seems to be a bug in our build system?
Yes, quite possible that the install cleaner only handles Python modules and extensions but not package data.
comment:48 Changed 2 years ago by
Commit: | f24d6466798713a884877b0fa4afb1c6b20d29fb → c0805f54143cab58d56e351c19cb5b28944f0148 |
---|
Branch pushed to git repo; I updated commit sha1. New commits:
c0805f5 | build/pkgs/pynac: Use 9c89d8bbc723456b2d94efb78ca1df1820694ff0
|
Changed 2 years ago by
Attachment: | py-ginac-0.7.27.tar.gz added |
---|
comment:50 Changed 2 years ago by
Commit: | c0805f54143cab58d56e351c19cb5b28944f0148 → 93fe11f753cff24f7d81f00b91ec393bcbfc1721 |
---|
Branch pushed to git repo; I updated commit sha1. New commits:
93fe11f | build/pkgs/pynac: Use 5701911df8f00c99bc7c7ed676360729200fb726
|
comment:52 Changed 2 years ago by
This is fixed with the new attachment. Before there was a version where ginac/pynac.pxd
was missing from configure for some reason.
Replying to mkoeppe:
Replying to gh-kliem:
Yes, same error here.
ginac/pynax.pdx
is missing from the configure file now.
pynac.pxd
is generated bysetup.py
frompynac.pxd.in
via theconfigure
script. Installation of the new package can be seen to succeed in https://github.com/mkoeppe/pynac/runs/2010476345
comment:53 Changed 2 years ago by
With the new branch I can now do from ginac.pynac import *
without an error raising in ipython
. The ImportError
when loading sage itself still persists.
comment:54 Changed 2 years ago by
So maybe this module needs to include something on runtime and this doesn't work. So from ginac.pynac import *
works fine, but the cimport
doesn't.
In my experiments, the os.path.dirname(__file__)
in the setup file didn't do anything. Maybe the include_dirs
are set up wrong?
comment:55 Changed 2 years ago by
OK, tried with all the new stuff. Stopped at the documentation again but different symbol than in comment:23
[dochtml] ImportError: /home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so: undefined symbol: _ZN5GiNaC8py_funcsE
which is
$ c++filt -n _ZN5GiNaC8py_funcsE GiNaC::py_funcs
comment:56 Changed 2 years ago by
Running ldd
, removing python calls and going through c++filt -n
the following symbols are undefined and should be replaced by calls since you cannot link to the python module
ldd -r /home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so | grep _Z | c++filt -n undefined symbol: GiNaC::py_funcs (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: vtable for GiNaC::container<std::vector> (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: typeinfo for GiNaC::numeric (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: typeinfo for GiNaC::symbol (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::library_init::~library_init() (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::container<std::vector>::tinfo_static (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: vtable for GiNaC::constant (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::I (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::_num0_bp (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: typeinfo for GiNaC::pseries (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: typeinfo for GiNaC::basic (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::function::function(unsigned int, GiNaC::ex const&, GiNaC::ex const&) (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::library_init::library_init() (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: ginac_pyinit_Float(_object*) (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::function::function(unsigned int, std::vector<GiNaC::ex, std::allocator<GiNaC::ex> > const&, bool) (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: ginac_pyinit_I(_object*) (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: std::vector<GiNaC::ex, std::allocator<GiNaC::ex> >::~vector() (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::relational::decide() const (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::function::function(unsigned int, GiNaC::ex const&, GiNaC::ex const&, GiNaC::ex const&) (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::ex::sorted_op(unsigned long) const (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::ex::construct_from_basic(GiNaC::basic const&) (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::pseries::is_terminating() const (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::function::function(unsigned int) (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::function::registered_functions() (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::ex::compare(GiNaC::ex const&) const (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::numeric::to_pyobject() const (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::relational::the_operator() const (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::function::function(unsigned int, GiNaC::ex const&) (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::ex::construct_from_int(int) (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::constant::constant() (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::basic::hold() const (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: ginac_pyinit_Integer(_object*) (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so) undefined symbol: GiNaC::basic::operator=(GiNaC::basic const&) (/home/fbissey/sandbox/git-fork/sage/local/lib/python3.9/site-packages/sage/libs/pynac/pynac.cpython-39-x86_64-linux-gnu.so)
comment:57 follow-up: 58 Changed 2 years ago by
Looks like trying to pull in everything to sage.libs.pynac
by from ginac.pynac cimport *
is a mistake and instead all modules that need pynac should cimport
more specifically from ginac.pynac
comment:58 Changed 2 years ago by
Replying to mkoeppe:
Looks like trying to pull in everything to
sage.libs.pynac
byfrom ginac.pynac cimport *
is a mistake and instead all modules that need pynac shouldcimport
more specifically fromginac.pynac
But why does
sage: cython(''' ....: from sage.libs.pynac.pynac cimport * ....: ''')
works just fine on develop? I would say that things should work just fine, once everything is set up correctly. However, I don't know how to resolve the problems as I'm currently not even able to cimport
from my own little two files project.
comment:59 Changed 2 years ago by
I might have found the problem:
In setup.py
:
- package_data = {'ginac': ['*.pxd', '*.h', '*.hpp']}, + package_data = {'ginac': ['*.pxd', '*.h', '*.hpp', '*.cpp']}, This will place the `.cpp` files in the correct directory. We need them, because when we cimport we recompile some of them. When the `.cpp` files are not there, we get the missing symbols. After doing this, I get a new error: {{{ [sagelib-9.3.beta7] g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-rpath-link,/srv/public/kliem/sage/local/lib -L/srv/public/kliem/sage/local/lib -Wl,-rpath,/srv/public/kliem/sage/local/lib -march=native -O3 -g -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.7/build/cythonized/sage/libs/pynac/constant.o -lflint -lgmp -lginac -o build/lib.linux-x86_64-3.7/sage/libs/pynac/constant.cpython-37m-x86_64-linux-gnu.so -L/srv/public/kliem/sage/local/lib -lfactory -lntl -lgmp -lomalloc -lsingular_resources [sagelib-9.3.beta7] /usr/bin/ld: cannot find -lginac [sagelib-9.3.beta7] collect2: error: ld returned 1 exit status [sagelib-9.3.beta7] g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-rpath-link,/srv/public/kliem/sage/local/lib -L/srv/public/kliem/sage/local/lib -Wl,-rpath,/srv/public/kliem/sage/local/lib -march=native -O3 -g -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.7/build/cythonized/sage/symbolic/ring.o -lflint -lgmp -lginac -o build/lib.linux-x86_64-3.7/sage/symbolic/ring.cpython-37m-x86_64-linux-gnu.so -L/srv/public/kliem/sage/local/lib -lfactory -lntl -lgmp -lomalloc -lsingular_resources [sagelib-9.3.beta7] /usr/bin/ld: cannot find -lginac [sagelib-9.3.beta7] collect2: error: ld returned 1 exit status [sagelib-9.3.beta7] g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-rpath-link,/srv/public/kliem/sage/local/lib -L/srv/public/kliem/sage/local/lib -Wl,-rpath,/srv/public/kliem/sage/local/lib -march=native -O3 -g -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.7/build/cythonized/sage/symbolic/function.o -lflint -lgmp -lginac -o build/lib.linux-x86_64-3.7/sage/symbolic/function.cpython-37m-x86_64-linux-gnu.so -L/srv/public/kliem/sage/local/lib -lfactory -lntl -lgmp -lomalloc -lsingular_resources [sagelib-9.3.beta7] /usr/bin/ld: cannot find -lginac [sagelib-9.3.beta7] collect2: error: ld returned 1 exit status [sagelib-9.3.beta7] g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-rpath-link,/srv/public/kliem/sage/local/lib -L/srv/public/kliem/sage/local/lib -Wl,-rpath,/srv/public/kliem/sage/local/lib -march=native -O3 -g -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.7/build/cythonized/sage/libs/pynac/pynac.o -L/srv/public/kliem/sage/local/lib -lgsl -lflint -lgmp -lm -lopenblas -lginac -o build/lib.linux-x86_64-3.7/sage/libs/pynac/pynac.cpython-37m-x86_64-linux-gnu.so -L/srv/public/kliem/sage/local/lib -lfactory -lntl -lgmp -lomalloc -lsingular_resources [sagelib-9.3.beta7] /usr/bin/ld: cannot find -lginac [sagelib-9.3.beta7] collect2: error: ld returned 1 exit status [sagelib-9.3.beta7] g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-rpath-link,/srv/public/kliem/sage/local/lib -L/srv/public/kliem/sage/local/lib -Wl,-rpath,/srv/public/kliem/sage/local/lib -march=native -O3 -g -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.7/build/cythonized/sage/symbolic/expression.o -lflint -lgmp -lginac -o build/lib.linux-x86_64-3.7/sage/symbolic/expression.cpython-37m-x86_64-linux-gnu.so -L/srv/public/kliem/sage/local/lib -lfactory -lntl -lgmp -lomalloc -lsingular_resources -lpari [sagelib-9.3.beta7] /usr/bin/ld: cannot find -lginac [sagelib-9.3.beta7] collect2: error: ld returned 1 exit status [sagelib-9.3.beta7] error: command 'g++' failed with exit status 1 }}} But I think, this might be easier to resolve.
comment:60 follow-up: 64 Changed 2 years ago by
This can't be the right fix. The .cpp files should not be installed.
comment:62 follow-up: 63 Changed 2 years ago by
Commit: | 93fe11f753cff24f7d81f00b91ec393bcbfc1721 → b4da02f2732e16a646c097c67c06c80686ba391f |
---|
Branch pushed to git repo; I updated commit sha1. New commits:
b4da02f | cimport directly from ginac.pynac
|
comment:63 Changed 2 years ago by
comment:64 follow-up: 65 Changed 2 years ago by
Replying to mkoeppe:
This can't be the right fix. The .cpp files should not be installed.
When you cimport you recompile .pxd
file. And by this it is trying to recompile everything that was included there. If the corresponding .cpp
files aren't found, there is a problem with that.
If we don't want to recompile everything, then we should put the cdef extern from
into pynac.pyx
and only do the type definitions in pynax.pxd
.
comment:65 Changed 2 years ago by
Replying to gh-kliem:
we should put the
cdef extern from
intopynac.pyx
and only do the type definitions inpynax.pxd
.
I don't know about this mechanism, but it sounds like what I am looking for. Could you elaborate on this or push a change to the branch?
comment:67 Changed 2 years ago by
I don't know, if that will work. I get many errors and by no means do I know how to fix them.
E.g.
[pynac-0.7.28.p18] gcc -DNDEBUG -g -fwrapv -O2 -Wall -march=native -O3 -g -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Iginac -I./ginac -I. -I. -I/srv/public/kliem/sage/local/lib/python3.7/site-packages/pip/_vendor/pep517 -I/srv/public/kliem/sage/local -I/usr/lib/python37.zip -I/usr/lib/python3.7 -I/usr/lib/python3.7/lib-dynload -I/srv/public/kliem/sage/local/lib/python3.7/site-packages -I/srv/public/kliem/sage/local/include -I/usr/include/python3.7m -c ginac/pynac.cpp -o build/temp.linux-x86_64-3.7/ginac/pynac.o -std=c++11 -DSING_NDEBUG -DOM_NDEBUG -DSING_NDEBUG -DOM_NDEBUG -I/srv/public/kliem/sage/local/include [pynac-0.7.28.p18] ginac/pynac.cpp:1875:50: error: no declaration matches '__pyx_t_5ginac_5pynac_GFunctionOptVector& GiNaC::function::registered_functions()' [pynac-0.7.28.p18] static __pyx_t_5ginac_5pynac_GFunctionOptVector &GiNaC::function::registered_functions(void); /*proto*/ [pynac-0.7.28.p18] ^~~~~ [pynac-0.7.28.p18] In file included from ginac/ginac.h:54, [pynac-0.7.28.p18] from ginac/pynac_wrap.h:13, [pynac-0.7.28.p18] from ginac/pynac.cpp:786: [pynac-0.7.28.p18] ginac/function.h:409:41: note: candidate is: 'static std::vector<GiNaC::function_options>& GiNaC::function::registered_functions()' [pynac-0.7.28.p18] static std::vector<function_options> & registered_functions(); [pynac-0.7.28.p18] ^~~~~~~~~~~~~~~~~~~~ [pynac-0.7.28.p18] ginac/function.h:340:7: note: 'class GiNaC::function' defined here [pynac-0.7.28.p18] class function : public exprseq
I don't even know if it is possible to import a cppclass
in the .pyx
part and to define in in the header then.
This option is definitely not mentioned here: https://cython.readthedocs.io/en/latest/src/userguide/wrapping_CPlusPlus.html
The problem with this entire setup is that ginac was never properly compiled and linked as a C++ module. I have the feeling that you are trying to achieve something that cython doesn't want you to do.
- If you have a C++ library you can create a cython header for this and directly expose the cpp classes via that header (e.g. you can create a pip installable GMP header that uses an existing GMP installation).
- If you have some C++ files and you want to expose them directly by a cython wrapper, but the header of this wrapper will recompile your files and thus every module cimporting from this wrapper needs to be able to recompile this as well.
- You can create a proper wrapper class of a cpp class. This class can be used by other modules without recompiling the actual cpp class.
This is what the picture looks like to me.
comment:68 Changed 20 months ago by
I digged into a bit. Not that I know a solution now, but at least I understand somehow what is going on.
The question is really what we want. cdef extern from
in a .pxd
-file assumes that every file cimporting this is linked against the same library or has access to the same sources. I don't think we can change this. So the way this is we must either compile ginac as a library and link against it or ship the sources (the later being really awful, because we have plenty of duplications and it takes forever to compile).
You can use cdef extern
in the pyx
however and expose the definitions in the header. This seems a very clean way of doing it, but there are tons of obstructions along the way. E.g. the functions must be declared static: https://cython.readthedocs.io/en/latest/src/userguide/external_C_code.html#implementing-functions-in-c
comment:69 Changed 20 months ago by
And apparently with a cppclass it works even in the header? (Different project, but there it seemed to work.)
I'm confused. I will see, if I can find out, what is causing the problem.
comment:71 Changed 19 months ago by
Milestone: | sage-9.4 → sage-9.5 |
---|
comment:72 Changed 18 months ago by
Authors: | Matthias Koeppe, ... |
---|---|
Milestone: | sage-9.5 → sage-duplicate/invalid/wontfix |
New approach: #32386 (Merge pynac as src/sage/symbolics/ginac)
comment:73 Changed 18 months ago by
Status: | new → needs_review |
---|
comment:74 Changed 18 months ago by
Reviewers: | → François Bissey |
---|---|
Status: | needs_review → positive_review |
Sure, let's try the other way.
comment:75 Changed 18 months ago by
Resolution: | → invalid |
---|---|
Status: | positive_review → closed |
comment:76 Changed 18 months ago by
Branch: | u/mkoeppe/repackage_pynac_as_a_pip_installable_package |
---|---|
Commit: | b4da02f2732e16a646c097c67c06c80686ba391f |
Removing the branch to avoid confusion in the future.
Cc-ing Ralf Stephan who maintains Pynac.