Opened 16 months ago

Last modified 6 weeks ago

#29024 new enhancement

Make location of the Singular shared library configurable through sage-config

Reported by: mkoeppe Owned by:
Priority: major Milestone: sage-9.4
Component: build: configure Keywords: sd111
Cc: dimpase, fbissey, slelievre, isuruf, mjo, gh-tobiasdiez, arojas Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description (last modified by mkoeppe)

  1. For singular, the location of the dynamic library needs to be communicated to the sage runtime.

We add a configuration variable SINGULAR_SO to sage_conf.py.in.

The filename can be found from src/Singular/libSingular.la (currently not installed):

# The name that we can dlopen(3).
dlname='libSingular-4.1.1.dylib'
...
# Directory that this library needs to be installed in:
libdir='/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/local/lib'

This will:

  • enable venv builds of sagelib (#30578, #29013) -- for which _get_shared_lib_filename is wrong
  • provide preparation for adding an spkg-configure.m4 for singular.
  1. src/bin/sage-env unconditionally sets environment variable SINGULARPATH="$SAGE_LOCAL/share/singular"

If SINGULARPATH is already set by the user, perhaps the directory in $SAGE_LOCAL should be prepended instead of overwriting it.

  1. src/bin/sage-env unconditionally sets environment variable SINGULAR_EXECUTABLE="$SAGE_LOCAL/bin/Singular".

It should be investigated whether this is actually needed in any supported configuration. If not, remove.

  1. src/sage/interfaces/singular.py tries to use SINGULARPATH as if it is a directory name (it is a colon-separated list of directories):
    singular_docdir = SINGULARPATH + "/../info/"
    

This should be fixed so that it works when SINGULARPATH is set by a user to a colon-separated list.

Attachments (2)

spkg-configure.m4 (219 bytes) - added by gh-thierry-FreeBSD 13 months ago.
spkg-configure.m4 to be put under /build/pkgs/singular/
homebrew-singular-experiment.patch (3.0 KB) - added by dimpase 8 months ago.

Download all attachments as: .zip

Change History (30)

comment:1 follow-up: Changed 16 months ago by mkoeppe

Which versions of singular are good?

comment:2 in reply to: ↑ 1 Changed 16 months ago by gh-mwageringel

Replying to mkoeppe:

Which versions of singular are good?

Sage does not currently work correctly with any version of Singular, see #25993.

comment:3 Changed 16 months ago by dimpase

well, Debian does have a Singular version which is compatible with their Sage package (v8.6 or so, I guess).

comment:4 Changed 16 months ago by mkoeppe

  • Milestone changed from sage-9.1 to sage-9.2

Thanks for the info. I'll check back when #25993 is done.

comment:5 follow-up: Changed 13 months ago by gh-thierry-FreeBSD

This ticket should be listed in #27330 (ATM singular is listed in "No Tickets yet").

comment:6 in reply to: ↑ 5 Changed 13 months ago by mkoeppe

Replying to gh-thierry-FreeBSD:

This ticket should be listed in #27330 (ATM singular is listed in "No Tickets yet").

I've added it

comment:7 Changed 13 months ago by dimpase

  • Component changed from packages: standard to build: configure

Changed 13 months ago by gh-thierry-FreeBSD

spkg-configure.m4 to be put under /build/pkgs/singular/

comment:8 follow-up: Changed 13 months ago by gh-thierry-FreeBSD

[Copying elements from #29024]

  • my 1st problem was a build failure with Singular taken from an external package: to solve it, Singular must be built --with-ntl: it would be fine to add this check to spkg-configure.m4 (but I do not know how!)
  • my 2nd error was when building dochtml: NameError: Singular library 'freegb.lib' not found. To solve it, src/sage/env.py and src/bin/sage-env must be modified so that SINGULARPATH is set to singlar's share dir (and not SAGE_SHARE / DESTDIR), and SINGULAR_EXECUTABLE is set to the real Singular and not under the DESTDIR directory.

Now it is OK for me (on FreeBSD).

Last edited 12 months ago by slelievre (previous) (diff)

comment:9 Changed 12 months ago by mkoeppe

  • Cc mjo added

comment:10 Changed 10 months ago by mkoeppe

  • Description modified (diff)

comment:11 Changed 9 months ago by mkoeppe

  • Milestone changed from sage-9.2 to sage-9.3

comment:12 Changed 8 months ago by dimpase

the error message

        raise RuntimeError(
            "libSingular not found--a working Singular install in $SAGE_LOCAL "
            "is required for Sage to work")

should be modified, too. Perhaps just remove in $SAGE_LOCAL

comment:13 Changed 8 months ago by mkoeppe

Moving this ticket forward is great, but note that it is unrelated to the build problem on homebrew reported on sage-devel... and it is also unlikely that the homebrew version of singular actually works for us (homebrew does not like to patch).

As I explained in https://groups.google.com/d/msg/sage-devel/KXK_zxzfhIQ/RO3FryYWAwAJ, on the reporter's system the computed order of -I options is wrong -- something that I was not able reproduce on my system.

comment:14 in reply to: ↑ 8 Changed 8 months ago by dimpase

Replying to gh-thierry-FreeBSD:

[Copying elements from #29024]

  • my 1st problem was a build failure with Singular taken from an external package: to solve it, Singular must be built --with-ntl: it would be fine to add this check to spkg-configure.m4 (but I do not know how!)
  • my 2nd error was when building dochtml: NameError: Singular library 'freegb.lib' not found. To solve it, src/sage/env.py and src/bin/sage-env must be modified so that SINGULARPATH is set to singlar's share dir (and not SAGE_SHARE / DESTDIR), and SINGULAR_EXECUTABLE is set to the real Singular and not under the DESTDIR directory.

Now it is OK for me (on FreeBSD).

The fix in SageMath needs some programming - specifically, the code in singular.pyx calls dlopen on SINGULAR_SO, which is computed by _get_shared_lib_filename dynamically in env.py, and assuming that it is in SAGE_LOCAL. So this probably can instead by handled using cython_aliases from the same module (which does compute all the needed info for Singular, it seems). Or by fixing _get_shared_lib_filename so that it also searches elsewhere.

comment:15 Changed 8 months ago by mkoeppe

Is it known whether this comment added in 2009 to singular.pyx is still valid:

    This initializes the SINGULAR library. This is a hack to some
    extent.

    SINGULAR has a concept of compiled extension modules similar to
    Sage. For this, the compiled modules need to see the symbols from
    the main program. However, SINGULAR is a shared library in this
    context these symbols are not known globally. The work around so
    far is to load the library again and to specify ``RTLD_GLOBAL``.

comment:16 follow-up: Changed 8 months ago by dimpase

Someone working on innards of Singular might know. (I'd be surprised if it was changed).

Singular is not alone - this hack is also applied to libgap. I'm inclined to fix _get_shared_lib_filename so that it can accept other libdirs, not only SAGE_LOCAL/lib

Changed 8 months ago by dimpase

comment:17 in reply to: ↑ 16 Changed 8 months ago by mkoeppe

Replying to dimpase:

I'm inclined to fix _get_shared_lib_filename so that it can accept other libdirs, not only SAGE_LOCAL/lib

Yes, this function certainly would need changing if we continue to use it. For example, sysconfig.get_config_var('LIBDIR') gives a directory of the system python, not of the current venv.

But I think better solution would be to wrap loading of modules that are linked to singular by sys.setdlopenflags.

This is what pytorch does when it has to use RTLD_GLOBAL (see https://github.com/pytorch/pytorch/blob/master/torch/__init__.py#L132)

comment:19 Changed 8 months ago by mkoeppe

I have opened #30561 for this

comment:20 Changed 8 months ago by mkoeppe

  • Description modified (diff)
  • Milestone changed from sage-9.3 to sage-9.2
  • Summary changed from spkg-configure.m4 for singular to Make location of the Singular shared library configurable through sage-config

comment:21 Changed 8 months ago by mkoeppe

  • Description modified (diff)

comment:22 Changed 8 months ago by mkoeppe

  • Description modified (diff)

comment:23 Changed 7 months ago by mkoeppe

  • Cc gh-tobiasdiez added
  • Description modified (diff)

comment:24 Changed 7 months ago by mkoeppe

  • Description modified (diff)

comment:25 Changed 7 months ago by mkoeppe

  • Milestone changed from sage-9.2 to sage-9.3

comment:26 Changed 5 months ago by mkoeppe

  • Keywords sd111 added

Hoping we can make progress on this ticket this week - https://wiki.sagemath.org/days111

comment:27 Changed 5 months ago by mkoeppe

  • Cc arojas added

comment:28 Changed 6 weeks ago by mkoeppe

  • Milestone changed from sage-9.3 to sage-9.4

Sage development has entered the release candidate phase for 9.3. Setting a new milestone for this ticket based on a cursory review of ticket status, priority, and last modification date.

Note: See TracTickets for help on using tickets.