Opened 8 years ago

Closed 4 years ago

#14887 closed task (duplicate)

libgap package

Reported by: felixs Owned by: tbd
Priority: minor Milestone: sage-duplicate/invalid/wontfix
Component: distribution Keywords: libgap
Cc: jpflori, fbissey Merged in:
Authors: Reviewers: Jeroen Demeyer
Report Upstream: None of the above - read trac for reasoning. Work issues:
Branch: Commit:
Dependencies: #14807 Stopgaps:

Status badges

Description (last modified by felixs)

libgap is a gap fork that installs a library that exposes all gap internals. this can be implemented as a wrapper library, without forking much, without exposing and without an extra libgap package. This is planned with libgap-2.0.

The known workarounds don't work with OSX toolchains, so it doesn't make sense to merge them. For now, sagelib must be patched for packaging on systems without libgap.

Change History (27)

comment:1 Changed 8 years ago by felixs

  • Type changed from PLEASE CHANGE to task

comment:2 follow-up: Changed 8 years ago by vbraun

Maybe it would be nice to discuss your plans with me, unless you want to take over future libGAP development. Exposing all of the GAP symbols is a feature, it allows me to experiment with the GAP kernel easily just from Python/Cython. Obviously one can reduce namespace pollution by only exporting select symbols but that would make future development much harder, in particular since the GAP kernel internals are only rather sparsely documented.

comment:3 in reply to: ↑ 2 ; follow-up: Changed 8 years ago by felixs

Replying to vbraun:

Maybe it would be nice to discuss your plans with me.

The discussion so far has not lead to a solution for the issue, I didn't mean to bug you more about it until I have a clear concept... but here you are:

I have implemented a wrapper library that wraps all symbols that don't match lib[gG][aA][pP]_*. currently this is in proof of concept state and is intended to work without libgap package (everything compiles and links, but something's wrong).

I have not factored in that you might want to experiment with python/cython, but it should be easy to switch to another libgap.so to replace the the wrapper (of course somebody would have to maintain the fork...)

How is your planned c++-wrapper intended to implement experiments with gap internals?

comment:4 in reply to: ↑ 3 ; follow-up: Changed 8 years ago by vbraun

Who is going to maintain the wrapper library and the necessary changes to the GAP build system, you or upstream? And if its not going into upstream, then it looks to me like we are jut debianizing GAP here which will make Sage less distributable, not more.

Replying to felixs:

How is your planned c++-wrapper intended to implement experiments with gap internals?

That would be after libgap integration in Sage is complete, which would end up a fairly comprehensive api to the gap kernel. Then we wouldn't need to eperiment with the gap kernel regularly, so it would be fine to hide all the gap private symbols and only expose a C++ api that matches the class hierarchy in Sage.

But first we need to wrap GAP permutations and then switch permutation groups from the gap interface to libgap.

comment:5 in reply to: ↑ 4 ; follow-up: Changed 8 years ago by felixs

Replying to vbraun:

Who is going to maintain the wrapper library and the necessary changes to the GAP build system, you or upstream? And if its not going into upstream, then it looks to me like we are jut debianizing GAP here which will make Sage less distributable, not more.

there are no upstream changes involved. and it has nothing to do with debian, the implementation also works (well, almost...) on top of sage, no more no less.

Replying to felixs:

How is your planned c++-wrapper intended to implement experiments with gap internals?

That would be after libgap integration in Sage is complete, which would end up a fairly comprehensive api to the gap kernel. Then we wouldn't need to eperiment with the gap kernel regularly, so it would be fine to hide all the gap private symbols and only expose a C++ api that matches the class hierarchy in Sage.

for the experiments one could adapt your gapify script to generate modified headers and a more complete linker script.

But first we need to wrap GAP permutations and then switch permutation groups from the gap interface to libgap.

Won't it be easier to implement the permutation thing only once, after the C++-wrapper exists?

Will the planned C++ wrapper be a part of sagelib or replace libgap or be another gap fork or something else (before it will be merged into gap-upstream)?

comment:6 in reply to: ↑ 5 ; follow-up: Changed 8 years ago by vbraun

Replying to felixs:

for the experiments one could adapt your gapify script to generate modified headers and a more complete linker script.

And by "one" I hope you mean "Felix" if you decide to change things.

Won't it be easier to implement the permutation thing only once, after the C++-wrapper exists?

The problem isn't implementing it, the problem is figuring out how to interact with the GAP kernel correctly.

While we are on the topic of feature requests: GMP integers don't work with libgap currently, only GAP's older native integer implementation.

comment:7 follow-up: Changed 8 years ago by vbraun

Also, what do you mean by "no upstream changes"? You definitely need to compile as PIC code for a shared library, make the GAP signal handler optional, and provide a mechanism to hook into input/output.

The C++ api library will be libgap-2.0 and supersede (but not conflict with) the current libgap.

comment:8 in reply to: ↑ 6 ; follow-up: Changed 8 years ago by felixs

Replying to vbraun:

Replying to felixs:

for the experiments one could adapt your gapify script to generate modified headers and a more complete linker script.

And by "one" I hope you mean "Felix" if you decide to change things.

I am not so sure whether experiments without libgap are necessary, but we can talk about that later. also, I am not "changing" things, i'm implementing a way aroung libgap (which is not packageable), I am not going to break libgap use.

Won't it be easier to implement the permutation thing only once, after the C++-wrapper exists?

The problem isn't implementing it, the problem is figuring out how to interact with the GAP kernel correctly.

And the kernel is changing that fast?

comment:9 in reply to: ↑ 7 Changed 8 years ago by felixs

Replying to vbraun:

Also, what do you mean by "no upstream changes"? You definitely need to compile as PIC code for a shared library, make the GAP signal handler optional, and provide a mechanism to hook into input/output.

fPIC is a compile flag, not an upstrem change. the signal handler and i/o can be compiled independently.

The C++ api library will be libgap-2.0 and supersede (but not conflict with) the current libgap.

yet another fork...

comment:10 in reply to: ↑ 8 ; follow-up: Changed 8 years ago by vbraun

Replying to felixs:

I am not so sure whether experiments without libgap are necessary

Maybe not, but if it does make development of libgap significantly more difficult then its not going into Sage anytime soon.

i'm implementing a way aroung libgap (which is not packageable)

Is packageable and has been packaged before.

comment:11 in reply to: ↑ 10 Changed 8 years ago by felixs

  • Dependencies set to #14807

Replying to vbraun:

Maybe not, but if it does make development of libgap significantly more difficult then its not going into Sage anytime soon.

I have pushed a preview into my WIP branch on github. it uses a linker script to prefix the symbols and does not recompile anything. now, sagelib works with or without libgap on sage and on debian (but the gap debian package has not been uploaded yet).

currently, libgap_wrap.h is generated manually. it probably should just contain all typedefs, the definitions and the externs from the gap headers eventually. if you don't insist on keeping libgap, i can adapt the libggapify script to do that automatically.

(libgap) Is packageable and has been packaged before.

yes, like the patched cliquer has been packaged.

comment:12 follow-up: Changed 8 years ago by vbraun

We shouldn't copy whole GAP source files into GAP, it is is under continual development and there are regular changes to the sources. Any necessary changes should be in the form of patches to the GAP sources.

Also, the GAP shared library should be usable as a stand-alone project. Having it used outside of Sage is probably the only chance to convince upstream to integrate patches for a shared library.

Your github WIP branch doesn't have libgap_wrap.h or the linker script, so I haven't seen them yet. In principle I don't care whether the symbols are renamed before or after compiling. But note that it has to work with the OSX and Solaris toolchains, not just with GNU.

(libgap) Is packageable and has been packaged before.

yes, like the patched cliquer has been packaged.

At least that one did not just copy the cliquer sources into the Sage library ;-)

comment:13 follow-up: Changed 8 years ago by vbraun

  • Cc jpflori added

Also, this is similar to the hack that I introduced in the new ATLAS-3,10 to build shared libraries with the ATLAS-3.8 names. There, Jean-Pierre Flori found that libtool on Cygwin fails to build shared libraries from objects that it didn't compile itself.

comment:14 in reply to: ↑ 12 ; follow-up: Changed 8 years ago by felixs

Replying to vbraun:

We shouldn't copy whole GAP source files into GAP,

into sagelib? I agree.

Also, the GAP shared library should be usable as a stand-alone project. Having it used outside of Sage is probably the only chance to convince upstream to integrate patches for a shared library.

as long as it's not a stand alone project it should not be packaged as such, because that exposes the hacked/duplicate headers/library.

Your github WIP branch doesn't have libgap_wrap.h or the linker script, so I haven't seen them yet.

because i forgot to add them. i intended to autogenerate them earlier (sorry)

In principle I don't care whether the symbols are renamed before or after compiling. But note that it has to work with the OSX and Solaris toolchains, not just with GNU.

isn't sage gcc only? are there any limitations to libtool that i should know about?

(libgap) Is packageable and has been packaged before.

yes, like the patched cliquer has been packaged.

At least that one did not just copy the cliquer sources into the Sage library ;-)

i have moved the non-upstream parts to the sage library, to resolve the patchwork.

comment:15 in reply to: ↑ 13 ; follow-up: Changed 8 years ago by felixs

Replying to vbraun:

Also, this is similar to the hack that I introduced in the new ATLAS-3,10 to build shared libraries with the ATLAS-3.8 names. There, Jean-Pierre Flori found that libtool on Cygwin fails to build shared libraries from objects that it didn't compile itself.

hopefully this isn't a real problem. gap/spkg-install also needs to compile the objects accordingly on cygwin...

comment:16 in reply to: ↑ 15 Changed 8 years ago by jpflori

Replying to felixs:

Replying to vbraun:

Also, this is similar to the hack that I introduced in the new ATLAS-3,10 to build shared libraries with the ATLAS-3.8 names. There, Jean-Pierre Flori found that libtool on Cygwin fails to build shared libraries from objects that it didn't compile itself.

hopefully this isn't a real problem. gap/spkg-install also needs to compile the objects accordingly on cygwin...

Not really unfortunately.

The problem is that libtool will refuse to gather object files into a shared library if it does not find the corresponding .lo files he produces when it is used to compile the files. A "fix" for that is to directly invoke the linker (ld, or ld through gcc/g++, on Cygwin), but of course you lose the benefits from using libtool.

In fact on Linux already you can see it ranting about that. On Cygwin it will not only rant, but also abort.

comment:17 Changed 8 years ago by fbissey

  • Cc fbissey added

comment:18 in reply to: ↑ 14 ; follow-up: Changed 8 years ago by vbraun

Replying to felixs:

isn't sage gcc only? are there any limitations to libtool that i should know about?

We rely on GCC as the compiler, but the linker is not necessarily GNU ld. On OSX the Darwin linker is totally different. On Solaris one can in principle use both GNU ld and Sun/Oracle ld but in practice Sage can only be built with Sun ld. I decided on renaming symbols before compilation because I didn't want to wade into that portability mess, though perhaps it could be sorted out. We'd need to be quite sure that it can be done on all supported and suppported-soon platforms, though.

A similar approach would be to use the objcopy utility from the GNU binutils to rename symbols in object files, but that is not portable to OSX either.

comment:19 in reply to: ↑ 18 Changed 8 years ago by felixs

i have now seperated the patches from gap upstream source (for convenience) and also moved to objcopy instead of using an ld script. also i've replaced the manually created libGAP_-headers by a modified GAPify.py. and the related doctests eventually pass.

it's really difficult to simplify this any further -- other than drop libgap compatibilty and then get rid of the prefixing. (I should have done it like that, but I didn't know better).

Replying to vbraun:

A similar approach would be to use the objcopy utility from the GNU binutils to rename symbols in object files, but that is not portable to OSX either.

if objcopy doesn't work for OSX, there must be some alternative. all that is required is the --redefine-syms functionality. here, e.g. somebody claims that an ld will cut it: http://stackoverflow.com/questions/3475854/an-objcopy-equivalent-for-mac-iphone.

comment:20 follow-up: Changed 8 years ago by vbraun

Does it work on all OSX versions? In particular, the old PPC toolchain..

This all sounds like a rather platform-specific hack for a problem that is currently solved in a completely portable way.

comment:21 in reply to: ↑ 20 ; follow-up: Changed 8 years ago by felixs

Replying to vbraun:

Does it work on all OSX versions? In particular, the old PPC toolchain..

i don't know.

This all sounds like a rather platform-specific hack for a problem that is currently solved in a completely portable way.

this depends on what you call a solution, imo forking is certainly not. If you intend to remove the symbol-renaming weirdness with libgap-2.0 (which looks like a requirement for upstream inclusion), no further platform-specific hacking will be required some day.

comment:22 in reply to: ↑ 21 ; follow-up: Changed 8 years ago by vbraun

Replying to felixs:

i don't know.

So, basically, your hack is neither in the interest of Sage, GAP, nor LibGAP development and serves only to make your Sage-on-debian fork easier. To me, that means you fail the primary goal of your GSoC project.

comment:23 in reply to: ↑ 22 Changed 8 years ago by felixs

  • Description modified (diff)
  • Priority changed from major to minor
  • Report Upstream changed from N/A to None of the above - read trac for reasoning.

Replying to vbraun:

So, basically, your hack is neither in the interest of Sage, GAP, nor LibGAP development and serves only to make your Sage-on-debian fork easier.

just to be sure, http://en.wikipedia.org/wiki/Fork_(software_development).

comment:24 Changed 7 years ago by vbraun_spam

  • Milestone changed from sage-6.1 to sage-6.2

comment:25 Changed 7 years ago by vbraun_spam

  • Milestone changed from sage-6.2 to sage-6.3

comment:26 Changed 7 years ago by vbraun_spam

  • Milestone changed from sage-6.3 to sage-6.4

comment:27 Changed 4 years ago by jdemeyer

  • Authors Felix Salfelder deleted
  • Milestone changed from sage-6.4 to sage-duplicate/invalid/wontfix
  • Resolution set to duplicate
  • Reviewers set to Jeroen Demeyer
  • Status changed from new to closed

Obsoleted by #22626.

Note: See TracTickets for help on using tickets.