Opened 2 years ago

Closed 20 months ago

#28905 closed enhancement (fixed)

spkg-configure for cddlib

Reported by: dimpase Owned by:
Priority: major Milestone: sage-9.1
Component: build: configure Keywords:
Cc: saraedum, mjo Merged in:
Authors: Dima Pasechnik, Michael Orlitzky Reviewers: Matthias Koeppe
Report Upstream: N/A Work issues:
Branch: de9fb2e (Commits, GitHub, GitLab) Commit: de9fb2ee41fb6a083addd6a74b6b2da7504dd9b0
Dependencies: Stopgaps:

Status badges

Description (last modified by mkoeppe)

Provide spkg-configure.m4 to use libcdd/cdd from the system.

This ticket is limited to systems that install headers into /usr/include rather than /usr/include/cdd (Debian) or /usr/include/cddlib (Fedora).

Follow-up: #29413 (Upgrade cddlib, fix cddlib header search for packages latte_int, gfan, topcom)

Attachments (1)

Dockerfile-sage_gentoo (660 bytes) - added by mkoeppe 23 months ago.

Download all attachments as: .zip

Change History (48)

comment:1 Changed 2 years ago by dimpase

  • Summary changed from spkg-configure for libcdd to spkg-configure for cddlib

comment:2 Changed 2 years ago by mkoeppe

See also https://github.com/cddlib/cddlib/issues/26 "Consider installing headers in ${prefix}/include/cdd/ instead of ${prefix}/include"

comment:3 Changed 2 years ago by mkoeppe

  • Cc saraedum added

comment:4 Changed 2 years ago by saraedum

Sorry, I fail to understand the description of this ticket. What are you trying to achieve here exactly? Would be nice if you could elaborate a bit.

comment:5 Changed 2 years ago by dimpase

  • Description modified (diff)
  • Milestone changed from sage-9.0 to sage-9.1

The goal is to use libcdd/cdd from the system (as in all tickets from #27330). There should not be a problem with headers, after all we can install links to $SAGE_LOCAL/include/. I don't know which cdd executables are used in Sagemath. Again,for these we can install links in SAGE_LOCAL/bin/.

comment:6 Changed 2 years ago by dimpase

Am I right that the only parts of cdd(lib) currently used in Sage are executables cddexec* ? If so, it would be trivial to provide spkg-configure.m4.

comment:7 Changed 2 years ago by dimpase

  • Authors set to Dima Pasechnik
  • Branch set to u/dimpase/packages/cddlibconfig
  • Commit set to 5b8ca8f75540804a6b9ce53f539bb0b86e3173a6

New commits:

5b8ca8fspkg-configure for cddlib

comment:8 Changed 2 years ago by dimpase

cddlib is used as a library in gfan and latte. So one needs a check for headers and libraries.

comment:9 follow-up: Changed 23 months ago by mjo

  • Branch changed from u/dimpase/packages/cddlibconfig to u/mjo/ticket/28905
  • Cc mjo added
  • Commit changed from 5b8ca8f75540804a6b9ce53f539bb0b86e3173a6 to 61117fd8a9fb77f3bd494ca4281dad2b24ab3cd3

I did some work on this but the version of cddlib in Gentoo is too old to detect, so now I have to go fix that first.

comment:10 in reply to: ↑ 9 Changed 23 months ago by fbissey

Replying to mjo:

I did some work on this but the version of cddlib in Gentoo is too old to detect, so now I have to go fix that first.

Yes the version from the main tree is too old. I have a newer suitable version in the sage-on-gentoo overlay.

comment:11 Changed 23 months ago by mkoeppe

Is there a branch to test?

comment:12 Changed 23 months ago by mjo

  • Status changed from new to needs_review

Yeah, the one up there works for me. We did get cddlib upgraded in Gentoo and my branch detects it.

comment:13 follow-up: Changed 23 months ago by mkoeppe

Are there Dockerfiles or Docker images available that allow testing of the Gentoo Sage distribution? Off-topic for this ticket - but I would like to add a continuous integration test for https://github.com/mkoeppe/sage-numerical-backends-coin

comment:14 in reply to: ↑ 13 Changed 23 months ago by fbissey

Replying to mkoeppe:

Are there Dockerfiles or Docker images available that allow testing of the Gentoo Sage distribution? Off-topic for this ticket - but I would like to add a continuous integration test for https://github.com/mkoeppe/sage-numerical-backends-coin

coinor is too old in Gentoo. I tried to do an upgrade privately sometime last year. Unfortunately I lost the ebuilds in a mishap (and they weren't on vcs yet when it happened). I couldn't get it to work with sage which was annoying.

comment:15 Changed 23 months ago by mkoeppe

I primarily want to test with Gentoo's sage, not Gentoo's coinor. I can bring my own coinor from coinbrew.

comment:17 Changed 23 months ago by fbissey

Well I don't produce any docker images and I don't know if someone else did. Would be neat.

comment:18 Changed 23 months ago by mjo

The closest thing to a Docker image we have are the KVM images that the openstack project maintains:

http://gentoo.osuosl.org/experimental/amd64/openstack/

In theory you can boot one of those up, configure the sage-on-gentoo overlay,

https://github.com/cschwan/sage-on-gentoo

and then emerge sage. But each of those steps is tricky if you've never done it.

I'm slowly working on moving things from the overlay to the main gentoo repository, so that one day you can "emerge sage" without the overlay. (Basically, I'm just taking Francois's work and committing it to the main repository.)

This goes hand-in-hand with the spkg-configure work because in the past, while sage-on-gentoo provides the most stable and usable sage, it doesn't help me rebuild sage.git for testing tickets. But now whenever a package gets an spkg-configure.m4, adding that package to gentoo both (1) speeds up the sage.git build, and (2) brings the overlay one step closer to the main repository.

comment:19 Changed 23 months ago by mkoeppe

Thanks. For now I'm trying to bootstrap gentoo prefix on top of Ubuntu on Docker.

comment:20 Changed 23 months ago by fbissey

@ mjo you do know that the overlay has a "vbraun" branch where sgae-9999 checkouts Volker's merging branch do you? Pointing to some other branch is not that hard but I have never tried to test ticket that way. I mostly work on my vbraun branch so I know when something is being merged that causes problems.

@ mkoeppe for info sage-on-gentoo has a tester on prefix on top of ubuntu. So at least we know it works - mostly. There a few unidentified issue in doctesting going on at the moment https://github.com/cschwan/sage-on-gentoo/issues/560

comment:21 follow-up: Changed 23 months ago by mkoeppe

a tester = a person who tests it?

comment:22 in reply to: ↑ 21 Changed 23 months ago by fbissey

Replying to mkoeppe:

a tester = a person who tests it?

Yes.

Changed 23 months ago by mkoeppe

comment:23 Changed 23 months ago by mkoeppe

Anyway, this is how far I got in case someone would like to take a look. Sorry for hijacking the ticket.

comment:24 follow-up: Changed 23 months ago by dimpase

this branch does not go far enough: on Debian the headers are in /usr/include/cdd/, and on Fedora they are in /usr/include/cddlib/.

I think Singular is able to use cddlib headers from any of these locations.

I think libcdd* ought to be tested too.

comment:25 Changed 23 months ago by mkoeppe

Still hoping for upstream to make a decision on the canonical install location of the headers.

comment:26 Changed 23 months ago by mkoeppe

  • Status changed from needs_review to needs_work

comment:27 in reply to: ↑ 24 Changed 23 months ago by mjo

Replying to dimpase:

this branch does not go far enough: on Debian the headers are in /usr/include/cdd/, and on Fedora they are in /usr/include/cddlib/.

I left a comment about this in the macro before I got sidetracked. It looks to me like the sage lattE and gfan spkgs only look for the cddlib headers in the standard locations, so the system cddlib will only be usable if the headers are in the standard location. Is that wrong?

comment:28 follow-up: Changed 23 months ago by dimpase

spkg-confugure.m4 can figure out the prefix, AC_SUBST it into an env.var., which can be used to modify CFLAGS/CXXFLAGS in spkg-install scripts of lattE etc.

comment:29 in reply to: ↑ 28 Changed 23 months ago by mjo

Replying to dimpase:

spkg-confugure.m4 can figure out the prefix, AC_SUBST it into an env.var., which can be used to modify CFLAGS/CXXFLAGS in spkg-install scripts of lattE etc.

Well, sure, but that's a lot more involved. Checking only the standard locations is correct unless we modify those spkgs, right?

comment:30 Changed 23 months ago by dimpase

well, Singular goes out of its way to figure out the header location of cddlib. Indeed, from the log:

checking whether to check for gfanlib... yes
checking setoper.h usability... yes
checking setoper.h presence... yes
checking for setoper.h... yes
checking cdd/setoper.h usability... no
checking cdd/setoper.h presence... no
checking for cdd/setoper.h... no
checking cddlib/setoper.h usability... no
checking cddlib/setoper.h presence... no
checking for cddlib/setoper.h... no
checking whether libcddgmp is usable... checking for dd_free_global_constants... yes

comment:31 Changed 22 months ago by mjo

I think the best solution is to:

  1. Merge this now, since it's correct with respect to the rest of the (current) tree.
  2. Report the fact that the headers can be installed in non-standard locations to lattE/gfan upstream.
  3. (a) If those packages make the header search more robust, great; we upgrade the spkgs and we're done. (b) If they say they won't, then we hack something together as in https://trac.sagemath.org/ticket/28905#comment:29.
  4. We update the spkg-configure.m4 for cddlib to look in the non-standard locations as well.
  5. If cddlib upstream ever moves the headers, we can (at our leisure) simplify all of the checks.
Last edited 22 months ago by mjo (previous) (diff)

comment:32 Changed 22 months ago by saraedum

I am happy to accept a PR at cddlib upstream to install the headers into cdd/ on GitHub btw.

Last edited 22 months ago by saraedum (previous) (diff)

comment:33 Changed 21 months ago by mjo

  • Status changed from needs_work to needs_review

The next release of cddlib will use the cddlib prefix for its headers. But for now, our latte and gfan packages only look for the bare cdd.h, so that's what we need to check for. Can we merge this now, and then update things as they change?

comment:34 Changed 20 months ago by mkoeppe

Could you update the comment in the added file regarding the header file locations?

comment:35 Changed 20 months ago by git

  • Commit changed from 61117fd8a9fb77f3bd494ca4281dad2b24ab3cd3 to 27d5c6b13e79f4ea04126253d3107068bbd5f2ba

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

8508639spkg-configure for cddlib
27d5c6bTrac #28905: add header/library checks for cddlib.

comment:36 Changed 20 months ago by mjo

Sure, I added a real commit message too.

comment:37 Changed 20 months ago by mjo

  • Authors changed from Dima Pasechnik to Dima Pasechnik, Michael Orlitzky

comment:38 Changed 20 months ago by mkoeppe

  • Branch changed from u/mjo/ticket/28905 to u/mkoeppe/ticket/28905

comment:39 Changed 20 months ago by mkoeppe

  • Commit changed from 27d5c6b13e79f4ea04126253d3107068bbd5f2ba to 16b78ac93c0f5fcd19a3964a27ad5b7842e637e3

Rebased to 9.1.beta8


New commits:

082be67spkg-configure for cddlib
16b78acTrac #28905: add header/library checks for cddlib.

comment:40 Changed 20 months ago by mkoeppe

sorry, I messed this one up

comment:41 Changed 20 months ago by git

  • Commit changed from 16b78ac93c0f5fcd19a3964a27ad5b7842e637e3 to 27d5c6b13e79f4ea04126253d3107068bbd5f2ba

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

8508639spkg-configure for cddlib
27d5c6bTrac #28905: add header/library checks for cddlib.

comment:42 Changed 20 months ago by mkoeppe

Set it back to your commit

comment:43 Changed 20 months ago by git

  • Commit changed from 27d5c6b13e79f4ea04126253d3107068bbd5f2ba to de9fb2ee41fb6a083addd6a74b6b2da7504dd9b0

Branch pushed to git repo; I updated commit sha1. New commits:

0f04c3bbuild/pkgs/cddlib/distros/*.txt: New
de9fb2ebuild/pkgs/cddlib: Add upstream_url

comment:45 Changed 20 months ago by mkoeppe

  • Description modified (diff)

comment:46 Changed 20 months ago by mkoeppe

  • Description modified (diff)
  • Reviewers set to Matthias Koeppe
  • Status changed from needs_review to positive_review

Detects cddlib correctly on archlinux-latest-standard (https://github.com/mkoeppe/sage/runs/537774833).

We don't have a test for Gentoo (#29105, hint hint), but I assume you have tested it there.

As expected, not detected on debian and fedora. I have created follow-up ticket #29413 for that and edited the ticket description.

comment:47 Changed 20 months ago by vbraun

  • Branch changed from u/mkoeppe/ticket/28905 to de9fb2ee41fb6a083addd6a74b6b2da7504dd9b0
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.