Opened 7 years ago

Closed 6 years ago

#15594 closed enhancement (wontfix)

Use autotools for csage

Reported by: ohanar Owned by:
Priority: major Milestone: sage-duplicate/invalid/wontfix
Component: c_lib Keywords:
Cc: jdemeyer, vbraun, fbissey Merged in:
Authors: Reviewers: R. Andrew Ohana, Jeroen Demeyer
Report Upstream: N/A Work issues:
Branch: u/ohanar/csage_autotools (Commits) Commit: 440dcf13049208a3c650607a072ed1c49e4b02f1
Dependencies: #15604 Stopgaps:

Description (last modified by jdemeyer)

Use autotools instead of scons to build c_lib. After this change, polybori is the only part of Sage using scons.

Change History (54)

comment:1 Changed 7 years ago by ohanar

  • Authors set to R. Andrew Ohana
  • Branch set to u/ohanar/csage_autotools
  • Cc jdemeyer vbraun added
  • Commit set to cbfaef3db00847b4cfc912169607081d94fff840

Mainly threw this together to test python3 compatibility (since scons is not yet python3 compatible). I personally prefer autotools over scons, so I think it would be good to finish this, although I'm not sure what the best way to integrate it into the build system would be.

comment:2 Changed 7 years ago by git

  • Commit changed from cbfaef3db00847b4cfc912169607081d94fff840 to 7e6264cbe137b83f2f9da28263453f2c076ff462

Branch pushed to git repo; I updated commit sha1. New commits:

7e6264cc_lib: derp, continuing ignoring so's

comment:3 Changed 7 years ago by vbraun

Sounds good to me. I guess that would make polybori the only scons user (but C++ only)

comment:4 follow-up: Changed 7 years ago by jdemeyer

One important question is the following: are you going to track the autotools-generated files with git? If yes, that will enlarge the git repo by a significant amount. If no, you will require Sage developers to have autotools installed.

comment:5 in reply to: ↑ 4 Changed 7 years ago by ohanar

Replying to jdemeyer:

One important question is the following: are you going to track the autotools-generated files with git?

No.

If yes, that will enlarge the git repo by a significant amount. If no, you will require Sage developers to have autotools installed.

The source distribution will obviously have the autotools-generated output, but for those who are directly cloning the sage repository, they would need autotools (as is standard for most projects), or we would need to make autotools a standardish package (akin to gcc).

Integrating the prereq package runs into the same issue.

comment:6 Changed 7 years ago by vbraun_spam

  • Milestone changed from sage-6.1 to sage-6.2

comment:7 Changed 7 years ago by jdemeyer

  • Authors changed from R. Andrew Ohana to R. Andrew Ohana, Jeroen Demeyer
  • Summary changed from use autotools for csage to Use autotools for csage

Working on this.

comment:8 Changed 7 years ago by jdemeyer

  • Branch changed from u/ohanar/csage_autotools to u/jdemeyer/ticket/15594
  • Created changed from 12/26/13 23:21:25 to 12/26/13 23:21:25
  • Modified changed from 02/04/14 16:11:06 to 02/04/14 16:11:06

comment:9 Changed 7 years ago by jdemeyer

  • Commit changed from 7e6264cbe137b83f2f9da28263453f2c076ff462 to 42f4c25c6628477a7b83cc82a037aa2747ea73a3

It probably needs further testing, but this version should work.


New commits:

46c079fMerge remote-tracking branch 'origin/develop' into ticket/15594
144d006csage: remove the psuedo-depedencies polybori and pynac
d751a08Merge commit '144d006f9185299c4121eb94bd606ee1fb16d992' into HEAD
42f4c25Further develop c_lib autotools project

comment:10 Changed 7 years ago by jdemeyer

  • Dependencies set to #15604

comment:11 Changed 7 years ago by git

  • Commit changed from 42f4c25c6628477a7b83cc82a037aa2747ea73a3 to 6be41d8e83929a892646dbe6cd218c0f47ee95be

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

6be41d8Further develop c_lib autotools project

comment:12 Changed 7 years ago by git

  • Commit changed from 6be41d8e83929a892646dbe6cd218c0f47ee95be to 3f0d9907b3e6e5930b4164cfa33ba106f527f6e7

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

3f0d990Further develop c_lib autotools project

comment:13 Changed 7 years ago by git

  • Commit changed from 3f0d9907b3e6e5930b4164cfa33ba106f527f6e7 to bb9af9fbffa2debf1748f4b55363f40f0497fcf7

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

bb9af9fFurther develop c_lib autotools project

comment:14 Changed 7 years ago by git

  • Commit changed from bb9af9fbffa2debf1748f4b55363f40f0497fcf7 to 578f57890a69ed135c5c253c17146806601f5d3a

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

578f578Further develop c_lib autotools project

comment:15 Changed 7 years ago by git

  • Commit changed from 578f57890a69ed135c5c253c17146806601f5d3a to f0f16c4e83b888bafc6769d6cd58cc32f78b1405

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

f0f16c4Further develop c_lib autotools project

comment:16 Changed 7 years ago by jdemeyer

  • Description modified (diff)
  • Status changed from new to needs_review

comment:17 follow-up: Changed 7 years ago by jdemeyer

Despite what the plugin says, it merges cleanly for me...

comment:18 Changed 7 years ago by vbraun

Andrew: can you review this?

comment:19 in reply to: ↑ 17 Changed 7 years ago by ohanar

Replying to jdemeyer:

Despite what the plugin says, it merges cleanly for me...

Yes, I've been meaning to change that error message...

Replying to vbraun:

Andrew: can you review this?

Sure.

comment:20 Changed 7 years ago by ohanar

  • Branch changed from u/jdemeyer/ticket/15594 to u/ohanar/csage_autotools
  • Commit changed from f0f16c4e83b888bafc6769d6cd58cc32f78b1405 to 30508465580ac8f12e4967232742ad18827ec81b

Mostly looks fine. I just added a commit to reflect that on OS X the libtoolize binary is almost aways renamed to glibtoolize (so as not to conflict with OS X's libtool).


New commits:

3050846bootstrap: use glibtoolize by default on OS X

comment:21 Changed 7 years ago by ohanar

  • Status changed from needs_review to positive_review

comment:22 Changed 7 years ago by vbraun

  • Reviewers set to R. Andrew Ohana

comment:23 Changed 7 years ago by vbraun

  • Reviewers changed from R. Andrew Ohana to R. Andrew Ohana, Jeroen Demeyer

comment:24 Changed 7 years ago by vbraun

  • Branch changed from u/ohanar/csage_autotools to u/vbraun/csage_autotools

comment:25 Changed 7 years ago by git

  • Commit changed from 30508465580ac8f12e4967232742ad18827ec81b to 2b8f3f99920b6721f3f073b4c3e6b2bd8e7e555c
  • Status changed from positive_review to needs_review

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

2b8f3f9Merge tag '6.2.beta1' into t/15594/csage_autotools

comment:27 Changed 7 years ago by jdemeyer

  • Branch changed from u/vbraun/csage_autotools to u/jdemeyer/ticket/15594
  • Modified changed from 02/09/14 12:17:35 to 02/09/14 12:17:35

comment:28 Changed 7 years ago by jdemeyer

  • Commit changed from 2b8f3f99920b6721f3f073b4c3e6b2bd8e7e555c to 18aa06ca9172cb472aba6bd2aeb7c15bfde34f76

New commits:

18aa06cAdd src/c_lib/configure target in top-level Makefile

comment:29 Changed 7 years ago by ohanar

  • Status changed from needs_review to positive_review

comment:30 Changed 7 years ago by git

  • Commit changed from 18aa06ca9172cb472aba6bd2aeb7c15bfde34f76 to cc9d5d90b2e3237974ce43ee4fa2ce8195f4a3cc
  • Status changed from positive_review to needs_review

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. This was a forced push. New commits:

cc9d5d9Use autotools for csage

comment:31 Changed 7 years ago by jdemeyer

Since this keeps conflicting, I squashed the whole branch.


New commits:

cc9d5d9Use autotools for csage

comment:32 Changed 7 years ago by jdemeyer

  • Status changed from needs_review to positive_review

comment:33 Changed 7 years ago by vbraun

Fails on snapperkob with

./bootstrap -d
make[1]: Entering directory `/home/buildbot/build/sage/snapperkob/sage_git/build'
rm -rf config configure build/Makefile-auto.in
cd src/c_lib && rm -f config.h.in* configure Makefile.in src/Makefile.in include/Makefile.in
make[1]: Leaving directory `/home/buildbot/build/sage/snapperkob/sage_git/build'
configure.ac:61: installing `config/config.guess'
configure.ac:61: installing `config/config.sub'
configure.ac:30: installing `config/install-sh'
configure.ac:30: installing `config/missing'
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, `../../config'.
libtoolize: copying file `../../config/ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `../../m4'.
libtoolize: copying file `../../m4/libtool.m4'
libtoolize: copying file `../../m4/ltoptions.m4'
libtoolize: copying file `../../m4/ltsugar.m4'
libtoolize: copying file `../../m4/ltversion.m4'
libtoolize: copying file `../../m4/lt~obsolete.m4'
configure.ac:8: require Automake 1.13, but have 1.11.3
make: *** [src/c_lib/configure] Error 1
make: Target `start' not remade because of errors.
program finished with exit code 2

and on eno

./bootstrap -d
make[1]: Entering directory `/home/buildbot/build/sage/eno-1/sage_git/build'
rm -rf config configure build/Makefile-auto.in
cd src/c_lib && rm -f config.h.in* configure Makefile.in src/Makefile.in include/Makefile.in
make[1]: Leaving directory `/home/buildbot/build/sage/eno-1/sage_git/build'
configure.ac:61: installing `config/config.guess'
configure.ac:61: installing `config/config.sub'
configure.ac:30: installing `config/install-sh'
configure.ac:30: installing `config/missing'
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, `../../config'.
libtoolize: copying file `../../config/ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `../../m4'.
libtoolize: copying file `../../m4/libtool.m4'
libtoolize: copying file `../../m4/ltoptions.m4'
libtoolize: copying file `../../m4/ltsugar.m4'
libtoolize: copying file `../../m4/ltversion.m4'
libtoolize: copying file `../../m4/lt~obsolete.m4'
configure.ac:22: warning: macro `AM_PROG_AR' not found in library
configure.ac:8: require Automake 1.13, but have 1.11.1
make: *** [src/c_lib/configure] Error 1
make: Target `start' not remade because of errors.
program finished with exit code 2

The confball version is 13.

comment:34 Changed 7 years ago by git

  • Commit changed from cc9d5d90b2e3237974ce43ee4fa2ce8195f4a3cc to 5f6f9f17245504a2d22e4a82fd808ad1223ccd2c
  • Status changed from positive_review to needs_review

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

5f6f9f1Try downloading on every status code (instead of 63 and 127 only)

comment:35 Changed 7 years ago by jdemeyer

Volker, can you please test this on the buildbots again?

comment:36 Changed 7 years ago by vbraun

Still doesn't work on most machines, though it fails later on in csage. E.g. on mod:

make[3]: Entering directory `/mnt/SSD1/mod_buildslave/sage_git/build/src/c_lib'
make[3]: warning: -jN forced in submake: disabling jobserver mode.
CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/bash /mnt/SSD1/mod_buildslave/sage_git/build/config/missing aclocal-1.13 -I ../../m4
/mnt/SSD1/mod_buildslave/sage_git/build/config/missing: line 81: aclocal-1.13: command not found
WARNING: 'aclocal-1.13' is missing on your system.
         You should only need it if you modified 'acinclude.m4' or
         'configure.ac' or m4 files included by 'configure.ac'.
         The 'aclocal' program is part of the GNU Automake package:
         <http://www.gnu.org/software/automake>
         It also requires GNU Autoconf, GNU m4 and Perl in order to run:
         <http://www.gnu.org/software/autoconf>
         <http://www.gnu.org/software/m4/>
         <http://www.perl.org/>
make[3]: *** [aclocal.m4] Error 127

This was with configure-15.tar.gz

comment:37 Changed 7 years ago by ohanar

I think this could probably be resolved by disabling maintainer mode either in csage's configure.ac or in deps. Doing this with the top level configure script as well would make it so that we don't have to use tar with the m argument (I believe, haven't actually tested).

comment:38 Changed 7 years ago by ohanar

  • Branch changed from u/jdemeyer/ticket/15594 to u/ohanar/csage_autotools
  • Commit changed from 5f6f9f17245504a2d22e4a82fd808ad1223ccd2c to 9d3643642ac027378d67b29f5e62cee90873afa4

This has my proposed fix. It works for me when I remove autotools from my path.


New commits:

9d36436disable autoconf's maintainer mode when run in sage's build system

comment:39 Changed 7 years ago by ohanar

For reference, it looks like maintainer mode was disabled for the top level configure in a38f919, but it never got re-disabled when it was merged into #15606.

comment:40 Changed 7 years ago by vbraun

Jeroen didn't like it back then... Really I don't want to disable it on *my* machine since I do have a working autotools install. In an ideal world we'd only disable maintainer mode if we had to download the confball. Not sure how to implement that, it seems like we would need some file whose existence acts as a flag. Though its probably not too much of a loss.

comment:41 Changed 7 years ago by fbissey

  • Cc fbissey added

comment:42 Changed 6 years ago by vbraun_spam

  • Milestone changed from sage-6.2 to sage-6.3

comment:43 follow-up: Changed 6 years ago by leif

Cool. How do we get the PolyBoRi guys to drop SCons? :P

comment:44 in reply to: ↑ 43 Changed 6 years ago by fbissey

Replying to leif:

Cool. How do we get the PolyBoRi guys to drop SCons? :P

Wish I knew. It also hasn't been very active lately. While Alexander Drier wasn't the only active dev his departure seems to have left a big void. He was also the one who was fixing most of the problem we reported from sage-on-gentoo. If I had more free time I'd attempt a conversion to autotools myself.

comment:45 Changed 6 years ago by fbissey

I'd like to request one small change. Jean-Pierre Flori helped to make sure we could build against gmp instead of mpir at #12661.

libcsage_la_LIBADD = -lntl -lpari -lmpir $(PYTHON_LDFLAGS)

will just revert it completely to where we were before (but for different reasons). Can we replace -lmpir with -lgmp in that line please.

comment:46 Changed 6 years ago by git

  • Commit changed from 9d3643642ac027378d67b29f5e62cee90873afa4 to 440dcf13049208a3c650607a072ed1c49e4b02f1

Branch pushed to git repo; I updated commit sha1. New commits:

440dcf1maintain compatibility with gmp

comment:47 follow-up: Changed 6 years ago by fbissey

Thanks! One more thing I just spotted:

diff --git a/src/c_lib/Makefile.am b/src/c_lib/Makefile.am
new file mode 100644
index 0000000..e91cd48
--- /dev/null
+++ b/src/c_lib/Makefile.am
@@ -0,0 +1,6 @@
+SUBDIRS = src include
+
+ACLOCAL_AMFLAGS = -I ../../m4
+
+clean-local:
+	rm -f .sconsign.dblite

.sconsign.dblite is a file generated by scons, now that we are ditching it, it shouldn't be present. I am guessing you are trying to cover the upgrade path. Is it necessary for us to provide this target. Furthermore, I do not see when it is called. So we have a target to clean some old scons rubbish but it doesn't look like it is run automatically. If it was always run I could see some value but if it is voluntary people may just as well use rm.

comment:48 in reply to: ↑ 47 Changed 6 years ago by leif

Replying to fbissey:

Thanks! One more thing I just spotted:

diff --git a/src/c_lib/Makefile.am b/src/c_lib/Makefile.am
new file mode 100644
index 0000000..e91cd48
--- /dev/null
+++ b/src/c_lib/Makefile.am
@@ -0,0 +1,6 @@
+SUBDIRS = src include
+
+ACLOCAL_AMFLAGS = -I ../../m4
+
+clean-local:
+	rm -f .sconsign.dblite

If it was always run I could see some value but if it is voluntary people may just as well use rm.

Well, it's a hidden file... ;-)

(And we could run make clean-local from a script.)

comment:49 Changed 6 years ago by fbissey

We could, and in fact I looked to see if it was done within this ticket before writing.

comment:50 Changed 6 years ago by leif

In contrast to me, as I was assuming you did already. So I was right.

comment:51 Changed 6 years ago by vbraun_spam

  • Milestone changed from sage-6.3 to sage-6.4

comment:52 Changed 6 years ago by jdemeyer

Alternative: #17484

comment:53 Changed 6 years ago by jdemeyer

  • Authors R. Andrew Ohana, Jeroen Demeyer deleted
  • Milestone changed from sage-6.4 to sage-duplicate/invalid/wontfix
  • Status changed from needs_review to positive_review

Close as wontfix...

comment:54 Changed 6 years ago by vbraun

  • Resolution set to wontfix
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.