Only ./bootstrap should glob build/pkgs

[mkoeppe]

Right now both ./bootstrap and ./configure (via m4/sage_spkg_collect.m4) ​glob build/pkgs. This should be streamlined.

Currently ./configure is too sensitive to stray files/subdirectories in build/pkgs (such as those that are .gitignore'd in another branch), as noted in ​https://groups.google.com/d/msg/sage-release/eVvjvJ6Iii8/W51ouUKpDwAJ (thread). While #29120 provides a quick workaround, the present ticket has the proper fix.

[mkoeppe]

Branch is on top of #28095.

---

New commits:

​cf5607e	Add configure options '--enable-SPKG', --disable-SPKG' for optional/experimental SPKG
​e4675d1	Actually implement --enable-SPKG and --disable-SPKG
​a3e0e47	Change type of gmp, mpir to 'standard' (they are mutually exclusive standard packages)
​4ed7b05	Fix indentation of '--with-mp=' options in configure --help
​82ee84d	sage_spkg_enable.m4: Popdef what wass pushdef-ed
​f7a84e0	Enumerate packages in ./bootstrap; only warn about missing 'type' files; use the generated list in m4/sage_spkg_collect.m4 instead of globbing

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​6ec6413	Merge branch 'develop' of git://trac.sagemath.org/sage into t/29114/no-spkg-globbing-in-configure
​9295a74	sage_spkg_enable.m4: Popdef what wass pushdef-ed
​355d147	Enumerate packages in ./bootstrap; only warn about missing 'type' files; use the generated list in m4/sage_spkg_collect.m4 instead of globbing

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​bbd6521

Makefile [configure]: also depend on build/pkgs/type

[dimpase]

Could you explain what "glob" means here?

[dimpase]

This reminds me of #27373 - which becomes obsolete as soon as this ticket and #29113 is done.

[embray]

It seems this is adding a bunch of previously unnecessary complication only because #29038, which I strongly objected to, was merged. I didn't step in then because I was busy with other things. Can we please roll back #29038 instead? I'm absolute convinced there's no need for it.

[embray]

See #29118 for an alternative proposal.

[dimpase]

I think we should progress, even if the route is not optimal, and Matthias has a lot of time for working on the build system now.

I hate to see ./configure running many times as I launch make, it really slows working on build system down, and I hope #29113 will happen soon.

[embray]

I should add, some of the changes in this ticket look reasonable. But it's mixed up with a bunch of other stuff (the way this is built on top of #28095 in particular is confusing). I'd like to start with #29118 and then see some fixes to individual bugs that predate #29038 .

[embray]

Replying to dimpase:

I think we should progress, even if the route is not optimal, and Matthias has a lot of time for working on the build system now.

Progress on what though? I don't think you can "progress" on unsolid footing. I should note that part of the motivation behind #29038 is to enable the approach in #27824 which I also think is overcomplicated.

I don't think we should be making complicated changes on top of complicated changes to enable even more complicated changes. I hate to sound like Jeroen here because I've been on the other side of disagreements like this with him. But he'd probably be saying the same if he were here and I'd agree this time.

I would like to know what the bigger picture is to all this before making changes that don't seem well-motivated. Don't commit a sunk cost fallacy especially when it's relatively early...

[dimpase]

If you want to talk about simplification, why don't we just drop sage -i/-f instead of trying to keep this awful hack - something I proposed a year ago, and Jeroen objected...

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​1f193bf	Issue only a warning, not an error, if there are stray files in build/pkgs
​a04733a	Merge branch 't/29120/one-line-fix-configure-too-sensitive-to-stray-files-in-build-pkgs' into t/29114/no-spkg-globbing-in-configure
​25a55e0	Enumerate packages in ./bootstrap; only warn about missing 'type' files; use the generated list in m4/sage_spkg_collect.m4 instead of globbing
​7214ed7	Makefile [configure]: also depend on build/pkgs/type

[mkoeppe]

I've rebased on top of #29120, which has the "hot fix", which may be easier to review and should go into the next beta.

The present ticket, #29114, is not so urgent.

[mjo]

Since this list changes maybe once a year, we could just write it into configure.ac and avoid that part of the bootstrap process.

[mkoeppe]

I think that may be too radical a change (although I am personally in favor of it - agreeing with the autotools policies) - Sage has traditionally used source discovery instead of listing files explicitly. That's why in this ticket I just want to put the source discovery for build/pkgs into 1 script

[mkoeppe]

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.