Opened 3 years ago
Last modified 3 weeks ago
#27330 new task
Meta-ticket: spkg-configure: Try to use as many system packages as possible
Reported by: | embray | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | sage-9.6 |
Component: | build: configure | Keywords: | spkg-configure, sd111 |
Cc: | tmonteil, mjo, gh-timokau, fbissey, isuruf, mmezzarobba, thansen, gh-tobiasdiez | Merged in: | |
Authors: | Reviewers: | ||
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description (last modified by )
#24919 implemented a mechanism for checking for checking for sagelib's direct and indirect dependencies at the time of running the main ./configure
script. This was already done in an ad-hoc manner for a handful of packages (e.g. gcc, git, yasm, curl), but we wanted to be able to do the same for other packages, especially those that are common on Linux-like systems.
This has now been implemented for many other packages, or is in the process of being implemented. Those packages are listed below. However, there are many others yet to be done, and this ticket will attempt to list them in some rough order of priority.
For use of Python packages: See #29023 Meta-ticket: In a python 3 build, use system Python packages
Already done
The following packages already have spkg-configure.m4
checks that are working on most or all supported platforms:
- 4ti2 (#30887, optional package)
- arb (#27270)
- boost_cropped/boost (#29100) -- many systems have boost headers available
- brial (#29369) - see also #29490 - where it was disabled due to crashes on Fedora 30
- bzip2 (#27182)
- cbc (#28908)
- cddlib (#28905)
- cliquer (#28885)
- cmake (#27487)
- coxeter3 (#29721, optional)
- curl (#24919)
- eclib (#28333)
- ecl (#29617)
- ecm (#27271)
- fflas-ffpack (#29453)
- flint (#27264) - merged by #27270
- flintqs (#29430)
- fplll (#29245)
- freetype (#27168)
- gc (#28991)
- gcc (#24919)
- gf2x (#27238)
- gfan (#28985)
- gfortran (#24919)
- giac (#29541)
- git (#24919)
- givaro (#28342)
- glpk (#28459)
- gmp (#27212)
- gp2c (#28242)
- gsl (#28879)
- igraph (#30899, optional package)
- iml (#27272)
- isl (#27864)
- iconv (#27823)
- lcalc (#28224)
- libatomic (#28045)
- libbraiding (#29458)
- libffi (#25900, #27114)
- libgd (#27825)
- libhomfly (#29630)
- libpng (#27186)
- lrcalc (#28231)
- lrslib (#27804, optional package)
- m4ri(e) (#28342)
- mpc (#27259) - merged by #27822
- mpfi (#28295)
- mpfr (#27258) - merged by #27822
- mpir (#27212) - removed in #32549, #32727
- nauty (#28958)
- ncurses (#27277)
- ninja-build (#27801)
- ntl (#27265) - merged by #27822
- openblas (#27870)
- openssl (#30557, #32580)
- palp (#29672)
- pari (#28242)
- pari's packages, apart from pari_jupyter (#28242)
- patch (#24919)
- pcre (#27284)
- perl_term_readline_gnu (#27795)
- pkgconf (#27827)
- planarity (#29063)
- polymake (#31527)
- ppl (#29454)
- primecount (#25009)
- primesieve (#25009)
- pynac (#29542)
- python3 (#27824) this one is of course a very special case; see the ticket for details.
- qepcad (#21783)
- qhull (#31161)
- r (#28884) (alternative view: getting R from the system has been desirable for a long time and will require more than just an
spkg-configure.m4
; see #25503) - readline (#27277)
- rw (#27814)
- singular (#29024; dup: #29515)
- sqlite (#29002)
- suitesparse (#29502)
- symmetrica (#28208)
- sympow (#29673)
- tachyon (#28919)
- tox (#30544)
- xz (#26286)
- yasm (#24919)
- zeromq (#26286)
- zlib (#26286)
- zn_poly (#29649)
In Progress
The following packages have in-progress tickets to provide configure-time detection on the system:
Have tickets
The following packages have tickets for their conversions, but don't have working prototypes (i.e. branches) yet:
- BLAS (atlas and other non-openblas): resolving how to handle finding a usable BLAS should be one of the highest priorities; it faces many of the same issues we faced with MPIR/GMP in #27212, but perhaps with even more complexity). One can already use system's openblas, though (#27870), and we removed ATLAS in #30350.
- csdp (#31145, optional package)
- gap (using system GAP is desirable, but non-trivial, especially now that we need a working libgap) (#29644)
- jmol (#30316)
- lie (#32055, experimental package)
- mathjax (#30296)
- perl (#31839, script package)
- symengine (#32421, optional package)
- texlive (#31529)
No ticket yet
- bliss (optional package)
Use of system Python packages
This will be be handled by #29023. Please do not add Python packages here.
The following are exceptions (because it does not run in the same Python as sage):
- notebook (#30124), rst2ipynb, and their dependencies
Testing infrastructure
- #29053: Add debian/fedora package information to build/pkgs, generate Dockerfiles and installation help
- #29060: Meta-ticket: Add Dockerfiles and CI scripts for integration testing of source and binary distributions and of downstream packages
Meta infrastructure
- #31114 Add one file
build/pkgs/*/repology.txt
per spkg
Documentation
Unfortunately there is still not good documentation for how to implement these configure-time dependency checks.
- #26668: add documentation for implementing spkg-configure.m4 and distros/ for spkgs
- #29018: improve documentation of m4/spkg-configure.m4
- #30543: Document spkg-configure.m4 and distros directory
Bug-fixing
A number of bugs/deficiencies has been uncovered in spkg-configure.m4 files and related configuration, and a number of tickets opened/closed
- #26715: build/pkgs/gfortran/spkg-configure.m4 works incorrectly if CC and CXX are already there
- #27219: have bootstrap fail gracefully if no pkg-config is available
- #27373: some dummy packages must be only re-installable via ./configure --with-...
- #27567: Add --with-system-<spkg> options to configure
- #27641: SAGE_SPKG_CONFIGURE macro: Add new pre-check and post-check optional arguments
- #27642: Re-run configure+make after installing an SPKG with sage -i
- #27936: spkg-configure.m4 for pcre should test for UTF support etc
- #27941: R installation fails on macOS with libcurl from Anaconda
- #28095: Add --enable-OPTIONALSPKG options to configure
- #28317: Use -print-prog-name instead of -print-file-name in gcc spkg-configure.m4
- #28341: Allow doc to build with SAGE_PARI_CFG undefined
- #28380: bump up givaro version in spkg-configure.m4
- #28401: pass correct --with-flint to eclib's configure
- #28405: correct the test for galpol package in pari's spkg-configure.m4
- #28409: Flint configure fails when using system GMP in certain cases
- #28906: generate libpng.pc, zlib.pc if needed
- #28956: correct AC_CHECK_PROG call in spkg-configure for bzip2
- #29071: verify if cblas.pc and lapack.pc should be replaced by links to openblas.pc
- #30752: switch the default mp library to gmp
- #31348: build/pkgs/mpfr/spkg-configure.m4: Check pkg-config first
- #31358: python3 spkg-configure.m4: Do not reject python based on sysconfig LDFLAGS containing "-L."
- #31429: flint spkg-configure: Reject FLINT 2.7.x
- #31528: Cleanup spkg-configure.m4 files that mix tabs and spaces
- #31539: conda-forge-standard (linux): python3 spkg-configure.m4 rejects conda's python3
- #32789: Fix spkg-configure.m4 for singular
- #33042: givaro spkg-configure: Reject too new versions
- #33092: spkg-configure.m4 for ffmpeg/imagemagick need feature checks
Change History (158)
comment:1 Changed 3 years ago by
comment:3 Changed 3 years ago by
- Cc tmonteil added
comment:4 Changed 3 years ago by
- Cc mjo added
comment:5 Changed 3 years ago by
- Cc gh-timokau added
comment:6 Changed 3 years ago by
- Description modified (diff)
comment:7 Changed 3 years ago by
- Description modified (diff)
comment:8 Changed 3 years ago by
- Description modified (diff)
There is also #27373, which concerns packages that change build/run environment, e.g. mpir/gmp/mpfr, that should be resolved in some way.
comment:9 Changed 3 years ago by
- Description modified (diff)
comment:10 Changed 3 years ago by
- Description modified (diff)
comment:11 Changed 3 years ago by
- Description modified (diff)
comment:12 Changed 3 years ago by
- Description modified (diff)
comment:13 Changed 3 years ago by
- Milestone changed from sage-8.7 to sage-pending
Removing most of the rest of my open tickets out of the 8.7 milestone, which should be closed.
comment:14 Changed 3 years ago by
- Description modified (diff)
comment:15 Changed 3 years ago by
- Description modified (diff)
comment:16 Changed 3 years ago by
- Description modified (diff)
comment:17 Changed 3 years ago by
- Description modified (diff)
comment:18 Changed 3 years ago by
- Description modified (diff)
comment:19 Changed 3 years ago by
- Cc fbissey added
- Description modified (diff)
comment:20 Changed 3 years ago by
- Description modified (diff)
comment:21 Changed 3 years ago by
- Description modified (diff)
comment:22 Changed 3 years ago by
- Description modified (diff)
comment:23 Changed 3 years ago by
- Description modified (diff)
comment:24 Changed 3 years ago by
- Description modified (diff)
comment:25 Changed 3 years ago by
- Description modified (diff)
comment:26 Changed 3 years ago by
- Description modified (diff)
comment:27 Changed 3 years ago by
- Description modified (diff)
comment:28 Changed 3 years ago by
- Description modified (diff)
comment:29 Changed 3 years ago by
- Description modified (diff)
comment:30 Changed 3 years ago by
- Description modified (diff)
comment:31 Changed 3 years ago by
- Description modified (diff)
comment:32 Changed 3 years ago by
- Description modified (diff)
comment:33 Changed 3 years ago by
- Description modified (diff)
comment:34 Changed 3 years ago by
- Description modified (diff)
comment:35 Changed 3 years ago by
- Description modified (diff)
comment:36 Changed 3 years ago by
- Description modified (diff)
comment:37 Changed 3 years ago by
- Description modified (diff)
comment:38 Changed 3 years ago by
- Description modified (diff)
comment:39 Changed 3 years ago by
- Description modified (diff)
comment:40 Changed 3 years ago by
- Description modified (diff)
comment:41 Changed 3 years ago by
- Cc isuruf added
comment:42 Changed 3 years ago by
- Description modified (diff)
comment:43 Changed 3 years ago by
- Description modified (diff)
comment:44 Changed 3 years ago by
- Description modified (diff)
comment:45 Changed 2 years ago by
- Description modified (diff)
comment:46 Changed 2 years ago by
- Description modified (diff)
comment:47 Changed 2 years ago by
- Description modified (diff)
comment:48 Changed 2 years ago by
- Description modified (diff)
comment:49 Changed 2 years ago by
- Description modified (diff)
comment:50 Changed 2 years ago by
- Description modified (diff)
comment:51 Changed 2 years ago by
- Description modified (diff)
comment:52 Changed 2 years ago by
- Description modified (diff)
comment:53 Changed 2 years ago by
- Description modified (diff)
comment:54 Changed 2 years ago by
- Description modified (diff)
comment:55 Changed 2 years ago by
- Cc mmezzarobba added
- Description modified (diff)
comment:56 Changed 2 years ago by
- Description modified (diff)
comment:57 Changed 2 years ago by
- Description modified (diff)
comment:58 Changed 2 years ago by
- Description modified (diff)
comment:59 Changed 2 years ago by
- Description modified (diff)
comment:60 Changed 2 years ago by
- Description modified (diff)
comment:61 Changed 2 years ago by
- Description modified (diff)
comment:62 Changed 2 years ago by
- Description modified (diff)
comment:63 Changed 2 years ago by
- Description modified (diff)
comment:64 Changed 2 years ago by
- Description modified (diff)
comment:65 Changed 2 years ago by
- Description modified (diff)
comment:66 Changed 2 years ago by
- Description modified (diff)
comment:67 Changed 2 years ago by
- Description modified (diff)
comment:68 Changed 2 years ago by
- Description modified (diff)
comment:69 Changed 2 years ago by
- Description modified (diff)
comment:70 Changed 2 years ago by
Nice to see so much progress on this. I'm sorry I haven't had more time to help, but I'm glad to see that the infrastructure I put in place has continued to be useful as support for more packages is added.
comment:71 Changed 2 years ago by
- Description modified (diff)
comment:72 Changed 2 years ago by
- Description modified (diff)
comment:73 Changed 2 years ago by
- Description modified (diff)
comment:74 Changed 2 years ago by
- Description modified (diff)
comment:75 Changed 2 years ago by
- Description modified (diff)
comment:76 Changed 2 years ago by
- Description modified (diff)
comment:77 Changed 2 years ago by
- Description modified (diff)
comment:78 follow-up: ↓ 79 Changed 2 years ago by
Should we find a better way to keep track of the distribution packages that provide what spkg-configure
is looking for?
Something autogenerated from new per-package information build/pkgs/*
that can keep the section in the installation manual current (src/doc/en/installation/source.rst
) - last updated by Dima?
Related: https://trac.sagemath.org/ticket/28745 (environment.yml
for conda).
comment:79 in reply to: ↑ 78 Changed 2 years ago by
Replying to mkoeppe:
Should we find a better way to keep track of the distribution packages that provide what
spkg-configure
is looking for?Something autogenerated from new per-package information
build/pkgs/*
that can keep the section in the installation manual current (src/doc/en/installation/source.rst
) - last updated by Dima?
A good idea, I only don't know how exactly to implement this. E.g. on Debian one could create a package with all dependencies (in fact, basically almost the same as their Sage package)
Related: https://trac.sagemath.org/ticket/28745 (
environment.yml
for conda).
comment:80 Changed 2 years ago by
I would just have thought to generate a shell script with one line of apt-get install
.
comment:81 Changed 2 years ago by
- Description modified (diff)
comment:82 Changed 2 years ago by
- Description modified (diff)
comment:83 Changed 2 years ago by
- Description modified (diff)
- Summary changed from Try to use as many system packages as possible to spkg-configure: Try to use as many system packages as possible
comment:84 Changed 2 years ago by
- Summary changed from spkg-configure: Try to use as many system packages as possible to Meta-ticket: spkg-configure: Try to use as many system packages as possible
comment:85 Changed 2 years ago by
- Description modified (diff)
comment:86 Changed 2 years ago by
- Description modified (diff)
comment:87 Changed 2 years ago by
- Description modified (diff)
comment:88 Changed 2 years ago by
- Description modified (diff)
comment:89 Changed 2 years ago by
- Description modified (diff)
comment:90 Changed 22 months ago by
- Description modified (diff)
comment:91 Changed 22 months ago by
- Description modified (diff)
comment:92 Changed 22 months ago by
- Description modified (diff)
comment:93 Changed 22 months ago by
- Cc thansen added
comment:94 Changed 22 months ago by
- Description modified (diff)
comment:95 Changed 22 months ago by
- Cc thansen removed
- Description modified (diff)
rw is listed (#27814), but since it depends on igraph, it would be fine to add igraph too.
comment:96 Changed 22 months ago by
- Cc thansen added
comment:97 Changed 22 months ago by
- Description modified (diff)
comment:98 Changed 22 months ago by
- Description modified (diff)
comment:99 Changed 22 months ago by
- Description modified (diff)
comment:100 Changed 22 months ago by
- Description modified (diff)
comment:101 Changed 22 months ago by
- Description modified (diff)
comment:102 Changed 22 months ago by
- Description modified (diff)
comment:103 Changed 22 months ago by
spkg-configure friends, I have drafted an update on the support for using system packages for the 9.1 release notes at https://wiki.sagemath.org/ReleaseTours/sage-9.1; let me know if anything needs changing or just make edits.
comment:104 Changed 22 months ago by
- Description modified (diff)
comment:105 Changed 21 months ago by
- Description modified (diff)
comment:106 Changed 21 months ago by
Should "BLAS" be moved to "done"?
comment:107 Changed 21 months ago by
There should be a way to use other blas implementations from the system, not only openblas
comment:108 Changed 21 months ago by
- Description modified (diff)
comment:109 Changed 21 months ago by
- Description modified (diff)
Adding a reference for giac and pynac.
comment:110 Changed 21 months ago by
- Description modified (diff)
comment:111 Changed 21 months ago by
- Description modified (diff)
Adding tickets for libhomfly and linbox.
comment:114 Changed 21 months ago by
- Description modified (diff)
comment:116 Changed 21 months ago by
- Description modified (diff)
comment:117 Changed 21 months ago by
- Description modified (diff)
comment:118 Changed 21 months ago by
- Description modified (diff)
comment:119 Changed 21 months ago by
- Description modified (diff)
comment:120 Changed 20 months ago by
- Description modified (diff)
comment:121 Changed 20 months ago by
- Description modified (diff)
comment:122 Changed 20 months ago by
- Description modified (diff)
comment:123 Changed 20 months ago by
- Description modified (diff)
comment:124 Changed 20 months ago by
- Description modified (diff)
comment:125 Changed 19 months ago by
- Description modified (diff)
comment:126 Changed 18 months ago by
- Description modified (diff)
comment:127 Changed 18 months ago by
- Description modified (diff)
comment:128 Changed 18 months ago by
- Description modified (diff)
comment:129 Changed 18 months ago by
- Description modified (diff)
comment:130 Changed 17 months ago by
- Description modified (diff)
- Milestone changed from sage-pending to sage-9.3
comment:131 Changed 17 months ago by
- Description modified (diff)
comment:132 Changed 17 months ago by
- Description modified (diff)
comment:133 Changed 17 months ago by
- Description modified (diff)
comment:134 Changed 17 months ago by
- Description modified (diff)
comment:135 Changed 16 months ago by
- Description modified (diff)
comment:136 Changed 15 months ago by
- Description modified (diff)
comment:137 Changed 15 months ago by
- Description modified (diff)
comment:138 Changed 15 months ago by
- Description modified (diff)
comment:139 Changed 14 months ago by
- Description modified (diff)
comment:140 Changed 14 months ago by
- Keywords sd111 added
comment:141 Changed 13 months ago by
- Description modified (diff)
comment:142 Changed 13 months ago by
- Description modified (diff)
comment:143 Changed 13 months ago by
- Description modified (diff)
comment:144 Changed 13 months ago by
- Description modified (diff)
comment:145 Changed 13 months ago by
- Description modified (diff)
comment:146 Changed 13 months ago by
- Cc gh-tobiasdiez added
comment:147 Changed 10 months ago by
- Description modified (diff)
comment:148 Changed 10 months ago by
- Milestone changed from sage-9.3 to sage-9.4
Sage development has entered the release candidate phase for 9.3. Setting a new milestone for this ticket based on a cursory review of ticket status, priority, and last modification date.
comment:149 Changed 7 months ago by
- Description modified (diff)
comment:150 Changed 6 months ago by
- Component changed from build to build: configure
- Description modified (diff)
comment:151 Changed 5 months ago by
- Milestone changed from sage-9.4 to sage-9.5
comment:152 Changed 5 months ago by
- Description modified (diff)
comment:153 Changed 5 months ago by
- Description modified (diff)
comment:154 Changed 4 months ago by
- Description modified (diff)
comment:155 Changed 2 months ago by
- Description modified (diff)
comment:156 Changed 8 weeks ago by
- Description modified (diff)
comment:157 Changed 5 weeks ago by
- Milestone changed from sage-9.5 to sage-9.6
comment:158 Changed 3 weeks ago by
- Description modified (diff)
Note: The list of packages in the description is not yet exhaustive; please add others that are missing as you see fit and/or reprioritize. This is just a rough sketch.