Opened 21 months ago

Last modified 4 months ago

#30930 needs_work enhancement

Add/update the system package information for the "_recommended" dummy package

Reported by: mkoeppe Owned by:
Priority: major Milestone: sage-9.7
Component: build: configure Keywords: sd111
Cc: jhpalmieri, egourgoulhon, gh-tobiasdiez, slabbe, slelievre Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: #30910, #31042 Stopgaps:

Status badges

Description (last modified by slabbe)

Follow-up from #29557 (Add script package _recommended and generate "recommended system packages" info).

From egourgoulhon #30624 and similar reports by gh-tobiasdiez, jhpalmieri #29557:

configure ends with the following recommendations:

  configure: hint: installing the following system packages is recommended and may 
avoid building some of the above SPKGs from source:
configure:   $ sudo apt-get update 
  $ sudo apt-get install texlive-generic-extra texlive-xetex latexmk pandoc dvipng 
default-jdk ffmpeg libavdevice-dev libcdd-dev libcdd-tools libec-dev eclib-tools 
libgc-dev libgiac-dev xcas pari-gp2c lcalc liblfunction-dev libnauty-dev nauty 
pari-gp2c libpari-dev pari-doc pari-elldata pari-galdata pari-galpol pari-seadata 
sympow
  • #30910 replaced texlive-generic-extra by a package that exists in modern Debian distributions.
  • #31042 addresses pandoc.
  • #29372 changed the message, showing installation hints for non-enabled optional packages separately.
  • #32956 create dummy script packages for ffmpeg and imagemagick

The message is printed regardless of whether ffmpeg et al. are installed because build/pkgs/_recommended/spkg-configure.m4 does not check anything.

To this end, tests need to be added for the presence of the various recommended packages in the system.

We do this by splitting out individual script packages, similar to the existing build/pkgs/pandoc and build/pkgs/graphviz.

Change History (33)

comment:1 Changed 21 months ago by mkoeppe

  • Cc jhpalmieri egourgoulhon gh-tobiasdiez slabbe slelievre added
  • Description modified (diff)

comment:2 Changed 21 months ago by mkoeppe

  • Description modified (diff)

comment:3 Changed 21 months ago by mkoeppe

  • Dependencies set to #30910

The debian package providing latex is updated in #30910, which I have made a dependency.

comment:4 Changed 21 months ago by mkoeppe

  • Branch set to u/mkoeppe/add_update_the_system_package_information_for_the___recommended__dummy_package

comment:5 Changed 21 months ago by mkoeppe

  • Commit set to afdd20f36102da5afe1b8708820d656b12ee7b8b

This is taking care of the issue with pandoc.


Last 10 new commits:

050dcb8tox.ini (local-sudo): Use --yes --no-install-recommends
c8fbe0btox.ini (local-sudo): Ignore errors when IGNORE_MISSING_SYSTEM_PACKAGES=yes
ff34897tox.ini (local): Guess the package system if it is not provided as a factor
e27ff19Merge branch 'u/mkoeppe/tox_ini__add_environments_local_sudo_ubuntu_standard__etc_' of git://trac.sagemath.org/sage into public/build/fixwsl
6ff45c4Cleanup workflow
5348144Replace texlive-generic-extra
0fd029fNarrow down events that trigger the workflow
3ccae02build/pkgs/_recommended/distros/debian.txt: Update latex package
6d0e673Merge branch 't/30910/public/build/fixwsl' into t/30930/add_update_the_system_package_information_for_the___recommended__dummy_package
afdd20fbuild/pkgs/_recommended/distros, src/doc/bootstrap: Remove duplication of pandoc

comment:6 Changed 21 months ago by mkoeppe

That configure keeps suggesting packages from the _recommended dummy package is because build/pkgs/_recommended/spkg-configure.m4 does not check anything.

To improve this, tests need to be added for the presence of the various recommended packages in the system. Either in build/pkgs/_recommended/spkg-configure.m4 or in individual "script packages" similar to build/pkgs/pandoc.

comment:7 Changed 21 months ago by mkoeppe

  • Authors set to Matthias Koeppe

comment:8 follow-up: Changed 21 months ago by dimpase

OK, this does the trick for pandoc. I guess ffmpeg, imagemagick, texinfo need semi-dummy packages created (i.e. with spkg-configure checking their presence, but without real spkg-install), right?

comment:9 in reply to: ↑ 8 ; follow-ups: Changed 21 months ago by mkoeppe

Replying to dimpase:

OK, this does the trick for pandoc. I guess ffmpeg, imagemagick, texinfo need semi-dummy packages created (i.e. with spkg-configure checking their presence, but without real spkg-install), right?

That's right. Likewise for libxml in #30899.

comment:10 in reply to: ↑ 9 Changed 21 months ago by dimpase

Replying to mkoeppe:

Replying to dimpase:

OK, this does the trick for pandoc. I guess ffmpeg, imagemagick, texinfo need semi-dummy packages created (i.e. with spkg-configure checking their presence, but without real spkg-install), right?

That's right. Likewise for libxml in #30899.

should this be done on this ticket, or rather elsewhere?

comment:11 Changed 21 months ago by mkoeppe

Fine to do it here if it's not holding up this ticket too long

comment:12 Changed 21 months ago by mkoeppe

  • Status changed from new to needs_review

comment:13 in reply to: ↑ 9 Changed 21 months ago by mkoeppe

Replying to mkoeppe:

Likewise for libxml in #30899.

This is now done in #27921

comment:14 Changed 21 months ago by mkoeppe

Needs review

comment:15 Changed 21 months ago by mkoeppe

  • Keywords sd111 added

Hoping we can make progress on this ticket this week - https://wiki.sagemath.org/days111

comment:16 Changed 21 months ago by slabbe

  • Status changed from needs_review to needs_work

The branch has a conflict on top of 9.3.beta3 (I confirm there is one)

comment:17 Changed 21 months ago by slabbe

  • Work issues set to branch has conflict

comment:18 Changed 21 months ago by git

  • Commit changed from afdd20f36102da5afe1b8708820d656b12ee7b8b to bdd5b9d606dafa14224fcf63092e877d1e24158d

Branch pushed to git repo; I updated commit sha1. New commits:

bdd5b9dMerge tag '9.3.beta3' into t/30930/add_update_the_system_package_information_for_the___recommended__dummy_package

comment:19 Changed 21 months ago by mkoeppe

  • Status changed from needs_work to needs_review
  • Work issues branch has conflict deleted

comment:20 follow-up: Changed 21 months ago by slabbe

  • Status changed from needs_review to needs_info

I tested the branch. make build works.

With the branch, I see the following message on my computer (Ubuntu 20.04):

    notice: the following SPKGs did not find equivalent system packages:

        _recommended boost coxeter3 gp2c igraph isl libsemigroups pari_elldata pari_galpol pari_nftables pari_seadata
        
checking for the package system in use... debian
configure:

    hint: installing the following system packages, if not
    already present, is recommended and may avoid having to
    build them (though some may have to be built anyway):

      $ sudo apt-get update 
      $ sudo apt-get install  texlive-latex-extra texlive-xetex latexmk dvipng default-jdk ffmpeg libavdevice-dev libboost-dev pari-gp2c libigraph-dev libisl-dev

    After installation, re-run configure using:

      $ ./config.status --recheck && ./config.status

And tox -e docker-archlinux-minimal -- config.status gives this:

    notice: the following SPKGs did not find equivalent system packages:

        _recommended arb boost boost_cropped brial cbc cddlib cliquer cmake coxeter3 eclib ecm fflas_ffpack flint flintqs fplll freetype gc gf2x gfan gfortran giac git givaro glpk gp2c gsl igraph iml isl lcalc libatomic_ops libbraiding libgd libhomfly libnauty libpng libsemigroups lrcalc m4ri m4rie mpfi nauty ninja_build ntl openblas palp pandoc pari pari_elldata pari_galdata pari_galpol pari_nftables pari_seadata pari_seadata_small patch perl_cpan_polymake_prereq perl_term_readline_gnu planarity ppl python3 r rw suitesparse symmetrica sympow tachyon tox zeromq zn_poly
        
checking for the package system in use... arch
configure:

    hint: installing the following system packages, if not
    already present, is recommended and may avoid having to
    build them (though some may have to be built anyway):

      $ sudo pacman -Sy
      $ sudo pacman -S  arb boost boost brial coin-or-cbc cddlib coxeter eclib fflas-ffpack flintqs gc gf2x gfan gcc-fortran libgiac giac glpk gsl igraph iml lcalc libatomic_ops libbraiding gd libhomfly lrcalc m4ri m4rie nauty ninja openblas lapack cblas palp pandoc pari pari-galdata pari-seadata pari-elldata pari-galpol pari-elldata pari-galdata pari-galpol pari-seadata pari-seadata patch perl-term-readline-gnu planarity ppl r rankwidth suitesparse symmetrica sympow tachyon zn_poly

    After installation, re-run configure using:

      $ ./config.status --recheck && ./config.status

We see that pandoc still appears above. Is this what we want?

Could we update the description of the ticket (which is too large), because I don't know what the branch is suppose to do exactly?

comment:21 in reply to: ↑ 20 Changed 21 months ago by mkoeppe

  • Branch u/mkoeppe/add_update_the_system_package_information_for_the___recommended__dummy_package deleted
  • Commit bdd5b9d606dafa14224fcf63092e877d1e24158d deleted
  • Dependencies changed from #30910 to #30910, #31042

Replying to slabbe:

Could we update the description of the ticket (which is too large), because I don't know what the branch is suppose to do exactly?

I have instead split out the pandoc part (which is the only thing that the current branch took care of) out into the separate ticket #31042.

comment:22 Changed 21 months ago by mkoeppe

  • Authors Matthias Koeppe deleted
  • Description modified (diff)
  • Status changed from needs_info to needs_work

comment:23 Changed 18 months ago by mkoeppe

  • Milestone changed from sage-9.3 to sage-9.4

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

comment:24 Changed 17 months ago by slelievre

Should _recommended really be listed in this configure notice?

    notice: the following SPKGs did not find equivalent system packages:

        _recommended cbc coxeter3 gp2c ...

comment:25 Changed 17 months ago by mkoeppe

Yes, it may look funny but at least one can see that way where the system package recommendations are coming from

comment:26 Changed 17 months ago by slelievre

Makes sense.

comment:27 Changed 15 months ago by dimpase

texinfo keeps popping up on Homebrew for some reason --- perhaps, because texinfo is not a program, there no such executable anymore, it's replaced by tex2any - but the package name remains texinfo.

comment:28 Changed 15 months ago by mkoeppe

From the ticket description:

To this end, tests need to be added for the presence of the various recommended packages in the system. Either in build/pkgs/_recommended/spkg-configure.m4 or in individual script packages similar to build/pkgs/pandoc.

comment:29 Changed 14 months ago by mkoeppe

  • Description modified (diff)

comment:30 Changed 13 months ago by mkoeppe

  • Milestone changed from sage-9.4 to sage-9.5

comment:31 Changed 9 months ago by slabbe

  • Description modified (diff)

comment:32 Changed 8 months ago by mkoeppe

  • Milestone changed from sage-9.5 to sage-9.6

comment:33 Changed 4 months ago by mkoeppe

  • Milestone changed from sage-9.6 to sage-9.7
Note: See TracTickets for help on using tickets.