Opened 5 years ago

Last modified 4 months ago

#21700 needs_work enhancement

Meta-ticket: Declare "optional" dependencies of packages (a package manager's suggested/recommended packages)

Reported by: mkoeppe Owned by:
Priority: major Milestone: sage-9.4
Component: build: configure Keywords:
Cc: tscrim, dimpase, vdelecroix, jhpalmieri, slelievre Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description (last modified by mkoeppe)

Various packages can be built with or without the presence of some other packages.

For example, the latte_int configure script detects lrslib and lidia if they are installed, and they provide additional features. lidia is quite a big package, and so it is not clear whether it should be a dependency. (Linux package managers have the notion of suggested or recommended packages for this situation.)

GNU make's "order-only dependencies" "sound" as if they can do this, but as Jeroen explains, they can't.

Of course, the user can just work around this issue by doing sage -i lidia && sage -i latte_int (or similar invocations such as make lidia latte_int, which are sequential from left to right thanks to our top-level Makefile), but we must find a proper solution that ensures reproducible (deterministic) builds when a parallelized make is used. In particular, this is relevant after #21538, which provides ./configure --enable-lidia --enable-latte_int && MAKE=-j12 make, for which the sequential workaround is not available.

This feature can be implemented by Makefile trickery, of course. See #30560.

Tickets:

  • #30560: openssl an optional dependency of python3
  • giac detects cocoa (which we have added as an experimental package)
  • pynac is able to use giac if present (but we currently explicitly disable its use)
  • latte_int is able to use lidia for one subroutine -- a huge and unmaintained package; currently it is a normal dependency
  • polymake has lots of optional dependencies

Change History (24)

comment:1 Changed 5 years ago by mkoeppe

  • Branch set to u/mkoeppe/latte_int__add_order_only_dependencies

comment:2 Changed 5 years ago by mkoeppe

  • Authors set to Matthias Koeppe
  • Cc tscrim dimpase vdelecroix added
  • Commit set to cb50fb43ad0a2481489ac42bc0fcc1b5ebf7462c
  • Status changed from new to needs_review

New commits:

cb50fb4latte_int: Add order-only dependencies

comment:3 Changed 5 years ago by git

  • Commit changed from cb50fb43ad0a2481489ac42bc0fcc1b5ebf7462c to 0480c3486f12f76f2bf7200799676774a3f07fda

Branch pushed to git repo; I updated commit sha1. New commits:

0480c34latte_int: Update SPKG.txt

comment:4 Changed 5 years ago by dimpase

  • Reviewers set to Dima Pasechnik
  • Status changed from needs_review to positive_review

comment:5 Changed 5 years ago by jdemeyer

  • Status changed from positive_review to needs_work
  • Summary changed from latte_int: Add order-only dependencies to latte_int: Add dependencies

Given that lrslib and lidia are libraries, they should be real dependencies.

comment:6 follow-up: Changed 5 years ago by mkoeppe

They are optional for latte_int. This means that one can build latte_int with or without them.

comment:7 in reply to: ↑ 6 Changed 5 years ago by jdemeyer

Replying to mkoeppe:

They are optional for latte_int. This means that one can build latte_int with or without them.

That is an argument for not adding those dependencies. It's not an argument for adding them as order-only dependencies.

comment:8 follow-up: Changed 5 years ago by mkoeppe

If the user types "make latte_int lidia", then that user should get a latte built with lidia. Hence, an order only dependency.

comment:9 in reply to: ↑ 8 Changed 5 years ago by jdemeyer

Replying to mkoeppe:

If the user types "make latte_int lidia", then that user should get a latte built with lidia. Hence, an order only dependency.

That works with a real dependency as well as an order-only dependency, so it's still not an argument for an order-only dependency.

comment:10 Changed 5 years ago by jdemeyer

I see what you want now, but what you want doesn't work. There is no way to tell make that something should be a dependency only if it's installed. An order-only dependency is really the opposite of that: an order-only dependency is a dependency if it's not installed.

comment:11 follow-up: Changed 5 years ago by mkoeppe

If the user types "make latte_int" and lidia is not installed, then lidia should not be installed. Hence an order only dependency, not a real dependency.

comment:12 Changed 5 years ago by mkoeppe

I think you may be right, but it's past midnight here, so I can't be sure.

comment:13 in reply to: ↑ 11 Changed 5 years ago by jdemeyer

Replying to mkoeppe:

If the user types "make latte_int" and lidia is not installed, then lidia should not be installed. Hence an order only dependency, not a real dependency.

Like I said in 10, order-only dependencies don't work that way.

comment:14 Changed 5 years ago by mkoeppe

Thanks. You are right. I was confused about order-only dependencies.

I'll update the ticket description

comment:15 Changed 5 years ago by mkoeppe

  • Authors Matthias Koeppe deleted
  • Branch u/mkoeppe/latte_int__add_order_only_dependencies deleted
  • Commit 0480c3486f12f76f2bf7200799676774a3f07fda deleted
  • Description modified (diff)
  • Milestone changed from sage-7.4 to sage-wishlist
  • Reviewers Dima Pasechnik deleted
  • Summary changed from latte_int: Add dependencies to Packages with "optional" dependencies (a package manager's suggested/recommended packages)

comment:16 Changed 5 years ago by mkoeppe

I suppose a simple solution could be to have two packages:

  • latte_int (which perhaps disables lrslib and lidia detection in the name of reproducibility of the build) and latte_int_full (which has lrslib and lidia as dependencies).
  • and likewise polymake (a minimal configuration) and polymake_full (which pulls in a dozen packages).

comment:17 follow-up: Changed 5 years ago by jdemeyer

Why not add lrslib and lidia unconditionally as dependencies of latte_int?

comment:18 in reply to: ↑ 17 Changed 4 years ago by tmonteil

Replying to jdemeyer:

Why not add lrslib and lidia unconditionally as dependencies of latte_int?

See #22491 for such an implementation (it does not pretend to address the general issue mentioned in the current ticket).

According to Matthias, one reason for not doing that could be that lidia is rather big and long to compile.

comment:19 Changed 9 months ago by mkoeppe

  • Description modified (diff)

comment:20 Changed 9 months ago by mkoeppe

  • Cc jhpalmieri slelievre added
  • Component changed from packages: experimental to build: configure
  • Description modified (diff)
  • Milestone changed from sage-wishlist to sage-9.3
  • Summary changed from Packages with "optional" dependencies (a package manager's suggested/recommended packages) to Meta-ticket: Declare "optional" dependencies of packages (a package manager's suggested/recommended packages)

comment:21 in reply to: ↑ description Changed 9 months ago by jhpalmieri

Regarding this part of the ticket description:

Replying to mkoeppe:

Of course, the user can just work around this issue by doing sage -i lidia && sage -i latte_int, but we must find a proper solution that ensures reproducible (deterministic) builds when a parallelized make is used.

It is better to do make lidia && make latte_int since sage -i lidia will build lidia and everything after it. (So if you have a fresh tarball, sage -i lidia will build all of Sage). So sage -i lidia && sage -i latte_int may cause some packages to be built twice.

(This isn't questioning the main point, that the order in which these are built is important and not taken care of with the current build system.)

comment:22 Changed 9 months ago by mkoeppe

  • Description modified (diff)

Thanks, I have updated the ticket description

comment:23 Changed 9 months ago by mkoeppe

  • Description modified (diff)

comment:24 Changed 4 months ago by mkoeppe

  • Milestone changed from sage-9.3 to sage-9.4

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

Note: See TracTickets for help on using tickets.