Meta-ticket: Declare "optional" dependencies of packages (a package manager's suggested/recommended packages)

[mkoeppe]

Various packages can be built with or without the presence of some other packages.

For example, the latte_int configure script detects lrslib and lidia if they are installed, and they provide additional features. lidia is quite a big package, and so it is not clear whether it should be a dependency. (Linux package managers have the notion of suggested or recommended packages for this situation.)

GNU make's "order-only dependencies" "sound" as if they can do this, but as Jeroen explains, they can't.

Of course, the user can just work around this issue by doing sage -i lidia && sage -i latte_int (or similar invocations such as make lidia latte_int, which are sequential from left to right thanks to our top-level Makefile), but we must find a proper solution that ensures reproducible (deterministic) builds when a parallelized make is used. In particular, this is relevant after #21538, which provides ./configure --enable-lidia --enable-latte_int && MAKE=-j12 make, for which the sequential workaround is not available.

This feature can be implemented by Makefile trickery, of course. See #30560.

Tickets:

• #30560: openssl an optional dependency of python3

• giac detects cocoa (which we have added as an experimental package)

• pynac is able to use giac if present (but we currently explicitly disable its use)

• latte_int is able to use lidia for one subroutine -- a huge and unmaintained package; currently it is a normal dependency

• polymake has lots of optional dependencies

[mkoeppe]

New commits:

​cb50fb4

latte_int: Add order-only dependencies

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​0480c34

latte_int: Update SPKG.txt

[jdemeyer]

Given that lrslib and lidia are libraries, they should be real dependencies.

[mkoeppe]

They are optional for latte_int. This means that one can build latte_int with or without them.

[jdemeyer]

Replying to mkoeppe:

They are optional for latte_int. This means that one can build latte_int with or without them.

That is an argument for not adding those dependencies. It's not an argument for adding them as order-only dependencies.

[mkoeppe]

If the user types "make latte_int lidia", then that user should get a latte built with lidia. Hence, an order only dependency.

[jdemeyer]

Replying to mkoeppe:

If the user types "make latte_int lidia", then that user should get a latte built with lidia. Hence, an order only dependency.

That works with a real dependency as well as an order-only dependency, so it's still not an argument for an order-only dependency.

[jdemeyer]

I see what you want now, but what you want doesn't work. There is no way to tell make that something should be a dependency only if it's installed. An order-only dependency is really the opposite of that: an order-only dependency is a dependency if it's not installed.

[mkoeppe]

If the user types "make latte_int" and lidia is not installed, then lidia should not be installed. Hence an order only dependency, not a real dependency.

[mkoeppe]

I think you may be right, but it's past midnight here, so I can't be sure.

[jdemeyer]

Replying to mkoeppe:

If the user types "make latte_int" and lidia is not installed, then lidia should not be installed. Hence an order only dependency, not a real dependency.

Like I said in 10, order-only dependencies don't work that way.

[mkoeppe]

Thanks. You are right. I was confused about order-only dependencies.

I'll update the ticket description

[mkoeppe]

I suppose a simple solution could be to have two packages:

• latte_int (which perhaps disables lrslib and lidia detection in the name of reproducibility of the build) and latte_int_full (which has lrslib and lidia as dependencies).
• and likewise polymake (a minimal configuration) and polymake_full (which pulls in a dozen packages).

[jdemeyer]

Why not add lrslib and lidia unconditionally as dependencies of latte_int?

[tmonteil]

Replying to jdemeyer:

Why not add lrslib and lidia unconditionally as dependencies of latte_int?

See #22491 for such an implementation (it does not pretend to address the general issue mentioned in the current ticket).

According to Matthias, one reason for not doing that could be that lidia is rather big and long to compile.

[jhpalmieri]

Regarding this part of the ticket description:

Replying to mkoeppe:

Of course, the user can just work around this issue by doing sage -i lidia && sage -i latte_int, but we must find a proper solution that ensures reproducible (deterministic) builds when a parallelized make is used.

It is better to do make lidia && make latte_int since sage -i lidia will build lidia and everything after it. (So if you have a fresh tarball, sage -i lidia will build all of Sage). So sage -i lidia && sage -i latte_int may cause some packages to be built twice.

(This isn't questioning the main point, that the order in which these are built is important and not taken care of with the current build system.)

[mkoeppe]

Thanks, I have updated the ticket description

[mkoeppe]

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

[mkoeppe]

Setting a new milestone for this ticket based on a cursory review.