Opened 8 months ago
Last modified 2 months ago
#32753 needs_review enhancement
Prefer Linux package manager over linuxbrew; add nix detection
Reported by: | gh-tobiasdiez | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | sage-9.7 |
Component: | build: configure | Keywords: | |
Cc: | mkoeppe, gh-collares, jipilab | Merged in: | |
Authors: | Matthias Koeppe | Reviewers: | |
Report Upstream: | N/A | Work issues: | |
Branch: | u/mkoeppe/prefer_linux_package_manager_over_linuxbrew (Commits, GitHub, GitLab) | Commit: | 9af33a1b8678fc351413a1fa67daceb7b9439161 |
Dependencies: | Stopgaps: |
Description (last modified by )
If brew is installed on Linux, Sagemath thinks this is the primary package manager and, e.g., configure
suggests to run brew ...
to install new packages. However, linuxbrew is untested as a platform for Sage, and so this is not a good idea.
We change build/bin/sage-guess-package-system
so that brew
is only tested after the Linux package managers.
We also add a detection for nix-env
.
Change History (24)
comment:1 Changed 6 months ago by
- Milestone changed from sage-9.5 to sage-9.6
comment:2 Changed 5 months ago by
comment:3 Changed 5 months ago by
- Branch set to u/mkoeppe/prefer_linux_package_manager_over_linuxbrew
comment:4 Changed 5 months ago by
- Cc gh-collares added
- Commit set to 90f7ced00b74cca051c167c271a9a49155ad0710
- Description modified (diff)
- Status changed from new to needs_review
- Summary changed from Prefer Linux package manager over linuxbrew to Prefer Linux package manager over linuxbrew; add nix detection
comment:5 follow-ups: ↓ 6 ↓ 7 ↓ 8 Changed 5 months ago by
To be honest with you, I have never used nix-env
(the manual offers a declarative path and a nix-env
path, and strongly suggests nix-env
is the worst of the two alternatives). I think supporting nix-env
is valid, but given my lack of experience with it I am not sure what the workflow would be in this case. You can certainly install packages with nix-env
in your "default profile", and the executables would be in PATH
without having to activate an environment.
People who follow the declarative approach and want to build Sage from source would bypass nix-env
and write a derivation (a declarative recipe, basically) to pull in the required dependencies, and then pass this to nix-shell
to obtain an ephemeral virtualenv
-like environment with those deps available. Or, instead of writing the derivation themselves, they could also use Nixpkgs's Sage derivation to get the same build environment we use. Either way, they would be able to build Sage from Git sources using Nix packages for the dependencies, but nix-env
would be none the wiser.
It would be useful to know a few things so I can understand how this fits into the Nix model: Is it just for printing a command to install dependencies? Or does it query installed packages as well? What happens if Singular (say) just exists on PATH
but nix-env
doesn't report it as installed? Pointing me to the relevant source file would be enough, no need to answer in detail. Thanks!
comment:6 in reply to: ↑ 5 Changed 5 months ago by
Replying to gh-collares:
People who follow the declarative approach and want to build Sage from source would bypass
nix-env
and write a derivation (a declarative recipe, basically) to pull in the required dependencies, and then pass this tonix-shell
to obtain an ephemeralvirtualenv
-like environment with those deps available. Or, instead of writing the derivation themselves, they could also use Nixpkgs's Sage derivation to get the same build environment we use.
It would be great to have instructions how to do Sage development using nixpkgs in our installation manual.
Other than figuring out how to provision a nix environment with packages on top of the nixos/nix
Docker image (so I can say tox -e docker-nixos-standard
), I have no experience with nix.
By the way, I am getting the error:
Step 6/46 : RUN nix-env --install gd iml m4ri gfan singular mpfi cmake symmetrica flintqs freetype cddlib libbraiding brial ppl sympow zn_poly readline R sqlite tachyon giac m4rie blas lapack ntl glpk fflas-ffpack libmpc gsl boost nauty binutils gnumake gnum4 perl python3 gnutar bc gcc bash flint arb pari cliquer libhomfly openssl gengetopt planarity lrcalc palp ecm givaro ecl eclib lcalc rankwidth qhull gettext autoconf automake libtool pkg-config ---> Running in 0dcbda1f2003 error: selector 'libhomfly' matches no derivations
so it may be that some work on the nix.txt
files is needed.
comment:7 in reply to: ↑ 5 Changed 5 months ago by
Replying to gh-collares:
It would be useful to know a few things so I can understand how this fits into the Nix model: Is it just for printing a command to install dependencies?
Yes, it's for the system package advice that is printed at the end of a configure
run.
The code for this is in build/bin/sage-print-system-package-command
.
comment:8 in reply to: ↑ 5 Changed 5 months ago by
Replying to gh-collares:
What happens if Singular (say) just exists on
PATH
configure
only checks what is available in PATH
.
For packages where we don't find a system package in this way, if there is system package information in build/pkgs/*/nix.txt
for the package, we issue the system package advice. No package manager queries are done -- this is all left to the user.
(This is a design principle for this code -- it does not attempt to take care of system package installation tasks for the user; instead it only informs/educates the user about possible commands.)
comment:9 follow-up: ↓ 12 Changed 5 months ago by
Thanks for the information, this is very helpful! So, yeah, I think recommending nix-env
is fine, since people who use the declarative approach can just copy and paste the list of packages into their derivations.
As for the libhomfly error: The preferred way to install packages using nix-env
is nix-env -f '<nixpkgs>' -iA m4ri libhomfly
(say). The -A
flag specifies a saner way of finding the relevant package; in this case, the -A
-less version caught a typo in the Nix package. I've submitted a PR to fix it in Nixpkgs, but ideally the nix-env
invocation would be switched to something like the command I mentioned above.
I will try to write some docs for Sage development with nixpkgs as part of the Sage 9.6 release.
comment:10 Changed 5 months ago by
- Commit changed from 90f7ced00b74cca051c167c271a9a49155ad0710 to e28862883054256363cae86d6c4dfaad0fb844ca
Branch pushed to git repo; I updated commit sha1. New commits:
e288628 | build/bin/sage-print-system-package-command (nix): Use nix-env --install --attr
|
comment:11 Changed 5 months ago by
Not sure what the -f '<nixpkgs>'
is - is there something I need to add?
comment:12 in reply to: ↑ 9 Changed 5 months ago by
Replying to gh-collares:
The
-A
flag specifies a saner way of finding the relevant package
Thanks, this helped!
comment:13 Changed 5 months ago by
configure
now gives the following advice:
checking for the package system in use... nix configure: hint: installing the following system packages, if not already present, is recommended and may avoid having to build them (though some may have to be built anyway): $ nix-env --install --attr ecl eclib fflas-ffpack giac givaro gsl iml lcalc libhomfly blas lapack palp pari R configure: hint: installing the following system packages, if not already present, may provide additional optional features: $ nix-env --install --attr ffmpeg graphviz imagemagick texinfo graphviz libxml2 lrs pdf2svg polymake
comment:14 Changed 5 months ago by
This is perfect, many thanks! I don't know why I needed the -f <nixpkgs>
here, perhaps because I am on NixOS? It's best to leave it out if it works outside NixOS.
comment:15 Changed 5 months ago by
- Commit changed from e28862883054256363cae86d6c4dfaad0fb844ca to 3d96dc560bd838feda56b26ccc48bf90cd641fd2
Branch pushed to git repo; I updated commit sha1. New commits:
3d96dc5 | build/bin/sage-print-system-package-command, build/bin/write-dockerfile.sh: Use nix-env -f '<nixpkgs>' --install --attr
|
comment:16 Changed 5 months ago by
OK, looks like I do need this switch
comment:17 Changed 5 months ago by
- Commit changed from 3d96dc560bd838feda56b26ccc48bf90cd641fd2 to b07fece963b3a656f1a1be10635f8c24416edf04
Branch pushed to git repo; I updated commit sha1. New commits:
b07fece | build/pkgs/gfortran/distros/nix.txt: New
|
comment:18 Changed 5 months ago by
Now I am getting
building '/nix/store/r2w1z0912rilh4l1yyg9jv3d66bnjk99-user-environment.drv'... error: packages '/nix/store/9iry5bm9cxdbspap6qi835g67kvdninx-boost-1.77.0/lib/libboost_thread.so' and '/nix/store/6x1m911w8378qbxjf9pcli9hijrqx1km-nix-2.6.0/lib/libboost_thread.so' have the same priority 5; use 'nix-env --set-flag priority NUMBER INSTALLED_PKGNAME' to change the priority of one of the conflicting packages (0 being the highest priority) error: builder for '/nix/store/r2w1z0912rilh4l1yyg9jv3d66bnjk99-user-environment.drv' failed with exit code 1 The command '/bin/sh -c nix-env -f '<nixpkgs>' --install --attr gd iml m4ri gfan gfortran singular mpfi cmake symmetrica flintqs freetype cddlib libbraiding brial ppl sympow zn_poly readline R sqlite tachyon giac m4rie blas lapack ntl glpk fflas-ffpack libmpc gsl boost nauty binutils gnumake gnum4 perl python3 gnutar bc gcc bash flint arb pari cliquer libhomfly openssl gengetopt planarity lrcalc palp ecm givaro ecl eclib lcalc libatomic_ops rankwidth qhull gettext autoconf automake libtool pkg-config' returned a non-zero code: 100
comment:19 Changed 5 months ago by
I've never seen this error and I am not exactly sure how to fix it. Can we just remove Boost's nix.txt
, as a workaround? Hopefully there are no other conflicts like this.
comment:20 Changed 5 months ago by
- Commit changed from b07fece963b3a656f1a1be10635f8c24416edf04 to 9af33a1b8678fc351413a1fa67daceb7b9439161
Branch pushed to git repo; I updated commit sha1. New commits:
9af33a1 | build/pkgs/boost_cropped/distros/nix.txt: Remove
|
comment:21 Changed 5 months ago by
That worked, thanks. Next error:
building '/nix/store/4vmcjc3jqzvzsy1mvydsl6k5glbbvvvm-user-environment.drv'... error: packages '/nix/store/cm5d8zx8jf24f6vjxzdx9by3k6rn5zd2-gfortran-wrapper-9.3.0-man/share/man/man7/fsf-funding.7.gz' and '/nix/store/310zm0mxwmk7ynsk0qjb7hiaf734s8p8-gcc-wrapper-10.3.0-man/share/man/man7/fsf-funding.7.gz' have the same priority 10; use 'nix-env --set-flag priority NUMBER INSTALLED_PKGNAME' to change the priority of one of the conflicting packages (0 being the highest priority) error: builder for '/nix/store/4vmcjc3jqzvzsy1mvydsl6k5glbbvvvm-user-environment.drv' failed with exit code 1 The command '/bin/sh -c nix-env -f '<nixpkgs>' --install --attr gd iml m4ri gfan gfortran singular mpfi cmake symmetrica flintqs freetype cddlib libbraiding brial ppl sympow zn_poly readline R sqlite tachyon giac m4rie blas lapack ntl glpk fflas-ffpack libmpc gsl nauty binutils gnumake gnum4 perl python3 gnutar bc gcc bash flint arb pari cliquer libhomfly openssl gengetopt planarity lrcalc palp ecm givaro ecl eclib lcalc libatomic_ops rankwidth qhull gettext autoconf automake libtool pkg-config' returned a non-zero code: 100
comment:22 Changed 5 months ago by
- Status changed from needs_review to needs_info
comment:23 Changed 3 months ago by
- Milestone changed from sage-9.6 to sage-9.7
comment:24 Changed 2 months ago by
- Cc jipilab added
- Status changed from needs_info to needs_review
Perhaps good enough for merging?
I now see this in our gitpod.