Opened 7 months ago
Last modified 2 weeks ago
#30371 needs_review enhancement
Inplace (editable) installs of sagelib in a venv
Reported by:  mkoeppe  Owned by:  

Priority:  major  Milestone:  sage9.3 
Component:  build  Keywords:  sd111 
Cc:  ghtobiasdiez, fbissey, jhpalmieri, isuruf  Merged in:  
Authors:  Tobias Diez  Reviewers:  
Report Upstream:  N/A  Work issues:  
Branch:  public/build/inplace (Commits, GitHub, GitLab)  Commit:  fec55234a00e46d023ad583f8c6284b970141ab0 
Dependencies:  #30779, #30672, #30673, #30709, #30706, #30901, #31080, #29850, #29314, #30937, #30935, #30731 , #30859, #30923, #30910, #30944, #30770, #31216, #31029, #30912, #31357, #31362, #31365, #31377  Stopgaps: 
Description (last modified by )
We modify src/setup.py
to support editable installs via pipenv (pipenv install e .
, which invokes pip install editable
). Since distribution continues using build/pkgs/sagelib/src/setup.py, the changes to src/setup.py made in this ticket do not affect the existing build process of the Sage library.
With this ticket, the following code works:
cd src mkdir .venv pipenv install pipenv run sageruntests all
It generates a virtual env in src/.venv
that contains all necessary libraries, and an editable install of sage.
Note: This procedure might not work out of the box, and a few manual steps are necessary to install the required libraries (see below for instructions for Ubuntu 20.04 in WSL, and the github action for Ubuntu 20.10).
The output of the added GH workflow is available here: https://github.com/sagemath/sagetracmirror/actions?query=workflow%3A%22Build+%26+Test+using+pipenv%22+branch%3Apublic%2Fbuild%2Finplace Most tests are working, but for some reason after more or less exactly 2.5h of execution time the doctest run out of memory. This still needs investigation. #31341
TODO (as followup tickets):
 Add documentation on how to set up the virtual environment, using either plain
python m venv
,virtualenv
(20.x), orpipenv
. This is #31342.  Consider moving package dependencies from pipfile to install_requires in
src/setup.py
. Let's discuss this as part of #30913.  Consider not commit pipfile.lock but instead generate it as build artifact on ci (so that build errors can be easily reproduced by downloading the lock file, and recreating the exact venv locally): https://github.com/pypa/pipenv/issues/1911#issuecomment378989597. However, normally you want to put the lock file under version control for the reasons described in https://trac.sagemath.org/timeline?from=20210203T13%3A15%3A22Z. The official statement is "always include in version control", see https://github.com/pypa/pipenv/issues/598.
 Investigate what needs to be installed on macOS to have a successful build, and add these steps to the github action. (I don't have a mac, so I'll not work on this.) Then fix MacOS github action.
Previous tickets on inplace builds, editable installs:
Change History (380)
comment:1 Changed 7 months ago by
 Branch set to public/build/inplace
 Commit set to bc6e2726cb87273de0c92297156dccf4b5107f06
comment:2 Changed 7 months ago by
I don't really have time to look into this before the 9.2 release. But if you are looking for where include directories are determined, take a look at sage_setup.command.sage_build_cython
comment:3 Changed 7 months ago by
 Commit changed from bc6e2726cb87273de0c92297156dccf4b5107f06 to 58452ce79c2d8863f139e72d75ec7b0c5904a989
Branch pushed to git repo; I updated commit sha1. New commits:
58452ce  Make inplace compilation work

comment:4 Changed 7 months ago by
The command ./sage src/setup.py build_ext inplace
runs now fine for me. I had to change quite a few things to make it work, and found it easier to completely start over using cythonize
instead of using the existing build_cython
code. The resulting code is a minimal working version. Please let me know which features of the existing build_cython
file you would like to keep (to be honest, I don't really understand what some of the code there is doing, and why its needed).
The code definitely needs cleanup, but otherwise is ready for a first round of review.
comment:5 Changed 7 months ago by
 Status changed from new to needs_review
comment:6 Changed 7 months ago by
 Commit changed from 58452ce79c2d8863f139e72d75ec7b0c5904a989 to 1a99228dafea2b7b676e1754b3d4b43f794ec7a7
Branch pushed to git repo; I updated commit sha1. New commits:
1a99228  Update gitignore

comment:7 Changed 7 months ago by
 a/.gitignore +++ b/.gitignore @@ 99,7 +99,18 @@ gitlabbuilddocker.log /src/build /src/Makefile /src/bin/sageenvconfig /build/bin/sagebuildenvconfig +/build
this is not good. Our /build/
contains important files of the sage distribution
comment:8 Changed 7 months ago by
And the changes removing old python2 code really need to go on a separate ticket
comment:9 Changed 7 months ago by
 Commit changed from 1a99228dafea2b7b676e1754b3d4b43f794ec7a7 to 22b27a051b7f1a429b7f91f965b33154ee776aea
Branch pushed to git repo; I updated commit sha1. New commits:
22b27a0  Remove global gitignore of build folder

comment:10 followup: ↓ 11 Changed 7 months ago by
Ok, I was under the assumption that the build folder is for build artifacts. If this would be the case, then I suppose it would be better to whitelist those files in the build folder that needs to be checkedin. I've now replaced the global build ignore, by a more finegrained statement which only ignores the "temporary" files.
Concerning the changes about old Python2 code, I've created #30397.
comment:11 in reply to: ↑ 10 Changed 7 months ago by
Replying to ghtobiasdiez:
I was under the assumption that the build folder is for build artifacts.
No, not at all.
comment:12 Changed 7 months ago by
comment:13 followup: ↓ 14 Changed 7 months ago by
I wasn't aware of your changes in #28925. I guess what I'm proposing is a different strategy for the source discovery. The approach in #28925 is to change the source files themselves and enrich them by some form of metadata that decides in which distribution / package they land. Here I was proposing to specify the distribution of the files in the setup.py file. I guess both strategies have pros and cons. The main reason for me was that it seems the responsibility of the setup.py file to construct the distribution; why should the source file need to care about where they are used/included? I.e classical example of separation of concerns. I think, it's also the more flexible approach since it's easy to include a source file into multiple distributions etc. Moreover, you also gain a bit of performance at build time since you don't need to parse the files for some magic comments. What do you think?
comment:14 in reply to: ↑ 13 ; followup: ↓ 15 Changed 7 months ago by
Replying to ghtobiasdiez:
[...] Here I was proposing to specify the distribution of the files in the setup.py file. I guess both strategies have pros and cons. The main reason for me was that it seems the responsibility of the setup.py file to construct the distribution; why should the source file need to care about where they are used/included? I.e classical example of separation of concerns. [...]
One more thing to be aware of is that (1) setup.py
by itself actually is not able to control what goes into the source distribution. In #29865 (where I prepare some experimental subset distributions), I additionally use MANIFEST.in
to control that.
Separation of concerns is, of course, desirable. But so is locality: (2) My plan for splitting up the distribution is driven to a large extent by the dependencies on different sets of C/C++ libraries. And these are already encoded in Cython distutils directives.
Also I have explored a little bit the packaging options beyond using setuptools (in part motivated because of the shortcoming (1)). See #29854 (flit), #29810 (poetry). Unfortunately currently there does not seem to be a proper mainstream solution for the source layout that we want to keep (a monolithic source tree in src/sage
). For example, flit
(and even pip
outofdirectory installs) do not handle symbolic links well. In the long term, I was hoping to transform sage_setup
to implement the PEP 517 API directly (see #29845). That's another reason why I would be reluctant to hardcode distribution information in setup.py
.
In any case, I am hoping to have a broader discussion about all of this on sagedevel at the beginning of the 9.3 cycle.
comment:15 in reply to: ↑ 14 ; followups: ↓ 16 ↓ 17 Changed 7 months ago by
Separation of concerns is, of course, desirable. But so is locality: (2) My plan for splitting up the distribution is driven to a large extent by the dependencies on different sets of C/C++ libraries. And these are already encoded in Cython distutils directives.
One could also take this as an argument to remove the distutil dependency directives and put them into the setup.py, giving you locality and separation of concerns.
Anyway, my goal with this PR is not to redesign or modularize the build. I just wanted to have a working inplace build. For this moving the distribution declaration to setup.py was the easiest option, and I quite liked this also from a conceptual point of view. But if the current way is preferred, I change it back of course. Anything else that needs to be changed?
comment:16 in reply to: ↑ 15 Changed 7 months ago by
Replying to ghtobiasdiez:
One could also take this as an argument to remove the distutil dependency directives and put them into the setup.py, giving you locality and separation of concerns.
Ha, no, we just moved them there. Getting rid of the messy file modules_list.py
was an achievement in the present cycle. See #29706
comment:17 in reply to: ↑ 15 Changed 7 months ago by
Replying to ghtobiasdiez:
Anything else that needs to be changed?
I haven't really had a chance to look at it in more detail yet, sorry. Soon...
comment:18 Changed 6 months ago by
 Commit changed from 22b27a051b7f1a429b7f91f965b33154ee776aea to 55d00d7f055628848bbdeef9075b8574c44316b2
Branch pushed to git repo; I updated commit sha1. New commits:
55d00d7  Merge branch 'develop' of git://trac.sagemath.org/sage into public/build/inplace

comment:19 Changed 6 months ago by
 Dependencies set to #30397
comment:20 Changed 6 months ago by
 Commit changed from 55d00d7f055628848bbdeef9075b8574c44316b2 to 40f35033ce241f8a6bf7fe2307a8c8b946c981e1
Branch pushed to git repo; I updated commit sha1. New commits:
8404a5d  Remove old python 2 code from cython files

e9c06e3  Readd constants in build script

811b78c  Remove unused import

32daec5  Merge branch 'public/build/removeOldCode' of git://trac.sagemath.org/sage into public/build/inplace

40f3503  Fix merge problem

comment:21 Changed 6 months ago by
 Commit changed from 40f35033ce241f8a6bf7fe2307a8c8b946c981e1 to 851c557ccb33c61c78b005a20b908721deaf31a0
Branch pushed to git repo; I updated commit sha1. New commits:
851c557  Merge branch 'develop' of git://github.com/sagemath/sage into public/build/inplace

comment:22 Changed 6 months ago by
 Commit changed from 851c557ccb33c61c78b005a20b908721deaf31a0 to 1899dda1def19c3d7d518a5b4ccce67738415611
Branch pushed to git repo; I updated commit sha1. New commits:
1899dda  Remove unnessary changes

comment:23 Changed 6 months ago by
 Commit changed from 1899dda1def19c3d7d518a5b4ccce67738415611 to 74c9f7d2a328df4a2f436d3523daa12dddf655a6
Branch pushed to git repo; I updated commit sha1. New commits:
74c9f7d  Cleanup code

comment:24 Changed 6 months ago by
I've now changed the code to use the % sage_setup: distribution = xyz
way. Moreover, I've cleanup up the code a bit.
Before I continue removing the code that is no longer needed, I would like to ask you which features of the current cythonize build script are missing from my new minimalistic implementation and should be reimplemented. (As I've said above, I don't really understand some of the code in build_cython
.
comment:25 Changed 6 months ago by
These changes:
# distutils: sources = sage/libs/ppl_shim.cc +# distutils: sources = src/sage/libs/ppl_shim.cc
are not good. The root of the sagelib source tree is src
, and paths are relative to this  this is where setup.py
lives, after all.
As a way forward, I would suggest to actually work on #29864 instead of the present ticket. #29864 is a step in the modularization task (#29705), creating two new distributions  sagecore
and sagetdlib
. sagecore
is like sagelib
minus sagetdlib
and without the fancy incremental installation tricks.
comment:26 Changed 6 months ago by
 Status changed from needs_review to needs_work
comment:27 followup: ↓ 28 Changed 6 months ago by
Thanks for the feedback. I'll try to find a way to specify the paths relative to src
. I think my problem there was that setup.py
is invoked from the central sage
script, which sits in the root and not in the src folder.
Can you please explain the connection to #29864 in detail? For my the main point of this ticket is that I can compile the cython files to obtain compiled so
files in the same folder as the original cython file (e.g. src/sage/rings
). This is needed so that VS code can pick up these dependencies, resolve the imports, provide intellisense for it etc. My longterm goal would be to have a normal python installation (central or virtual env), point VS towards this runtime, and be able to use the sage code (in /src) without the need to use make
at any point. How are #29864 and #29705 helping for this?
comment:28 in reply to: ↑ 27 Changed 6 months ago by
Replying to ghtobiasdiez:
Can you please explain the connection to #29864 in detail?
Package sagecore
created in #29864 will be a version of sagelib with a plain setuptools build system, intended for (modular) deployment of the Sage library on PyPI. Since setuptools supports setup.py develop
, this has synergy with your goal.
And we will be able to work on the build system of sagecore
, across several tickets, without having to worry about breaking sagelib
builds  or what current users/developers are expecting of it.
comment:29 followup: ↓ 30 Changed 6 months ago by
That make sense, thanks for the explanation.
The only thing that I still don't understand is how the modularization will help with the compilation of cython files. If if everything is modularized, I guess the setup.py will still be used for this. And the point of this ticket is to improve the cyhonization...
comment:30 in reply to: ↑ 29 Changed 6 months ago by
Replying to ghtobiasdiez:
The only thing that I still don't understand is how the modularization will help with the compilation of cython files. If if everything is modularized, I guess the setup.py will still be used for this.
Simply by having a separate setup.py
in build/pkgs/sagecore/src
.
comment:31 Changed 6 months ago by
 Dependencies changed from #30397 to #29500, #30527
 Description modified (diff)
 Summary changed from Inplace installs with "./sage pip install editable src/" to Inplace (editable) installs of sagelib in a venv
comment:32 Changed 6 months ago by
Slight change of plans, forget about sagecore
.
comment:33 Changed 6 months ago by
 Commit changed from 74c9f7d2a328df4a2f436d3523daa12dddf655a6 to 5bda3321b88289cbd3f0dfe97308b07dd9dd1459
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
0a64674  build/pkgs/gambit/spkginstall.in: Install via bdist_wheel

ca58693  build/pkgs/pillow/spkginstall.in: Install via bdist_wheel

207d80f  build/pkgs/pip2pi: New

2555e19  build/make/install: At the end, update the repository index

5a747c4  build/bin/sagepip{install,uninstall}: Remove pip2 support

e66243f  Merge branch 't/29500/install_all_python_packages_via_pip_wheel__create_pep_503_simple_repository_for_wheels' into t/30527/pep503simplerepositoryforwheels

9937126  Merge branch 't/30527/pep503simplerepositoryforwheels' into t/30371/public/build/inplace

1a39442  build/pkgs/sagelib/src/setup.py: Replace symlink to SAGE_SRC/setup.py by a copy

54fe0c7  src/tox.ini, src/MANIFEST.in: Move here from build/pkgs/sagelib/src, leave symlinks behind

5bda332  Inplace (editable) installs of sagelib (#30371 squashed)

comment:34 followup: ↓ 41 Changed 6 months ago by
TODO:
 The changes to
src/sage/cpython/debugimpl.c
need to go on a separate ticket  The changes to
src/sage/misc/persist.pyx
need to go on a separate ticket  Don't change
find_python_sources
becausebuild/pkgs/sagelib/src/setup.py
continues to use it
comment:35 Changed 6 months ago by
 Description modified (diff)
comment:36 Changed 6 months ago by
 Description modified (diff)
comment:37 Changed 6 months ago by
 Dependencies changed from #29500, #30527 to #29500, #30527, #22731
comment:38 Changed 6 months ago by
 Description modified (diff)
comment:39 Changed 6 months ago by
 Commit changed from 5bda3321b88289cbd3f0dfe97308b07dd9dd1459 to 274538190984ea38145ffdb4d7d1defc88784124
comment:40 Changed 6 months ago by
 Dependencies changed from #29500, #30527, #22731 to #29500, #30527, #22731, #30578
 Description modified (diff)
comment:41 in reply to: ↑ 34 Changed 6 months ago by
comment:42 Changed 5 months ago by
 Commit changed from 274538190984ea38145ffdb4d7d1defc88784124 to a493e6e2a972ab5344fdd3a953376278e807cf3b
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
851c557  Merge branch 'develop' of git://github.com/sagemath/sage into public/build/inplace

1899dda  Remove unnessary changes

74c9f7d  Cleanup code

81fdcf3  Make distutils relative to src again

faf6777  Fail gracefully for missing packages

8af6e23  Add minimal pipfile

456589c  Ignore build files

59e8361  Make it compile by ignoring a lot of errors

054fedc  Add psutil and pexpect

a493e6e  Merge branch 'public/build/inplace' of git://trac.sagemath.org/sage into public/build/inplace

comment:43 Changed 5 months ago by
 Commit changed from a493e6e2a972ab5344fdd3a953376278e807cf3b to c5f3873cc7ddb158646051861fc10c72c6be22f5
Branch pushed to git repo; I updated commit sha1. New commits:
c5f3873  Merge branch 'develop' of git://github.com/sagemath/sage into public/build/inplace

comment:44 followups: ↓ 46 ↓ 47 Changed 5 months ago by
I have now added a minimal pipfile including sage as an editable install. So running pipenv install
results in a semiworkable virtual environment. "Semi" because not all pyx files could be compiled successfully since they rely on external dependencies (e.g. singular, giac, ecl, ratpoints, ...).
For now, I've excluded the relevant pyx files from the compilation. Any ideas how to handle these dependencies are welcome! One approach would be to treat them similar to the the optional packages by putting a "sage_setup distribution" tag to the pyx file.
I also tried to run pipenv install
directly under Windows, but that's not possible at the moment as some of the dependencies cannot be installed on Windows currently:
https://github.com/sagemath/cypari2/issues/19
https://github.com/sagemath/cysignals/pull/104
comment:45 Changed 5 months ago by
 Commit changed from c5f3873cc7ddb158646051861fc10c72c6be22f5 to b7fb5a9d46d8acb5068cde9a94009adb800d4783
comment:46 in reply to: ↑ 44 Changed 5 months ago by
Replying to ghtobiasdiez:
[...] not all pyx files could be compiled successfully since they rely on external dependencies (e.g. singular, giac, ecl, ratpoints, ...). One approach would be to treat them similar to the the optional packages by putting a "sage_setup distribution" tag to the pyx file.
Yes, exactly this approach is part of the "modularization" that I have in mind. See #29705 under the items titled "Deploy mildly modularized distributions", "Further modularization" for a sketch of such distributions. Basically, one distribution for each major C/C++ library (such as sagentl
)  but some libraries come in packs that would not make much sense to separate (such as sageflintarbe_antic
).
comment:47 in reply to: ↑ 44 Changed 5 months ago by
Replying to ghtobiasdiez:
I also tried to run
pipenv install
directly under Windows, but that's not possible at the moment as some of the dependencies cannot be installed on Windows currently: https://github.com/sagemath/cypari2/issues/19 https://github.com/sagemath/cysignals/pull/104
Right, I hope that when modularization is done, a native Windows port of a tiny Sage subset would be an actionable project.
comment:48 Changed 5 months ago by
As in #30578 (where I am trying the same with requirements.txt
and plain venv
), I would like to get the following to work:
 Build the sage distribution (actually all prerequisites of sagelib, using
make sagelibbuilddeps
)
 Then within
sage sh
(where all the libraries installed in$SAGE_LOCAL
are available) do the inplace install into a new venv created bypipenv
 using the wheels built in$SAGE_LOCAL/var/lib/sage/wheels
.
comment:49 followups: ↓ 53 ↓ 54 ↓ 55 ↓ 56 Changed 5 months ago by
Thanks for the quick feedback.
I agree the modularization is a good way to approach these problems. Sadly, I feel like this requires too much indepth knowledge of sage for me to contribute directly. But after skimming the tickets, I suggest you have a look at my changes here to the way distributions are excluded. This seems to be somewhat opposite to say #29864 where this is handled by adding a new setuptdlib/setup.py. For the inplace installation it is important that the src/setup.py results in a working installation.
What do you suggest as the way forward to align this ticket with the modularization effort?
As for #30578, my motivation here is a bit different: I wanted to have a prototype for sagelib with pipenv that works without ever invoking sage/make
and directly uses the versions of the dependencies found on pypi instead of (re)building them locally. This seems to work in principle, and might lead to a workable solution for most purposes of development and ci testing, where the advantages of building the dependencies (like optimization) are not that important. That being said, I'm not sure if we really need to have a requirements.txt and a pipfile. I would opt for the more modern version and only keep the pipfile.
After installing singular via aptget install libsingular4dev libpynacdev
, a few more pyx files could be compiled. Should I add this in build\pkgs\singular\distros\debian ?
One more thing: Currently sage/rings/number_field/number_field_element_quadratic.pyx relies on the arb lib, but is also (indirectly) imported in sage.all. What's the best way to handle these imports that depend on external C/C++ libs? For now I've uncommented some of these allimports.
comment:50 Changed 5 months ago by
 Commit changed from b7fb5a9d46d8acb5068cde9a94009adb800d4783 to 12e6d6711ca76bee221336093fa03e162962a896
comment:51 Changed 5 months ago by
Yet another question: when/how is src/sage/ext/interpreters generated?
comment:52 Changed 5 months ago by
 Commit changed from 12e6d6711ca76bee221336093fa03e162962a896 to 7a7e5764d619f0207c9d57d1c24b6198f944b799
Branch pushed to git repo; I updated commit sha1. New commits:
7a7e576  Make import sage.manifolds.all work

comment:53 in reply to: ↑ 49 Changed 5 months ago by
Replying to ghtobiasdiez:
As for #30578, my motivation here is a bit different: I wanted to have a prototype for sagelib with pipenv that works without ever invoking
sage/make
and directly uses the versions of the dependencies found on pypi instead of (re)building them locally.
Yes, I agree with this goal. But I would strongly suggest to focus on a smaller scope for this ticket: Make the inplace install work when all the dependencies are available. At the same time, make sure that the normal Sage build process does not get broken (e.g. take care of comment 34 "Don't change find_python_sources because build/pkgs/sagelib/src/setup.py continues to use it"). This ensures that the ticket can be merged in Sage, thus, a way forward. This is better than preparing one ticket with huge changes.
comment:54 in reply to: ↑ 49 ; followup: ↓ 57 Changed 5 months ago by
Replying to ghtobiasdiez:
I suggest you have a look at my changes here to the way distributions are excluded. This seems to be somewhat opposite to say #29864 where this is handled by adding a new setuptdlib/setup.py.
As discussed, adding these sage_setup: distribution
tags to Cython source files is a way forward that aligns with the modularization effort.
comment:55 in reply to: ↑ 49 ; followup: ↓ 59 Changed 5 months ago by
Replying to ghtobiasdiez:
Currently sage/rings/number_field/number_field_element_quadratic.pyx relies on the arb lib, but is also (indirectly) imported in sage.all. What's the best way to handle these imports that depend on external C/C++ libs?
Try lazy imports with PythonModule
features, from #30616/#30647.
comment:56 in reply to: ↑ 49 Changed 5 months ago by
Replying to ghtobiasdiez:
I suggest you have a look at my changes here to the way distributions are excluded.
Well, it's the same as before in sagelib's setup.py, filtering by distributions, plus some hardcoded ones. Except that you have changed the interface of the function that does the filtering (see above  this change is not mergeable because build/pkgs/sagelib/src/setup.py
needs to keep working!)
comment:57 in reply to: ↑ 54 Changed 5 months ago by
Replying to mkoeppe:
As discussed, adding these
sage_setup: distribution
tags to Cython source files is a way forward that aligns with the modularization effort.
I've opened #30666 for this. (The new distribution names have to be added to setup.py
, of course, so that the files continue to be built in the Sage distribution.)
comment:58 Changed 5 months ago by
 Dependencies changed from #29500, #30527, #22731, #30578 to #29500, #30527, #22731, #30578, #30666
comment:59 in reply to: ↑ 55 Changed 5 months ago by
Replying to mkoeppe:
Replying to ghtobiasdiez:
Currently sage/rings/number_field/number_field_element_quadratic.pyx relies on the arb lib, but is also (indirectly) imported in sage.all. What's the best way to handle these imports that depend on external C/C++ libs?
Try lazy imports with
PythonModule
features, from #30616/#30647.
For another approach regarding the ".all" imports, see #29865. Probably we will need a combination of this and lazy imports.
comment:60 Changed 5 months ago by
 Commit changed from 7a7e5764d619f0207c9d57d1c24b6198f944b799 to 4afd411dd86c37e90ca32c8f77de89a91c57a667
Branch pushed to git repo; I updated commit sha1. New commits:
4afd411  Readd old find_python_sources

comment:61 Changed 5 months ago by
I've now restored the old find_python_sources method. Is there something else that needs to be done, except waiting for the other tickets to be finished?
comment:62 Changed 5 months ago by
 Description modified (diff)
comment:63 Changed 5 months ago by
Nothing immediate. This is going in a good direction
comment:64 followup: ↓ 65 Changed 5 months ago by
 Commit changed from 4afd411dd86c37e90ca32c8f77de89a91c57a667 to c82bbd56e5ab828b73dfe09ad1c32c9b7d96f866
comment:65 in reply to: ↑ 64 Changed 5 months ago by
comment:66 Changed 5 months ago by
 Dependencies changed from #29500, #30527, #22731, #30578, #30666 to #29500, #30527, #22731, #30578, #30666, #30672
comment:67 Changed 5 months ago by
 Dependencies changed from #29500, #30527, #22731, #30578, #30666, #30672 to #29500, #30527, #22731, #30578, #30666, #30672, #30673
comment:68 Changed 5 months ago by
 Description modified (diff)
comment:69 Changed 5 months ago by
 Commit changed from c82bbd56e5ab828b73dfe09ad1c32c9b7d96f866 to 62930ee7ada36664e0fc2264e5395f8b8337b650
Branch pushed to git repo; I updated commit sha1. New commits:
62930ee  Add ipykernel and tox as dev packages

comment:70 Changed 5 months ago by
 Description modified (diff)
New commits:
62930ee  Add ipykernel and tox as dev packages

comment:71 Changed 5 months ago by
 Dependencies changed from #29500, #30527, #22731, #30578, #30666, #30672, #30673 to #29500, #30527, #22731, #30578, #30672, #30673
comment:72 Changed 5 months ago by
 Commit changed from 62930ee7ada36664e0fc2264e5395f8b8337b650 to cd196a0773e36e7e6c1fbe21962ef5b1b1485cf5
comment:73 Changed 5 months ago by
 Description modified (diff)
comment:74 Changed 5 months ago by
This ticket is way too big. I really strong recommend to get a working inplace installation of sagelib in the situation when its dependencies have been built using the Sage distribution.
comment:75 followup: ↓ 77 Changed 5 months ago by
I've now followed your advice, and tried to have a prototype that works if all library dependencies are fulfilled (instead of adding exceptions / distribution annotations everywhere). And I was successful indeed, import sage.all
works without error and I can do basic calculations with the manifold package. This required tinkering with a lot of systeminstalled packages (instructions added in the ticket).
The following is not yet working, and I would like to get input on how to resolve this:
sage/ext/interpreters
is empty. Where is the code generated? Can this easily be extracted to a script that can be invoked separately? The code relying on the arb library still doesn't compile although I've
libflintarbdev
installed. The compilation fails withld larb
was not found. Why is this, and what's the best way to solve it?  The
sage.doctest
package conflicted with thesympy
(I believe), i.e. sympy tried to load classes fromsage.doctest
for some reason, and couldn't locate them. I've fixed this by renamingsrc/sage/doctest
tosagedoctest
. Is thedoctest
package really required under thesage
folder, or could it be e.g moved up tosrc/doctest
?
I agree the ticket is too big at the moment. Once I found a solution to the above points, I'll clean the code. I sadly cannot work with the dependencies created by the sage distribution, as the build process fails due to compilation errors in the pip library...
comment:76 Changed 5 months ago by
 Commit changed from cd196a0773e36e7e6c1fbe21962ef5b1b1485cf5 to 1422cf461178919c2ccf1a30b6a82e1ac5dee396
Branch pushed to git repo; I updated commit sha1. New commits:
1422cf4  Merge branch 'develop' of git://github.com/sagemath/sage into public/build/inplace

comment:77 in reply to: ↑ 75 Changed 5 months ago by
Replying to ghtobiasdiez:
I've now followed your advice, and tried to have a prototype that works if all library dependencies are fulfilled (instead of adding exceptions / distribution annotations everywhere). And I was successful indeed,
import sage.all
works without error and I can do basic calculations with the manifold package. This required tinkering with a lot of systeminstalled packages (instructions added in the ticket).
Great.
The following is not yet working, and I would like to get input on how to resolve this:
sage/ext/interpreters
is empty. Where is the code generated? Can this easily be extracted to a script that can be invoked separately?
sage_setup.command.sage_build
 The code relying on the arb library still doesn't compile although I've
libflintarbdev
installed. The compilation fails withld larb
was not found. Why is this, and what's the best way to solve it?
Use sage.env.cython_aliases
. In particular ARB_LIBRARY
gives you the name. (The Debian installation uses a different name from upstream.)
 The
sage.doctest
package conflicted with thesympy
(I believe), i.e. sympy tried to load classes fromsage.doctest
for some reason, and couldn't locate them. I've fixed this by renamingsrc/sage/doctest
tosagedoctest
. Is thedoctest
package really required under thesage
folder, or could it be e.g moved up tosrc/doctest
?
Sounds like a bad PYTHONPATH. src/sage
should not be in it.
I agree the ticket is too big at the moment. Once I found a solution to the above points, I'll clean the code. I sadly cannot work with the dependencies created by the sage distribution, as the build process fails due to compilation errors in the pip library...
Hm, that's still happening? Sorry I lost track of this since various other things seemed to break on WSL. Have you tried with downgrading/upgrading the pip package?
comment:78 Changed 5 months ago by
 Commit changed from 1422cf461178919c2ccf1a30b6a82e1ac5dee396 to 05713b87fff7948a68562358a5d32a29701bfa2f
Branch pushed to git repo; I updated commit sha1. New commits:
05713b8  Remove dependency on #30527

comment:79 Changed 5 months ago by
By the way, for dev purposes until the WSL issues are resolved, you could try running Sage in Docker instead (bindmount the source tree into it)
comment:80 Changed 5 months ago by
 Dependencies changed from #29500, #30527, #22731, #30578, #30672, #30673 to #29500, #22731, #30672, #30673
comment:81 Changed 5 months ago by
 Dependencies changed from #29500, #22731, #30672, #30673 to #29500, #30672, #30673
comment:82 Changed 5 months ago by
 Commit changed from 05713b87fff7948a68562358a5d32a29701bfa2f to ac07cc0d005de5e067e29c22f154521537f5906d
Branch pushed to git repo; I updated commit sha1. New commits:
ac07cc0  Move requirements file back

comment:83 in reply to: ↑ description Changed 5 months ago by
 Fix installation of
ecl
:After
aptget install ecl
ecl is broken under WSL. To fix this, first runaptget install libtool autoconf
, then follow https://github.com/rdp/ffmpegwindowsbuildhelpers/issues/452#issuecomment638639182.
Could you try the related #30629 on WSL please?
comment:84 Changed 5 months ago by
 Dependencies changed from #29500, #30672, #30673 to #29500, #30672, #30673, #30590
 Description modified (diff)
comment:85 Changed 5 months ago by
 Commit changed from ac07cc0d005de5e067e29c22f154521537f5906d to 3023260e2c36e80113daf0af69e95d6091de461e
comment:86 Changed 5 months ago by
 Commit changed from 3023260e2c36e80113daf0af69e95d6091de461e to 5b12a924507cf85a2eca749993835927b9398668
comment:87 Changed 5 months ago by
 Description modified (diff)
 Status changed from needs_work to needs_review
comment:88 Changed 5 months ago by
Perfect, thanks a lot Matthias! I got now a working installation of all libraries, and could import sage.all
without the need to uncomment any imports. Yeaaaah!
I've also cleaned the code and extended the ticket description, so this ticket is now ready for review.
comment:89 Changed 5 months ago by
Could you put the changes to sage_setup/autogen/interpreters/
on a separate ticket please?
comment:90 Changed 5 months ago by
The changes regarding imports (mostly around sage.rings
) should also be on a separate ticket.
comment:91 Changed 5 months ago by
Also for the changes to src/sage/libs/eclsig.h
 which will certainly need discussion
comment:92 followup: ↓ 94 Changed 5 months ago by
+compile_time_env = dict( + PY_PLATFORM=sys.platform +) +cython_directives = dict( + language_level="3str", + cdivision=True, +)
This duplicates settings defined in sage_setup.command.sage_build_cython
.
Could you refactor this so that these directives are defined in a reusable way?
Perhaps in a new module sage_setup.cython_options
comment:93 Changed 5 months ago by
comment:94 in reply to: ↑ 92 Changed 5 months ago by
Replying to mkoeppe:
Could you refactor this so that these directives are defined in a reusable way? Perhaps in a new module
sage_setup.cython_options
... in a separate ticket of course.
comment:95 followup: ↓ 96 Changed 5 months ago by
Will do so. For the cython options, they are actually only a small subset of the ones in sage_build_cython
. Should I still refactor? (Feels a bit strange as there are more differences than similarities.) But I'm also not sure if the other cython options are needed...
comment:96 in reply to: ↑ 95 Changed 5 months ago by
Replying to ghtobiasdiez:
Will do so. For the cython options, they are actually only a small subset of the ones in
sage_build_cython
. Should I still refactor? (Feels a bit strange as there are more differences than similarities.) But I'm also not sure if the other cython options are needed...
Well, the idea would be to see whether you can just build with the options that Sage uses instead of using your own set of options.
comment:97 Changed 5 months ago by
Whether other Cython build options can/should be used  should be orthogonal to whether an inplace or regular build is done. So it should not be done on this ticket.
comment:98 in reply to: ↑ description Changed 5 months ago by
Some comments on items in the ticket description:
TODO (as followup tickets):
 Automatically generate the
aptget install
command with the necessary libraries (add them asdistros/debian.txt
?)
./configure
takes care of all of this. Do you plan to do system interface configuration and system package advice without using ./configure
(or, more generally, Unix shell scripts)?
A way forward is to reimplement (cleaner version of) some of the simple shell code for system package advice as Python code in sage_bootstrap
. See #29146.
 Remove
ARB_LIBRARY
env variable, and move check fromarb/config.m4
toenv.py
My suggestion: Instead of discussing to remove this variable (or other configuration variables that are set by ./configure
), rather discuss different ways of generating the sage_conf
module / or providing different implementations of the interface defined by the sage_conf
module for different use cases (such as supporting a platform that does not have a Unix shell).
 Move package dependencies from pipfile to install_requires in
src/setup.py
.
Best to put them in src/setup.cfg
instead of src/setup.py
(see #30578)
comment:99 Changed 5 months ago by
 Commit changed from 5b12a924507cf85a2eca749993835927b9398668 to 8abab5dfb588a3b6e6fe0c1d32525656ddc13cf6
comment:100 Changed 5 months ago by
Thanks for preparing this cleaned up branch.
Trying to test it, I seem to be running into something that looks like https://discuss.python.org/t/pip191andinstallingineditablemodewithpyprojecttoml/1553 :
Something is generating a pyproject.toml
in src/
and then it dies with
Writing supplied requirement line to temporary file: 'e .' Installing 'e1839a8' $ ['/Users/mkoeppe/.local/share/virtualenvs/srcBXov0SKG/bin/pip', 'install', 'verbose', 'upgrade', 'nousepep517', 'nodeps', 'existsaction=i', 'r', '/var/folders/38/wnh4gf1552g_crsjnv2vmmww0000gp/T/pipenv_yopr9l3requirements/pipenv7uv8tb0mrequirement.txt', 'i', 'https://pypi.org/simple'] Using source directory: '/Users/mkoeppe/.local/share/virtualenvs/srcBXov0SKG/src' Using pip 20.2.1 from /Users/mkoeppe/.local/share/virtualenvs/srcBXov0SKG/lib/python3.8/sitepackages/pip (python 3.8) Nonuser install by explicit request ... File "/Users/mkoeppe/.local/share/virtualenvs/srcBXov0SKG/lib/python3.8/sitepackages/pip/_internal/distributions/sdist.py", line 33, in prepare_distribution_metadata self.req.load_pyproject_toml() File "/Users/mkoeppe/.local/share/virtualenvs/srcBXov0SKG/lib/python3.8/sitepackages/pip/_internal/req/req_install.py", line 503, in load_pyproject_toml pyproject_toml_data = load_pyproject_toml( File "/Users/mkoeppe/.local/share/virtualenvs/srcBXov0SKG/lib/python3.8/sitepackages/pip/_internal/pyproject.py", line 95, in load_pyproject_toml raise InstallationError( pip._internal.exceptions.InstallationError: Disabling PEP 517 processing is invalid: project specifies a build backend of setuptools.build_meta in pyproject.toml Removed file:///Users/mkoeppe/s/sage/sagerebasing/src (from r /var/folders/38/wnh4gf1552g_crsjnv2vmmww0000gp/T/pipenv_yopr9l3requirements/pipenv7uv8tb0mrequirement.txt (line 1)) from build tracker '/private/var/folders/38/wnh4gf1552g_crsjnv2vmmww0000gp/T/pipreqtrackerlxrv7bcd' Removed build tracker: '/private/var/folders/38/wnh4gf1552g_crsjnv2vmmww0000gp/T/pipreqtrackerlxrv7bcd'
comment:101 Changed 5 months ago by
Strange. For some reason, for you pip is invoked with nousepep517
. For me it is not:
$ "/mnt/d/Programming/Projects/sage/src/.venv/bin/pip" install verbose nodeps e "." i https://pypi.org/simple existsaction w...
What version of pipenv do you use? For me: pipenv 11.9.0 (and the pip in the virtual env also differs: pip 20.0.2)
comment:102 Changed 5 months ago by
I'm using pipenv from homebrew, which is version 2020.8.13
comment:103 Changed 5 months ago by
I will try to update my pipenv later. For now you can have a look if setting the environment variable PIP_USE_PEP517 to true resolves the issue.
comment:104 Changed 5 months ago by
I'll try, but I think editable installs are not compatible with PEP517
comment:105 Changed 5 months ago by
No change after setting this environment variable
comment:106 Changed 5 months ago by
 Dependencies changed from #29500, #30672, #30673, #30590 to #29500, #30672, #30673, #30590, #30709
comment:107 Changed 5 months ago by
I had a closer look at this, and it turns out that the first time pip is correctly called without any nousepep517
argument, only on the retry this argument is used. Thus, the error actually shadows another error that occurs during the initial pip install
.
Can you thus run the following pipenv run pip install e . nobuildisolation
after pipenv install
. This should point to the real issue why the editable install fails.
I've created a github workflow because I don't have a mac. https://github.com/tobiasdiez/sage/blob/public/build/inplace/.github/workflows/cipipenv.yml It shows the same problem, the pipenv install
fails apparently due to a pep517 issue, but the editable install actually fails due to missing interpreters and other dependencies (which requires manual installations as in the ticket description).
comment:108 Changed 5 months ago by
Hm...
(sagesh) mkoeppe@egret:src$ pipenv run pip install e . nobuildisolation Obtaining file:///Users/mkoeppe/s/sage/sagerebasing/src Preparing wheel metadata ... error ERROR: Command errored out with exit status 1: command: /Users/mkoeppe/.local/share/virtualenvs/srcBXov0SKG/bin/python /Users/mkoeppe/.local/share/virtualenvs/srcBXov0SKG/lib/python3.8/sitepackages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /var/folders/38/wnh4gf1552g_crsjnv2vmmww0000gp/T/tmpvt150u93 cwd: /Users/mkoeppe/s/sage/sagerebasing/src Complete output (14 lines): Traceback (most recent call last): File "/Users/mkoeppe/.local/share/virtualenvs/srcBXov0SKG/lib/python3.8/sitepackages/pip/_vendor/pep517/_in_process.py", line 280, in <module> main() File "/Users/mkoeppe/.local/share/virtualenvs/srcBXov0SKG/lib/python3.8/sitepackages/pip/_vendor/pep517/_in_process.py", line 263, in main json_out['return_val'] = hook(**hook_input['kwargs']) File "/Users/mkoeppe/.local/share/virtualenvs/srcBXov0SKG/lib/python3.8/sitepackages/pip/_vendor/pep517/_in_process.py", line 133, in prepare_metadata_for_build_wheel return hook(metadata_directory, config_settings) File "/Users/mkoeppe/.local/share/virtualenvs/srcBXov0SKG/lib/python3.8/sitepackages/setuptools/build_meta.py", line 161, in prepare_metadata_for_build_wheel self.run_setup() File "/Users/mkoeppe/.local/share/virtualenvs/srcBXov0SKG/lib/python3.8/sitepackages/setuptools/build_meta.py", line 145, in run_setup exec(compile(code, __file__, 'exec'), locals()) File "setup.py", line 23, in <module> import sage.env ModuleNotFoundError: No module named 'sage'  ERROR: Command errored out with exit status 1: /Users/mkoeppe/.local/share/virtualenvs/srcBXov0SKG/bin/python /Users/mkoeppe/.local/share/virtualenvs/srcBXov0SKG/lib/python3.8/sitepackages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /var/folders/38/wnh4gf1552g_crsjnv2vmmww0000gp/T/tmpvt150u93 Check the logs for full command output.
comment:109 Changed 5 months ago by
No idea....for the github action this is not a problem: https://github.com/tobiasdiez/sage/runs/1205452529?check_suite_focus=true. Maybe pipenv run pip install e . verbose upgrade existsaction=i nobuildisolation
works, or you need to clean the venv first (pipenv rm).
comment:110 Changed 5 months ago by
Sure, I'll play with it more in the next days.
comment:111 Changed 5 months ago by
 Description modified (diff)
comment:112 Changed 5 months ago by
 Commit changed from 8abab5dfb588a3b6e6fe0c1d32525656ddc13cf6 to edde293052d7e4660df11ce13513f08edf44c5e5
Branch pushed to git repo; I updated commit sha1. New commits:
edde293  Merge branch 'develop' of git://github.com/sagemath/sage into public/build/inplace

comment:113 Changed 5 months ago by
 Commit changed from edde293052d7e4660df11ce13513f08edf44c5e5 to 74ecddbe6a71f133458cdb018f45e36ba0aaf818
Branch pushed to git repo; I updated commit sha1. New commits:
74ecddb  Update dependencies

comment:114 Changed 5 months ago by
 Commit changed from 74ecddbe6a71f133458cdb018f45e36ba0aaf818 to 7110841f2c816001c14205c1426088fb873499b3
comment:115 Changed 5 months ago by
 Dependencies changed from #29500, #30672, #30673, #30590, #30709 to #30779, #30672, #30673, #30709
comment:116 Changed 5 months ago by
 Commit changed from 7110841f2c816001c14205c1426088fb873499b3 to 0b5403699f9472f102b4781e3fd9ebae80e6bb8a
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
52e2107  sage_include_directories: Remove sage/ext

6ccce6b  Fix missing ccobject

1402129  Merge branch 'u/mkoeppe/remove__sage_ext__from__sage_include_directories_' of git://trac.sagemath.org/sage into t/30371/public/build/inplace

6eb3845  Extend gitignore and add VS code config

bbe73ee  Remove VS code settings, and add exceptions for already checkedin files

22b7cac  Merge branch 'public/build/enhancegitignore' of git://trac.sagemath.org/sage into t/30371/public/build/inplace

1aef376  Extract cython options to new module

68ab47b  Merge branch 'public/build/cythonConfig' of git://trac.sagemath.org/sage into t/30371/public/build/inplace

d1b1288  Narrow down to essenital changes

0b54036  Update dependencies

comment:117 Changed 5 months ago by
Rebased on top of the dependencies
comment:118 Changed 5 months ago by
 Status changed from needs_review to needs_work
 Work issues set to Use the cython options
comment:119 Changed 5 months ago by
 Commit changed from 0b5403699f9472f102b4781e3fd9ebae80e6bb8a to 55930254f941314d9996bfc6a9a4d5f9767c2e6d
Branch pushed to git repo; I updated commit sha1. New commits:
82e8b7d  Narrow down to essenital changes

8abab5d  Add copy of setup.py

edde293  Merge branch 'develop' of git://github.com/sagemath/sage into public/build/inplace

74ecddb  Update dependencies

1286fcb  Merge branch 'public/build/inplace' of git://trac.sagemath.org/sage into public/build/inplace

946737e  Use new cython option modules

5593025  Update dependencies

comment:120 Changed 5 months ago by
 Status changed from needs_work to needs_review
 Work issues Use the cython options deleted
Thanks for the rebase. Next time please try to not forcepush to a branch that others have checked out because it messes up their git history.
I've updated the code to use the new cython options module.
comment:121 Changed 5 months ago by
Sorry if the push made it harder for you to continue on this branch. But rebasing it on top of the branches of the dependency tickets was a necessary cleanup
comment:122 Changed 5 months ago by
 Description modified (diff)
comment:123 Changed 5 months ago by
What's the status of this branch? Does it work for you? Can you run doctests in the inplace installation and does it pass them?
comment:124 followup: ↓ 125 Changed 5 months ago by
It's working for me. pipenv install dev
gives a fully functional virtual environment (so the cython compilation works). Moreover, import sage.all
works as well, and one can then use it to run sagecode. So far, I've tested it with some code from the manifolds package. Not yet tried doctests, as I've no idea how to run them from the virtual environment.
comment:125 in reply to: ↑ 124 Changed 5 months ago by
Replying to ghtobiasdiez:
Not yet tried doctests, as I've no idea how to run them from the virtual environment.
Well, the virtual environment has a bin/sage
script, so sage t
"should" work. If it does not, this needs to be fixed. Likely some work in the direction of #22731 is necessary.
comment:126 followup: ↓ 127 Changed 5 months ago by
Yeah, sage t
still uses the python in sage/local
. So this does not work, and a mechanism is needed to use the python of the activated virtual env (without changing env variables which is too inflexible in my opinion).
Directly running bin/sageruntests all
also leads to errors:
no stored timings available Running doctests with ID 20201020211829b0a05b3d. Using optional=dochtml,memlimit,sage,sage_spkg Doctesting entire Sage library. Traceback (most recent call last): File "bin/sageruntests", line 182, in <module> err = DC.run() File "/mnt/d/Programming/Projects/sage/src/sage/doctest/control.py", line 1234, in run self.expand_files_into_sources() File "/mnt/d/Programming/Projects/sage/src/sage/doctest/control.py", line 817, in expand_files_into_sources self.sources = [FileDocTestSource(path, self.options) for path in expand()] File "/mnt/d/Programming/Projects/sage/src/sage/doctest/control.py", line 817, in <listcomp> self.sources = [FileDocTestSource(path, self.options) for path in expand()] File "/mnt/d/Programming/Projects/sage/src/sage/doctest/sources.py", line 528, in __init__ raise ValueError("unknown file extension %r"%ext) ValueError: unknown file extension ''
(I guess some files are not found.)
Anyway, that's probably a topic for another ticket.
comment:127 in reply to: ↑ 126 Changed 5 months ago by
Replying to ghtobiasdiez:
Yeah,
sage t
still uses the python insage/local
. So this does not work, and a mechanism is needed to use the python of the activated virtual env (without changing env variables which is too inflexible in my opinion).
Yes, this is precisely #22731
comment:128 Changed 5 months ago by
 Commit changed from 55930254f941314d9996bfc6a9a4d5f9767c2e6d to 9dc8a2b0bcd6ff1cada426c60184b322312618e4
Branch pushed to git repo; I updated commit sha1. New commits:
9dc8a2b  Generate code automatically

comment:129 Changed 5 months ago by
 Commit changed from 9dc8a2b0bcd6ff1cada426c60184b322312618e4 to 19482048d4fffe52e94fcd94b81fa8ef59000558
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
fcef2b6  Use pip to install pipenv

4160c9f  Update cipipenv.yml

d0e01f2  Update cipipenv.yml

711a2a6  Update cipipenv.yml

341be72  Update cipipenv.yml

918e1a5  Update cipipenv.yml

573ec51  Try with only one python

961abc3  Update cipipenv.yml

5a9b803  Update cipipenv.yml

1948204  Merge branch 'public/build/inplace' of https://github.com/tobiasdiez/sage into public/build/inplace

comment:130 Changed 5 months ago by
 Commit changed from 19482048d4fffe52e94fcd94b81fa8ef59000558 to fffea7a7a50dd8c79827a07d6ef4eb3568fc1817
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
0016538  Merge branch 'public/build/inplace' of https://github.com/tobiasdiez/sage into public/build/inplace

d65d85e  Update cipipenv.yml

c1b8aef  Merge branch 'public/build/inplace' of https://github.com/tobiasdiez/sage into public/build/inplace

23d17ad  Rename cblas to blas

3cfe1c9  Merge branch 'develop' of git://github.com/sagemath/sage into public/build/cblas

ea4a44c  Support different blas libraries

75966fc  Put cblas first

273bb93  Make cblas pc module search configurable via sage_conf

e6ff9ba  Merge branch 'public/build/cblas' of git://trac.sagemath.org/sage into public/build/inplace

fffea7a  Install more dependencies

comment:131 Changed 5 months ago by
 Dependencies changed from #30779, #30672, #30673, #30709 to #30779, #30672, #30673, #30709, #30706
comment:132 Changed 5 months ago by
 Commit changed from fffea7a7a50dd8c79827a07d6ef4eb3568fc1817 to 062746cad45c3c3edc9f0d905de8312df1c020ac
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
ed74c46  Fix powershell script

88f4529  Make powershell run

4171062  Cleanup

bfdc1c0  Fix problem with libhomfly

3bda21e  Fix arb lib

3d476d5  Install more deps for macos (as recommended by configure)

4302ac9  Also install ecl

e34a6af  Also install pynac

8679770  Disable copying of sageenvconfig

062746c  Try with hardcoded version

comment:133 followup: ↓ 135 Changed 5 months ago by
I've now added github actions to have a reproducible test in a clean environment. The compilation is working now (under linux), but the singular library cannot be found during runtime. https://github.com/tobiasdiez/sage/runs/1287237342?check_suite_focus=true
The problem appears to be that sage is looking for singular in sysconfig.get_config_var('LIBDIR')
(e.g. /usr/local/lib
) but it is installed as /usr/lib/x86_64linuxgnu/libsingularSingular.so
. Is there a reason why LIBDIR
is used in env.py
instead of the approach using pkgconfig
as during compilation time?
comment:134 Changed 5 months ago by
_get_shared_lib_filename
actually also looks in MULTILIB
and is supposed to find this Debianspecific install location via this mechanism. But it is possible that this is broken because we currently actually do not use system singular
(nor gap
) at all (see #29644, #29024, metaticket #27330). This needs more investigation.
comment:135 in reply to: ↑ 133 Changed 5 months ago by
Replying to ghtobiasdiez:
Is there a reason why
LIBDIR
is used inenv.py
instead of the approach usingpkgconfig
as during compilation time?
pkgconfig does not really reveal the libname directly, so one would need to parse pkgconfig libs
.
Another reason is that the .pc files are usually only available in "dev" packages. So the runtime of Sage should not depend on it.
#29024 intends to make the location configurable (at build time!) via sage_conf
.
comment:136 Changed 4 months ago by
 Commit changed from 062746cad45c3c3edc9f0d905de8312df1c020ac to 7523baa68b46732cc381387e58ea1a9559c11e96
comment:137 followup: ↓ 139 Changed 4 months ago by
Thanks for the input. I had a closer look at _get_shared_lib_filename
and the problem was that libsingularSingular.so
is in the subfolder /usr/lib/x86_64linuxgnu/
while _get_shared_lib_filename
only searched the top folder /usr/lib
. I've now changed it to also look in subfolders. That should be reasonable fix until #29024 is implemented.
With these changes, the github action workflow passes successful on linux: https://github.com/tobiasdiez/sage/runs/1303324520?check_suite_focus=true It's nice to see that it only takes about 1 hour to have a fully workable sage environment.
This is now really ready for review.
TODOs (for further tickets):
 Investigate what needs to be installed on macOS to have a successful build, and add these steps to the github action. (I don't have a mac, so I'll not work on this.)
 Run doctests in the github action as soon as they no longer rely on
SAGE_LOCAL
(#22731)
comment:138 Changed 4 months ago by
 Description modified (diff)
comment:139 in reply to: ↑ 137 Changed 4 months ago by
Replying to ghtobiasdiez:
Thanks for the input. I had a closer look at
_get_shared_lib_filename
and the problem was thatlibsingularSingular.so
is in the subfolder/usr/lib/x86_64linuxgnu/
while_get_shared_lib_filename
only searched the top folder/usr/lib
. I've now changed it to also look in subfolders.
This should have been taking care of by MULTILIB
. Is this not configured in your system?
comment:140 Changed 4 months ago by
MULTILIB
is empty on my system (WSL Ubuntu 20.04), nor was it working for the github actions (Ubuntu 20.10).
comment:141 Changed 4 months ago by
Can you show the full output of python3 m sysconfig
on this system?
comment:142 Changed 4 months ago by
Platform: "linuxx86_64" Python version: "3.8" Current installation scheme: "posix_prefix" Paths: data = "/mnt/d/Programming/Projects/sage/src/.venv" include = "/usr/include/python3.8" platinclude = "/usr/include/python3.8" platlib = "/mnt/d/Programming/Projects/sage/src/.venv/lib/python3.8/sitepackages" platstdlib = "/mnt/d/Programming/Projects/sage/src/.venv/lib/python3.8" purelib = "/mnt/d/Programming/Projects/sage/src/.venv/lib/python3.8/sitepackages" scripts = "/mnt/d/Programming/Projects/sage/src/.venv/bin" stdlib = "/usr/lib/python3.8" Variables: ABIFLAGS = "" AC_APPLE_UNIVERSAL_BUILD = "0" AIX_GENUINE_CPLUSPLUS = "0" ALT_SOABI = "0" ANDROID_API_LEVEL = "0" AR = "x86_64linuxgnugccar" ARFLAGS = "rcs" BASECFLAGS = "Wnounusedresult Wsigncompare" BASECPPFLAGS = "IObjects IInclude IPython" BASEMODLIBS = "" BINDIR = "/usr/bin" BINLIBDEST = "/usr/lib/python3.8" BLDLIBRARY = "lpython3.8" BLDSHARED = "x86_64linuxgnugcc pthread shared Wl,O1 Wl,Bsymbolicfunctions Wl,Bsymbolicfunctions Wl,z,relro g fwrapv O2 " BUILDEXE = "" BUILDPYTHON = "python" BUILD_GNU_TYPE = "x86_64pclinuxgnu" BYTESTR_DEPS = "\" CC = "x86_64linuxgnugcc pthread" CCSHARED = "fPIC" CFLAGS = "Wnounusedresult Wsigncompare DNDEBUG g fwrapv O2 Wall g fstackprotectorstrong Wformat Werror=formatsecurity g fwrapv O2 " CFLAGSFORSHARED = "fPIC" CFLAGS_ALIASING = "" CFLAGS_NODIST = "" CONFIGFILES = "configure configure.ac acconfig.h pyconfig.h.in Makefile.pre.in" CONFIGURE_CFLAGS = "g fstackprotectorstrong Wformat Werror=formatsecurity" CONFIGURE_CFLAGS_NODIST = "std=c99 Wextra Wnounusedresult Wnounusedparameter Wnomissingfieldinitializers Werror=implicitfunctiondeclaration" CONFIGURE_CPPFLAGS = "Wdatetime D_FORTIFY_SOURCE=2" CONFIGURE_LDFLAGS = "Wl,Bsymbolicfunctions Wl,z,relro g fwrapv O2 " CONFIGURE_LDFLAGS_NODIST = "" CONFIG_ARGS = "'enableshared' 'prefix=/usr' 'enableipv6' 'enableloadablesqliteextensions' 'withdbmliborder=bdb:gdbm' 'withcomputedgotos' 'withoutensurepip' 'withsystemexpat' 'withsystemlibmpdec' 'withdtrace' 'withsystemffi' 'CC=x86_64linuxgnugcc' 'CFLAGS=g fstackprotectorstrong Wformat Werror=formatsecurity ' 'LDFLAGS=Wl,Bsymbolicfunctions Wl,z,relro g fwrapv O2 ' 'CPPFLAGS=Wdatetime D_FORTIFY_SOURCE=2'" CONFINCLUDEDIR = "/usr/include" CONFINCLUDEPY = "/usr/include/python3.8" COREPYTHONPATH = "" COVERAGE_INFO = "/build/python3.86QL2k7/python3.83.8.2/buildshared/coverage.info" COVERAGE_REPORT = "/build/python3.86QL2k7/python3.83.8.2/buildshared/lcovreport" COVERAGE_REPORT_OPTIONS = "nobranchcoverage title "CPython lcov report"" CPPFLAGS = "IObjects IInclude IPython I. I../Include Wdatetime D_FORTIFY_SOURCE=2" CXX = "x86_64linuxgnug++ pthread" DESTDIRS = "/usr /usr/lib /usr/lib/python3.8 /usr/lib/python3.8/libdynload" DESTLIB = "/usr/lib/python3.8" DESTPATH = "" DESTSHARED = "/usr/lib/python3.8/libdynload" DFLAGS = "" DIRMODE = "755" DIST = "README.rst ChangeLog configure configure.ac acconfig.h pyconfig.h.in Makefile.pre.in Include Lib Misc Extdummy" DISTDIRS = "Include Lib Misc Extdummy" DISTFILES = "README.rst ChangeLog configure configure.ac acconfig.h pyconfig.h.in Makefile.pre.in" DLINCLDIR = "." DLLLIBRARY = "" DOUBLE_IS_ARM_MIXED_ENDIAN_IEEE754 = "0" DOUBLE_IS_BIG_ENDIAN_IEEE754 = "0" DOUBLE_IS_LITTLE_ENDIAN_IEEE754 = "1" DTRACE = "/usr/bin/dtrace" DTRACE_DEPS = "\" DTRACE_HEADERS = "Include/pydtrace_probes.h" DTRACE_OBJS = "Python/pydtrace.o" DYNLOADFILE = "dynload_shlib.o" ENABLE_IPV6 = "1" ENSUREPIP = "no" EXE = "" EXEMODE = "755" EXTRATESTOPTS = "" EXT_SUFFIX = ".cpython38x86_64linuxgnu.so" FILEMODE = "644" FLOAT_WORDS_BIGENDIAN = "0" FLOCK_NEEDS_LIBBSD = "0" GETPGRP_HAVE_ARG = "0" GETTIMEOFDAY_NO_TZ = "0" GITBRANCH = "" GITTAG = "" GITVERSION = "" GNULD = "yes" HAVE_ACCEPT4 = "1" HAVE_ACOSH = "1" HAVE_ADDRINFO = "1" HAVE_ALARM = "1" HAVE_ALIGNED_REQUIRED = "0" HAVE_ALLOCA_H = "1" HAVE_ALTZONE = "0" HAVE_ASINH = "1" HAVE_ASM_TYPES_H = "1" HAVE_ATANH = "1" HAVE_BIND_TEXTDOMAIN_CODESET = "1" HAVE_BLUETOOTH_BLUETOOTH_H = "1" HAVE_BLUETOOTH_H = "0" HAVE_BROKEN_MBSTOWCS = "0" HAVE_BROKEN_NICE = "0" HAVE_BROKEN_PIPE_BUF = "0" HAVE_BROKEN_POLL = "0" HAVE_BROKEN_POSIX_SEMAPHORES = "0" HAVE_BROKEN_PTHREAD_SIGMASK = "0" HAVE_BROKEN_SEM_GETVALUE = "0" HAVE_BROKEN_UNSETENV = "0" HAVE_BUILTIN_ATOMIC = "1" HAVE_CHFLAGS = "0" HAVE_CHOWN = "1" HAVE_CHROOT = "1" HAVE_CLOCK = "1" HAVE_CLOCK_GETRES = "1" HAVE_CLOCK_GETTIME = "1" HAVE_CLOCK_SETTIME = "1" HAVE_COMPUTED_GOTOS = "1" HAVE_CONFSTR = "1" HAVE_CONIO_H = "0" HAVE_COPYSIGN = "1" HAVE_COPY_FILE_RANGE = "1" HAVE_CRYPT_H = "1" HAVE_CRYPT_R = "1" HAVE_CTERMID = "1" HAVE_CTERMID_R = "0" HAVE_CURSES_FILTER = "1" HAVE_CURSES_H = "1" HAVE_CURSES_HAS_KEY = "1" HAVE_CURSES_IMMEDOK = "1" HAVE_CURSES_IS_PAD = "1" HAVE_CURSES_IS_TERM_RESIZED = "1" HAVE_CURSES_RESIZETERM = "1" HAVE_CURSES_RESIZE_TERM = "1" HAVE_CURSES_SYNCOK = "1" HAVE_CURSES_TYPEAHEAD = "1" HAVE_CURSES_USE_ENV = "1" HAVE_CURSES_WCHGAT = "1" HAVE_DECL_ISFINITE = "1" HAVE_DECL_ISINF = "1" HAVE_DECL_ISNAN = "1" HAVE_DECL_RTLD_DEEPBIND = "1" HAVE_DECL_RTLD_GLOBAL = "1" HAVE_DECL_RTLD_LAZY = "1" HAVE_DECL_RTLD_LOCAL = "1" HAVE_DECL_RTLD_MEMBER = "0" HAVE_DECL_RTLD_NODELETE = "1" HAVE_DECL_RTLD_NOLOAD = "1" HAVE_DECL_RTLD_NOW = "1" HAVE_DECL_TZNAME = "0" HAVE_DEVICE_MACROS = "1" HAVE_DEV_PTC = "0" HAVE_DEV_PTMX = "1" HAVE_DIRECT_H = "0" HAVE_DIRENT_D_TYPE = "1" HAVE_DIRENT_H = "1" HAVE_DIRFD = "1" HAVE_DLFCN_H = "1" HAVE_DLOPEN = "1" HAVE_DUP2 = "1" HAVE_DUP3 = "1" HAVE_DYNAMIC_LOADING = "1" HAVE_ENDIAN_H = "1" HAVE_EPOLL = "1" HAVE_EPOLL_CREATE1 = "1" HAVE_ERF = "1" HAVE_ERFC = "1" HAVE_ERRNO_H = "1" HAVE_EXECV = "1" HAVE_EXPLICIT_BZERO = "1" HAVE_EXPLICIT_MEMSET = "0" HAVE_EXPM1 = "1" HAVE_FACCESSAT = "1" HAVE_FCHDIR = "1" HAVE_FCHMOD = "1" HAVE_FCHMODAT = "1" HAVE_FCHOWN = "1" HAVE_FCHOWNAT = "1" HAVE_FCNTL_H = "1" HAVE_FDATASYNC = "1" HAVE_FDOPENDIR = "1" HAVE_FDWALK = "0" HAVE_FEXECVE = "1" HAVE_FINITE = "1" HAVE_FLOCK = "1" HAVE_FORK = "1" HAVE_FORKPTY = "1" HAVE_FPATHCONF = "1" HAVE_FSEEK64 = "0" HAVE_FSEEKO = "1" HAVE_FSTATAT = "1" HAVE_FSTATVFS = "1" HAVE_FSYNC = "1" HAVE_FTELL64 = "0" HAVE_FTELLO = "1" HAVE_FTIME = "1" HAVE_FTRUNCATE = "1" HAVE_FUTIMENS = "1" HAVE_FUTIMES = "1" HAVE_FUTIMESAT = "1" HAVE_GAI_STRERROR = "1" HAVE_GAMMA = "1" HAVE_GCC_ASM_FOR_MC68881 = "0" HAVE_GCC_ASM_FOR_X64 = "1" HAVE_GCC_ASM_FOR_X87 = "1" HAVE_GCC_UINT128_T = "1" HAVE_GETADDRINFO = "1" HAVE_GETC_UNLOCKED = "1" HAVE_GETENTROPY = "1" HAVE_GETGRGID_R = "1" HAVE_GETGRNAM_R = "1" HAVE_GETGROUPLIST = "1" HAVE_GETGROUPS = "1" HAVE_GETHOSTBYNAME = "0" HAVE_GETHOSTBYNAME_R = "1" HAVE_GETHOSTBYNAME_R_3_ARG = "0" HAVE_GETHOSTBYNAME_R_5_ARG = "0" HAVE_GETHOSTBYNAME_R_6_ARG = "1" HAVE_GETITIMER = "1" HAVE_GETLOADAVG = "1" HAVE_GETLOGIN = "1" HAVE_GETNAMEINFO = "1" HAVE_GETPAGESIZE = "1" HAVE_GETPEERNAME = "1" HAVE_GETPGID = "1" HAVE_GETPGRP = "1" HAVE_GETPID = "1" HAVE_GETPRIORITY = "1" HAVE_GETPWENT = "1" HAVE_GETPWNAM_R = "1" HAVE_GETPWUID_R = "1" HAVE_GETRANDOM = "1" HAVE_GETRANDOM_SYSCALL = "1" HAVE_GETRESGID = "1" HAVE_GETRESUID = "1" HAVE_GETSID = "1" HAVE_GETSPENT = "1" HAVE_GETSPNAM = "1" HAVE_GETTIMEOFDAY = "1" HAVE_GETWD = "1" HAVE_GLIBC_MEMMOVE_BUG = "0" HAVE_GRP_H = "1" HAVE_HSTRERROR = "1" HAVE_HTOLE64 = "1" HAVE_HYPOT = "1" HAVE_IEEEFP_H = "0" HAVE_IF_NAMEINDEX = "1" HAVE_INET_ATON = "1" HAVE_INET_PTON = "1" HAVE_INITGROUPS = "1" HAVE_INTTYPES_H = "1" HAVE_IO_H = "0" HAVE_IPA_PURE_CONST_BUG = "0" HAVE_KILL = "1" HAVE_KILLPG = "1" HAVE_KQUEUE = "0" HAVE_LANGINFO_H = "1" HAVE_LARGEFILE_SUPPORT = "0" HAVE_LCHFLAGS = "0" HAVE_LCHMOD = "0" HAVE_LCHOWN = "1" HAVE_LGAMMA = "1" HAVE_LIBDL = "1" HAVE_LIBDLD = "0" HAVE_LIBIEEE = "0" HAVE_LIBINTL_H = "1" HAVE_LIBREADLINE = "1" HAVE_LIBRESOLV = "0" HAVE_LIBSENDFILE = "0" HAVE_LIBUTIL_H = "0" HAVE_LINK = "1" HAVE_LINKAT = "1" HAVE_LINUX_CAN_BCM_H = "1" HAVE_LINUX_CAN_H = "1" HAVE_LINUX_CAN_RAW_FD_FRAMES = "1" HAVE_LINUX_CAN_RAW_H = "1" HAVE_LINUX_MEMFD_H = "1" HAVE_LINUX_NETLINK_H = "1" HAVE_LINUX_QRTR_H = "1" HAVE_LINUX_RANDOM_H = "1" HAVE_LINUX_TIPC_H = "1" HAVE_LINUX_VM_SOCKETS_H = "1" HAVE_LOCKF = "1" HAVE_LOG1P = "1" HAVE_LOG2 = "1" HAVE_LONG_DOUBLE = "1" HAVE_LSTAT = "1" HAVE_LUTIMES = "1" HAVE_MADVISE = "1" HAVE_MAKEDEV = "1" HAVE_MBRTOWC = "1" HAVE_MEMFD_CREATE = "1" HAVE_MEMORY_H = "1" HAVE_MEMRCHR = "1" HAVE_MKDIRAT = "1" HAVE_MKFIFO = "1" HAVE_MKFIFOAT = "1" HAVE_MKNOD = "1" HAVE_MKNODAT = "1" HAVE_MKTIME = "1" HAVE_MMAP = "1" HAVE_MREMAP = "1" HAVE_NCURSES_H = "1" HAVE_NDIR_H = "0" HAVE_NETPACKET_PACKET_H = "1" HAVE_NET_IF_H = "1" HAVE_NICE = "1" HAVE_OPENAT = "1" HAVE_OPENPTY = "1" HAVE_PATHCONF = "1" HAVE_PAUSE = "1" HAVE_PIPE2 = "1" HAVE_PLOCK = "0" HAVE_POLL = "1" HAVE_POLL_H = "1" HAVE_POSIX_FADVISE = "1" HAVE_POSIX_FALLOCATE = "1" HAVE_POSIX_SPAWN = "1" HAVE_POSIX_SPAWNP = "1" HAVE_PREAD = "1" HAVE_PREADV = "1" HAVE_PREADV2 = "1" HAVE_PRLIMIT = "1" HAVE_PROCESS_H = "0" HAVE_PROTOTYPES = "1" HAVE_PTHREAD_CONDATTR_SETCLOCK = "1" HAVE_PTHREAD_DESTRUCTOR = "0" HAVE_PTHREAD_GETCPUCLOCKID = "1" HAVE_PTHREAD_H = "1" HAVE_PTHREAD_INIT = "0" HAVE_PTHREAD_KILL = "1" HAVE_PTHREAD_SIGMASK = "1" HAVE_PTY_H = "1" HAVE_PUTENV = "1" HAVE_PWRITE = "1" HAVE_PWRITEV = "1" HAVE_PWRITEV2 = "1" HAVE_READLINK = "1" HAVE_READLINKAT = "1" HAVE_READV = "1" HAVE_REALPATH = "1" HAVE_RENAMEAT = "1" HAVE_RL_APPEND_HISTORY = "1" HAVE_RL_CATCH_SIGNAL = "1" HAVE_RL_COMPLETION_APPEND_CHARACTER = "1" HAVE_RL_COMPLETION_DISPLAY_MATCHES_HOOK = "1" HAVE_RL_COMPLETION_MATCHES = "1" HAVE_RL_COMPLETION_SUPPRESS_APPEND = "1" HAVE_RL_PRE_INPUT_HOOK = "1" HAVE_RL_RESIZE_TERMINAL = "1" HAVE_ROUND = "1" HAVE_RTPSPAWN = "0" HAVE_SCHED_GET_PRIORITY_MAX = "1" HAVE_SCHED_H = "1" HAVE_SCHED_RR_GET_INTERVAL = "1" HAVE_SCHED_SETAFFINITY = "1" HAVE_SCHED_SETPARAM = "1" HAVE_SCHED_SETSCHEDULER = "1" HAVE_SEM_GETVALUE = "1" HAVE_SEM_OPEN = "1" HAVE_SEM_TIMEDWAIT = "1" HAVE_SEM_UNLINK = "1" HAVE_SENDFILE = "1" HAVE_SETEGID = "1" HAVE_SETEUID = "1" HAVE_SETGID = "1" HAVE_SETGROUPS = "1" HAVE_SETHOSTNAME = "1" HAVE_SETITIMER = "1" HAVE_SETLOCALE = "1" HAVE_SETPGID = "1" HAVE_SETPGRP = "1" HAVE_SETPRIORITY = "1" HAVE_SETREGID = "1" HAVE_SETRESGID = "1" HAVE_SETRESUID = "1" HAVE_SETREUID = "1" HAVE_SETSID = "1" HAVE_SETUID = "1" HAVE_SETVBUF = "1" HAVE_SHADOW_H = "1" HAVE_SHM_OPEN = "1" HAVE_SHM_UNLINK = "1" HAVE_SIGACTION = "1" HAVE_SIGALTSTACK = "1" HAVE_SIGFILLSET = "1" HAVE_SIGINFO_T_SI_BAND = "1" HAVE_SIGINTERRUPT = "1" HAVE_SIGNAL_H = "1" HAVE_SIGPENDING = "1" HAVE_SIGRELSE = "1" HAVE_SIGTIMEDWAIT = "1" HAVE_SIGWAIT = "1" HAVE_SIGWAITINFO = "1" HAVE_SNPRINTF = "1" HAVE_SOCKADDR_ALG = "1" HAVE_SOCKADDR_SA_LEN = "0" HAVE_SOCKADDR_STORAGE = "1" HAVE_SOCKETPAIR = "1" HAVE_SPAWN_H = "1" HAVE_SSIZE_T = "1" HAVE_STATVFS = "1" HAVE_STAT_TV_NSEC = "1" HAVE_STAT_TV_NSEC2 = "0" HAVE_STDARG_PROTOTYPES = "1" HAVE_STDINT_H = "1" HAVE_STDLIB_H = "1" HAVE_STD_ATOMIC = "1" HAVE_STRDUP = "1" HAVE_STRFTIME = "1" HAVE_STRINGS_H = "1" HAVE_STRING_H = "1" HAVE_STRLCPY = "0" HAVE_STROPTS_H = "0" HAVE_STRSIGNAL = "1" HAVE_STRUCT_PASSWD_PW_GECOS = "1" HAVE_STRUCT_PASSWD_PW_PASSWD = "1" HAVE_STRUCT_STAT_ST_BIRTHTIME = "0" HAVE_STRUCT_STAT_ST_BLKSIZE = "1" HAVE_STRUCT_STAT_ST_BLOCKS = "1" HAVE_STRUCT_STAT_ST_FLAGS = "0" HAVE_STRUCT_STAT_ST_GEN = "0" HAVE_STRUCT_STAT_ST_RDEV = "1" HAVE_STRUCT_TM_TM_ZONE = "1" HAVE_SYMLINK = "1" HAVE_SYMLINKAT = "1" HAVE_SYNC = "1" HAVE_SYSCONF = "1" HAVE_SYSEXITS_H = "1" HAVE_SYS_AUDIOIO_H = "0" HAVE_SYS_BSDTTY_H = "0" HAVE_SYS_DEVPOLL_H = "0" HAVE_SYS_DIR_H = "0" HAVE_SYS_ENDIAN_H = "0" HAVE_SYS_EPOLL_H = "1" HAVE_SYS_EVENT_H = "0" HAVE_SYS_FILE_H = "1" HAVE_SYS_IOCTL_H = "1" HAVE_SYS_KERN_CONTROL_H = "0" HAVE_SYS_LOADAVG_H = "0" HAVE_SYS_LOCK_H = "0" HAVE_SYS_MEMFD_H = "0" HAVE_SYS_MKDEV_H = "0" HAVE_SYS_MMAN_H = "1" HAVE_SYS_MODEM_H = "0" HAVE_SYS_NDIR_H = "0" HAVE_SYS_PARAM_H = "1" HAVE_SYS_POLL_H = "1" HAVE_SYS_RANDOM_H = "1" HAVE_SYS_RESOURCE_H = "1" HAVE_SYS_SELECT_H = "1" HAVE_SYS_SENDFILE_H = "1" HAVE_SYS_SOCKET_H = "1" HAVE_SYS_STATVFS_H = "1" HAVE_SYS_STAT_H = "1" HAVE_SYS_SYSCALL_H = "1" HAVE_SYS_SYSMACROS_H = "1" HAVE_SYS_SYS_DOMAIN_H = "0" HAVE_SYS_TERMIO_H = "0" HAVE_SYS_TIMES_H = "1" HAVE_SYS_TIME_H = "1" HAVE_SYS_TYPES_H = "1" HAVE_SYS_UIO_H = "1" HAVE_SYS_UN_H = "1" HAVE_SYS_UTSNAME_H = "1" HAVE_SYS_WAIT_H = "1" HAVE_SYS_XATTR_H = "1" HAVE_TCGETPGRP = "1" HAVE_TCSETPGRP = "1" HAVE_TEMPNAM = "1" HAVE_TERMIOS_H = "1" HAVE_TERM_H = "1" HAVE_TGAMMA = "1" HAVE_TIMEGM = "1" HAVE_TIMES = "1" HAVE_TMPFILE = "1" HAVE_TMPNAM = "1" HAVE_TMPNAM_R = "1" HAVE_TM_ZONE = "1" HAVE_TRUNCATE = "1" HAVE_TZNAME = "0" HAVE_UCS4_TCL = "0" HAVE_UNAME = "1" HAVE_UNISTD_H = "1" HAVE_UNLINKAT = "1" HAVE_UNSETENV = "1" HAVE_USABLE_WCHAR_T = "0" HAVE_UTIL_H = "0" HAVE_UTIMENSAT = "1" HAVE_UTIMES = "1" HAVE_UTIME_H = "1" HAVE_UUID_CREATE = "0" HAVE_UUID_ENC_BE = "0" HAVE_UUID_GENERATE_TIME_SAFE = "1" HAVE_UUID_H = "0" HAVE_UUID_UUID_H = "1" HAVE_WAIT3 = "1" HAVE_WAIT4 = "1" HAVE_WAITID = "1" HAVE_WAITPID = "1" HAVE_WCHAR_H = "1" HAVE_WCSCOLL = "1" HAVE_WCSFTIME = "1" HAVE_WCSXFRM = "1" HAVE_WMEMCMP = "1" HAVE_WORKING_TZSET = "1" HAVE_WRITEV = "1" HAVE_X509_VERIFY_PARAM_SET1_HOST = "1" HAVE_ZLIB_COPY = "1" HAVE__GETPTY = "0" HOST_GNU_TYPE = "x86_64pclinuxgnu" INCLDIRSTOMAKE = "/usr/include /usr/include /usr/include/python3.8 /usr/include/python3.8" INCLUDEDIR = "/usr/include" INCLUDEPY = "/usr/include/python3.8" INSTALL = "/usr/bin/install c" INSTALL_DATA = "/usr/bin/install c m 644" INSTALL_PROGRAM = "/usr/bin/install c" INSTALL_SCRIPT = "/usr/bin/install c" INSTALL_SHARED = "/usr/bin/install c m 755" INSTSONAME = "libpython3.8.so.1.0" IO_H = "Modules/_io/_iomodule.h" IO_OBJS = "\" LDCXXSHARED = "x86_64linuxgnug++ pthread shared Wl,O1 Wl,Bsymbolicfunctions" LDFLAGS = "Wl,Bsymbolicfunctions Wl,z,relro g fwrapv O2 " LDFLAGS_NODIST = "" LDLIBRARY = "libpython3.8.so" LDLIBRARYDIR = "" LDSHARED = "x86_64linuxgnugcc pthread shared Wl,O1 Wl,Bsymbolicfunctions Wl,Bsymbolicfunctions Wl,z,relro g fwrapv O2 " LDVERSION = "3.8" LIBC = "" LIBDEST = "/usr/lib/python3.8" LIBDIR = "/usr/lib" LIBFFI_INCLUDEDIR = "" LIBM = "lm" LIBOBJDIR = "Python/" LIBOBJS = "" LIBPC = "/usr/lib/x86_64linuxgnu/pkgconfig" LIBPL = "/usr/lib/python3.8/config3.8x86_64linuxgnu" LIBPYTHON = "" LIBRARY = "libpython3.8.a" LIBRARY_OBJS = "\" LIBRARY_OBJS_OMIT_FROZEN = "\" LIBS = "lcrypt lpthread ldl lutil lm" LIBSUBDIRS = "tkinter tkinter/test tkinter/test/test_tkinter \" LINKCC = "x86_64linuxgnugcc pthread" LINKFORSHARED = "Xlinker exportdynamic Wl,O1 Wl,Bsymbolicfunctions" LIPO_32BIT_FLAGS = "" LLVM_PROF_ERR = "no" LLVM_PROF_FILE = "" LLVM_PROF_MERGER = "true" LN = "ln" LOCALMODLIBS = "lexpat L/usr/lib lz lexpat" MACHDEP = "linux" MACHDEP_OBJS = "" MACHDESTLIB = "/usr/lib/python3.8" MACOSX_DEPLOYMENT_TARGET = "" MAINCC = "x86_64linuxgnugcc pthread" MAJOR_IN_MKDEV = "0" MAJOR_IN_SYSMACROS = "1" MAKESETUP = "../Modules/makesetup" MANDIR = "/usr/share/man" MKDIR_P = "/bin/mkdir p" MODBUILT_NAMES = "array cmath math _struct _random _elementtree _pickle _datetime _bisect _heapq _statistics unicodedata fcntl spwd grp select _csv _socket _posixsubprocess _md5 _sha1 _sha256 _sha512 _sha3 _blake2 syslog binascii zlib posix errno pwd _sre _codecs _weakref _functools _operator _collections _abc itertools atexit _signal _stat time _thread _locale _io faulthandler _tracemalloc _symtable pyexpat xxsubtype" MODDISABLED_NAMES = "" MODLIBS = "lexpat L/usr/lib lz lexpat" MODOBJS = "$(sort Modules/arraymodule.o Modules/cmathmodule.o Modules/_math.o Modules/mathmodule.o Modules/_math.o Modules/_struct.o Modules/_randommodule.o Modules/_elementtree.o Modules/_pickle.o Modules/_datetimemodule.o Modules/_bisectmodule.o Modules/_heapqmodule.o Modules/_statisticsmodule.o Modules/unicodedata.o Modules/fcntlmodule.o Modules/spwdmodule.o Modules/grpmodule.o Modules/selectmodule.o Modules/_csv.o Modules/socketmodule.o Modules/_posixsubprocess.o Modules/md5module.o Modules/sha1module.o Modules/sha256module.o Modules/sha512module.o Modules/sha3module.o Modules/blake2module.o Modules/blake2b_impl.o Modules/blake2s_impl.o Modules/syslogmodule.o Modules/binascii.o Modules/zlibmodule.o Modules/posixmodule.o Modules/errnomodule.o Modules/pwdmodule.o Modules/_sre.o Modules/_codecsmodule.o Modules/_weakref.o Modules/_functoolsmodule.o Modules/_operator.o Modules/_collectionsmodule.o Modules/_abc.o Modules/itertoolsmodule.o Modules/atexitmodule.o Modules/signalmodule.o Modules/_stat.o Modules/timemodule.o Modules/_threadmodule.o Modules/_localemodule.o Modules/_iomodule.o Modules/iobase.o Modules/fileio.o Modules/bytesio.o Modules/bufferedio.o Modules/textio.o Modules/stringio.o Modules/faulthandler.o Modules/_tracemalloc.o Modules/hashtable.o Modules/symtablemodule.o Modules/pyexpat.o Modules/xxsubtype.o)" MODULE_OBJS = "\" MULTIARCH = "x86_64linuxgnu" MULTIARCH_CPPFLAGS = "DMULTIARCH=\"x86_64linuxgnu\"" MVWDELCH_IS_EXPRESSION = "1" NO_AS_NEEDED = "Wl,noasneeded" OBJECT_OBJS = "\" OPENSSL_INCLUDES = "" OPENSSL_LDFLAGS = "" OPENSSL_LIBS = "lssl lcrypto" OPT = "DNDEBUG g fwrapv O2 Wall" OTHER_LIBTOOL_OPT = "" PACKAGE_BUGREPORT = "0" PACKAGE_NAME = "0" PACKAGE_STRING = "0" PACKAGE_TARNAME = "0" PACKAGE_URL = "0" PACKAGE_VERSION = "0" PARSER_HEADERS = "\" PARSER_OBJS = "\ Parser/myreadline.o Parser/parsetok.o Parser/tokenizer.o" PGO_PROF_GEN_FLAG = "fprofilegenerate" PGO_PROF_USE_FLAG = "" POBJS = "\" POSIX_SEMAPHORES_NOT_ENABLED = "0" PROFILE_TASK = "m test pgo" PTHREAD_KEY_T_IS_COMPATIBLE_WITH_INT = "1" PTHREAD_SYSTEM_SCHED_SUPPORTED = "1" PURIFY = "" PY3LIBRARY = "libpython3.so" PYLONG_BITS_IN_DIGIT = "0" PYTHON = "python" PYTHONFRAMEWORK = "" PYTHONFRAMEWORKDIR = "noframework" PYTHONFRAMEWORKINSTALLDIR = "" PYTHONFRAMEWORKPREFIX = "" PYTHONPATH = "" PYTHON_FOR_BUILD = "./python E" PYTHON_FOR_REGEN = "python3.8" PYTHON_HEADERS = "\" PYTHON_OBJS = "\" PY_BUILTIN_MODULE_CFLAGS = "Wnounusedresult Wsigncompare DNDEBUG g fwrapv O2 Wall g fstackprotectorstrong Wformat Werror=formatsecurity g fwrapv O2 std=c99 Wextra Wnounusedresult Wnounusedparameter Wnomissingfieldinitializers Werror=implicitfunctiondeclaration I../Include/internal IObjects IInclude IPython I. I../Include Wdatetime D_FORTIFY_SOURCE=2 fPIC DPy_BUILD_CORE_BUILTIN" PY_CFLAGS = "Wnounusedresult Wsigncompare DNDEBUG g fwrapv O2 Wall g fstackprotectorstrong Wformat Werror=formatsecurity g fwrapv O2 " PY_CFLAGS_NODIST = "std=c99 Wextra Wnounusedresult Wnounusedparameter Wnomissingfieldinitializers Werror=implicitfunctiondeclaration I../Include/internal" PY_COERCE_C_LOCALE = "1" PY_CORE_CFLAGS = "Wnounusedresult Wsigncompare DNDEBUG g fwrapv O2 Wall g fstackprotectorstrong Wformat Werror=formatsecurity g fwrapv O2 std=c99 Wextra Wnounusedresult Wnounusedparameter Wnomissingfieldinitializers Werror=implicitfunctiondeclaration I../Include/internal IObjects IInclude IPython I. I../Include Wdatetime D_FORTIFY_SOURCE=2 fPIC DPy_BUILD_CORE" PY_CORE_LDFLAGS = "Wl,Bsymbolicfunctions Wl,z,relro g fwrapv O2 " PY_CPPFLAGS = "IObjects IInclude IPython I. I../Include Wdatetime D_FORTIFY_SOURCE=2" PY_FORMAT_SIZE_T = ""z"" PY_LDFLAGS = "Wl,Bsymbolicfunctions Wl,z,relro g fwrapv O2 " PY_LDFLAGS_NODIST = "" PY_SSL_DEFAULT_CIPHERS = "1" PY_SSL_DEFAULT_CIPHER_STRING = "0" PY_STDMODULE_CFLAGS = "Wnounusedresult Wsigncompare DNDEBUG g fwrapv O2 Wall g fstackprotectorstrong Wformat Werror=formatsecurity g fwrapv O2 std=c99 Wextra Wnounusedresult Wnounusedparameter Wnomissingfieldinitializers Werror=implicitfunctiondeclaration I../Include/internal IObjects IInclude IPython I. I../Include Wdatetime D_FORTIFY_SOURCE=2 fPIC" Py_DEBUG = "0" Py_ENABLE_SHARED = "1" Py_HASH_ALGORITHM = "0" Py_TRACE_REFS = "0" QUICKTESTOPTS = "x test_subprocess test_io test_lib2to3 \" READELF = "readelf" RESSRCDIR = "Mac/Resources/framework" RETSIGTYPE = "void" RUNSHARED = "LD_LIBRARY_PATH=/build/python3.86QL2k7/python3.83.8.2/buildshared" SCRIPTDIR = "/usr/lib" SETPGRP_HAVE_ARG = "0" SGI_ABI = "@SGI_ABI@" SHELL = "/bin/sh" SHLIBS = "lcrypt lpthread ldl lutil lm" SHLIB_SUFFIX = ".so" SHM_NEEDS_LIBRT = "1" SIGNED_RIGHT_SHIFT_ZERO_FILLS = "0" SITEPATH = "" SIZEOF_DOUBLE = "8" SIZEOF_FLOAT = "4" SIZEOF_FPOS_T = "16" SIZEOF_INT = "4" SIZEOF_LONG = "8" SIZEOF_LONG_DOUBLE = "16" SIZEOF_LONG_LONG = "8" SIZEOF_OFF_T = "8" SIZEOF_PID_T = "4" SIZEOF_PTHREAD_KEY_T = "4" SIZEOF_PTHREAD_T = "8" SIZEOF_SHORT = "2" SIZEOF_SIZE_T = "8" SIZEOF_TIME_T = "8" SIZEOF_UINTPTR_T = "8" SIZEOF_VOID_P = "8" SIZEOF_WCHAR_T = "4" SIZEOF__BOOL = "1" SO = ".cpython38x86_64linuxgnu.so" SOABI = "cpython38x86_64linuxgnu" SRCDIRS = "Parser Objects Python Modules Modules/_io Programs" SRC_GDB_HOOKS = "../Tools/gdb/libpython.py" STDC_HEADERS = "1" STRICT_SYSV_CURSES = "/* Don't use ncurses extensions */" STRIPFLAG = "s" SUBDIRS = "" SUBDIRSTOO = "Include Lib Misc" SYSLIBS = "lm" SYS_SELECT_WITH_SYS_TIME = "1" TCLTK_INCLUDES = "" TCLTK_LIBS = "" TESTOPTS = "" TESTPATH = "" TESTPYTHON = "LD_LIBRARY_PATH=/build/python3.86QL2k7/python3.83.8.2/buildshared ./python" TESTPYTHONOPTS = "" TESTRUNNER = "LD_LIBRARY_PATH=/build/python3.86QL2k7/python3.83.8.2/buildshared ./python ../Tools/scripts/run_tests.py" TESTTIMEOUT = "1200" TIMEMODULE_LIB = "0" TIME_WITH_SYS_TIME = "1" TM_IN_SYS_TIME = "0" UNICODE_DEPS = "\" UNIVERSALSDK = "" UPDATE_FILE = "python3.8 ../Tools/scripts/update_file.py" USE_COMPUTED_GOTOS = "1" VERSION = "3.8" VPATH = ".." WINDOW_HAS_FLAGS = "1" WITH_DOC_STRINGS = "1" WITH_DTRACE = "1" WITH_DYLD = "0" WITH_LIBINTL = "0" WITH_NEXT_FRAMEWORK = "0" WITH_PYMALLOC = "1" WITH_VALGRIND = "0" X87_DOUBLE_ROUNDING = "0" XMLLIBSUBDIRS = "xml xml/dom xml/etree xml/parsers xml/sax" abiflags = "" abs_builddir = "/build/python3.86QL2k7/python3.83.8.2/buildshared" abs_srcdir = "/build/python3.86QL2k7/python3.83.8.2/buildshared/.." base = "/mnt/d/Programming/Projects/sage/src/.venv" datarootdir = "/usr/share" exec_prefix = "/usr" installed_base = "/usr" installed_platbase = "/usr" multiarchsubdir = "/x86_64linuxgnu" platbase = "/mnt/d/Programming/Projects/sage/src/.venv" prefix = "/usr" projectbase = "/usr/bin" py_version = "3.8.2" py_version_nodot = "38" py_version_short = "3.8" srcdir = "/usr/lib/python3.8/config3.8x86_64linuxgnu" userbase = "/home/tobias/.local"
comment:143 Changed 4 months ago by
I see MULTIARCH = "x86_64linuxgnu"
and multiarchsubdir = "/x86_64linuxgnu"
.
comment:144 Changed 4 months ago by
One of these variables should be used in _get_shared_lib_filename
. Why it is instead looking for MULTILIB
, which indeed does not exist, should be investigated...
comment:145 Changed 4 months ago by
Let's make this a separate ticket please
comment:146 Changed 4 months ago by
This is now #30833
comment:147 Changed 4 months ago by
 Dependencies changed from #30779, #30672, #30673, #30709, #30706 to #30779, #30672, #30673, #30709, #30706, #30833
comment:148 Changed 4 months ago by
 Commit changed from 7523baa68b46732cc381387e58ea1a9559c11e96 to 443424cf878ac7fa9c57a1bad313f1b4d84ed031
comment:149 Changed 4 months ago by
 Dependencies changed from #30779, #30672, #30673, #30709, #30706, #30833 to #30779, #30672, #30673, #30709, #30706, #30833, #30748
comment:150 Changed 4 months ago by
 Commit changed from 443424cf878ac7fa9c57a1bad313f1b4d84ed031 to b3045684c751317f252ff61f59ab53782df76177
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
2351ae3  Update, extend and reorder dependencies

99ae00c  Replace startup exception by warning

11882e5  Use context manager

d858e46  Merge branch 'public/build/startupWarning' of git://trac.sagemath.org/sage into public/build/inplace

46ac365  Update numpy

3fcd71f  Merge tag '9.3.beta0' into t/30709/public/build/cythonConfig

a646a24  src/sage_setup/cython_options.py: Restore auto_pickle=False

8eae4bc  Merge branch 'public/build/cythonConfig' of git://trac.sagemath.org/sage into public/build/inplace

5ac3a29  Add matlibplot library

b304568  Run doctests in github action

comment:151 Changed 4 months ago by
 Commit changed from b3045684c751317f252ff61f59ab53782df76177 to 9a94a29bc1c6179282dbd94a7afd19247884e525
Branch pushed to git repo; I updated commit sha1. New commits:
9a94a29  Don't install texlivegenericextra

comment:152 Changed 4 months ago by
 Commit changed from 9a94a29bc1c6179282dbd94a7afd19247884e525 to 16674ba8da9daa53709c9ad4bfc6fca6bdba13ee
Branch pushed to git repo; I updated commit sha1. New commits:
16674ba  Remove lazy import finish startup

comment:153 Changed 4 months ago by
 Commit changed from 16674ba8da9daa53709c9ad4bfc6fca6bdba13ee to 511775ec71b3d2efef8f6e9ca2ea95f234c566ad
comment:154 Changed 4 months ago by
I've now added the doctest run to the github workflow. It works in principle although there are quite a lot of failing tests (mainly because some of the systeminstalled libs are not discovered). In order to keep the focus of this ticket narrow, fixing this should better be done in followup tickets.
comment:155 Changed 4 months ago by
 Description modified (diff)
comment:156 followup: ↓ 200 Changed 4 months ago by
comment:157 Changed 4 months ago by
 Commit changed from 511775ec71b3d2efef8f6e9ca2ea95f234c566ad to 373d1c12e77647541fa7fc8b315aae98ee0298cd
Branch pushed to git repo; I updated commit sha1. New commits:
373d1c1  Add rope as dev dependency

comment:158 in reply to: ↑ description Changed 4 months ago by
Replying to mkoeppe:
 A few libraries are not contained in
distros/debian.txt
files. Should they be added? In particular, this concerns libsingular4dev, liblinboxdev, libratpointsdev, libgapdev, ecl, libpynacdev.
Yes, let's add them in #30859.
Some of the information that you put in the ticket description above could go as comments into the debian.txt files.
comment:159 Changed 4 months ago by
Thanks for opening a new ticket for this!
Concerning the conda integration, this ticket here might actually help with this. You can use pipenv on top of conda, see https://pipenv.pypa.io/en/latest/advanced/#pipenvandotherpythondistributions.
comment:160 Changed 4 months ago by
 Description modified (diff)
comment:161 Changed 4 months ago by
 Commit changed from 373d1c12e77647541fa7fc8b315aae98ee0298cd to 096df15a1253e6cb0fa75c92e29458f29d5ad77d
Branch pushed to git repo; I updated commit sha1. New commits:
096df15  Merge branch 'develop' of git://github.com/sagemath/sage into public/build/inplace

comment:162 Changed 4 months ago by
 Commit changed from 096df15a1253e6cb0fa75c92e29458f29d5ad77d to 8eb2f8ec11943c8f7175ce4d8a293673ab18b774
Branch pushed to git repo; I updated commit sha1. New commits:
8eb2f8e  Remove homfly workaround

comment:163 Changed 4 months ago by
 Commit changed from 8eb2f8ec11943c8f7175ce4d8a293673ab18b774 to 17f367eebf0e329275773e0c048f3009b2de19ed
Branch pushed to git repo; I updated commit sha1. New commits:
17f367e  Refactor shared library discovery

comment:164 Changed 4 months ago by
 Dependencies changed from #30779, #30672, #30673, #30709, #30706, #30833, #30748 to #30779, #30672, #30673, #30709, #30706, #30901, #30748
comment:165 Changed 4 months ago by
 Commit changed from 17f367eebf0e329275773e0c048f3009b2de19ed to 680f85d72cf88ae96e0bc4bf7b422e694f353437
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
c47c4bf  Correct indent

23b6336  Merge branch 'public/build/multiarchsimple' of git://trac.sagemath.org/sage into public/build/inplace

6a52fbf  Remove lazy import finish startup

f96025e  Merge branch 'develop' of git://github.com/sagemath/sage into public/build/startupWarning

b952bb5  Fix doctests

c6857dd  Hopefully fix doctest

23da14b  Wrap each sage.all import in startup guard

2f4055e  Specify that exit was succesful

d4d94a8  Indeed replace exception by print statement

680f85d  Merge branch 'public/build/startupWarning' of git://trac.sagemath.org/sage into public/build/inplace

comment:166 Changed 4 months ago by
 Commit changed from 680f85d72cf88ae96e0bc4bf7b422e694f353437 to ddcf2e3ba09cb3731042c1372b93e4db84f93b13
Branch pushed to git repo; I updated commit sha1. New commits:
ddcf2e3  Cleanup code

comment:167 Changed 4 months ago by
 Commit changed from ddcf2e3ba09cb3731042c1372b93e4db84f93b13 to 772baf208259ff75edebdbf6f340c700be0681ab
comment:168 Changed 4 months ago by
I've now merged the latest version of the dependencies, and cleaned the code.
Matthias (and others, of course), it would be nice if you could review the code (and its dependencies). I have a hard time working on other tickets as long as this ticket is not merged. So I would be grateful if that happens as soon as possible. Thanks!
comment:169 followup: ↓ 171 Changed 4 months ago by
See if you can get doctesting to work on this branch  by merging #30578.
comment:170 followup: ↓ 172 Changed 4 months ago by
Also removing sageenvconfig
from the list of installed scripts is not good. Instead please review and merge #29850  which reassigns this script to the sage_conf
package.
comment:171 in reply to: ↑ 169 ; followup: ↓ 174 Changed 4 months ago by
comment:172 in reply to: ↑ 170 Changed 4 months ago by
Replying to mkoeppe:
Also removing
sageenvconfig
from the list of installed scripts is not good. Instead please review and merge #29850  which reassigns this script to thesage_conf
package.
I agree #29850 is a better solution. I had a look at that ticket, but don't really feel comfortable enough with the code to review it. Sorry! However, I've now added it as a dependency of this branch, and will merge as soon as it is reviewed.
comment:173 Changed 4 months ago by
 Dependencies changed from #30779, #30672, #30673, #30709, #30706, #30901, #30748 to #30779, #30672, #30673, #30709, #30706, #30901, #30748, #29850
comment:174 in reply to: ↑ 171 ; followup: ↓ 177 Changed 4 months ago by
Replying to mkoeppe:
Replying to mkoeppe:
See if you can get doctesting to work on this branch  by merging #30578.
In #30578 I had to add a few more packages to
requirements.txt
that you don't have inPipfile
yet, so that more doctests pass.
Thanks for the pointer! I've now added these packages also to the pipfile. However, it is way beyond the scope of this ticket to make the doctest pass in the GH pipenv workflow. There are just so many things that need to be changed in order for the doctests to find the correct dependencies. For example, here are a few relevant error messages:
FileNotFoundError: [Errno 2] No such file or directory: '/mnt/d/Programming/Projects/sage/src/.venv/bin/gap' RuntimeError: unable to start singular because the command 'Singular t tickspersec 1000 cntrlc=a' failed: The command was not found or was not executable: Singular. RuntimeError: In order to initialize the database, /__w/sage/sage/src/.venv/share/conway_polynomials/conway_polynomials.p must exist.
I know that some of these errors would disappear if one use your recent progress towards installation of sagebuild wheels.
Anyway, the aim of this ticket is to create a mostlyworking editable install of sagelib. Making the doctests pass relative to this editable install in a venv should be subject of a followup ticket.
comment:175 Changed 4 months ago by
 Commit changed from 772baf208259ff75edebdbf6f340c700be0681ab to ad0f33cd4c67d2eb31cbf864e8112caeb8958990
comment:176 Changed 4 months ago by
 Commit changed from ad0f33cd4c67d2eb31cbf864e8112caeb8958990 to 77e1af16bcfce183bfe072cfb7324e82ee95a478
comment:177 in reply to: ↑ 174 Changed 4 months ago by
Replying to ghtobiasdiez:
Anyway, the aim of this ticket is to create a mostlyworking editable install of sagelib.
Most of the changes that you are doing in this ticket have really nothing to do with this goal. As I have pointed out previously, you are trying to solve the orthogonal problem of making sagelib work outside of the Sage distribution, i.e., without SAGE_LOCAL.
comment:178 followup: ↓ 179 Changed 4 months ago by
I agree. The thing is that it is currently my only way to work with sage, and thus test the editable install. But it's good that you are pushing from the other side and work on the editable install based on a working sage distribution.
So, is there something that I can still do here? I see that verifying and testing that the editable install works is important. It's just not very realistic to have a fully functional doctest without SAGE_LOCAL at the moment. However, I can also not very actively contribute to the editable install with SAGE_LOCAL.
comment:179 in reply to: ↑ 178 ; followup: ↓ 181 Changed 4 months ago by
Replying to ghtobiasdiez:
I agree. The thing is that it is currently my only way to work with sage, and thus test the editable install.
Is the WSL build of the Sage distribution still failing for you? Have you tried a different Linux distribution? So far the WSL runs on GH Actions have not been very conclusive  they have failed for unrelated reasons  you could work on #30910 to improve it
comment:180 followup: ↓ 182 Changed 4 months ago by
Also, I am interested in a GH Actions workflow for the various new installations (venv based on requirements on top of SAGE_LOCAL, venv based on Pipfile on top of SAGE_LOCAL, conda without SAGE_LOCAL, etc.) This ticket adds something like this, but I don't like that it does Ubuntuspecific things again. We have a better way of doing such things already so that everything can be tested on all supported distributions, not just some Ubuntu. I would hope that we can add such workflows in #30845. #30859 will help with this. The powershell bits to work with system packages could also be valuable  see #30865.
comment:181 in reply to: ↑ 179 Changed 4 months ago by
Replying to mkoeppe:
Is the WSL build of the Sage distribution still failing for you? Have you tried a different Linux distribution?
Yes, it's still not working. In a few weeks I'll get a new PC and then will try again afresh using WSL 2.0.
comment:182 in reply to: ↑ 180 Changed 4 months ago by
Replying to mkoeppe:
This ticket adds something like this, but I don't like that it does Ubuntuspecific things again. We have a better way of doing such things already so that everything can be tested on all supported distributions, not just some Ubuntu.
Yes and no. Since the whole workflow only takes about 2 hours, my hope was that one can use it in the longterm to run it on each push to each branch, replacing/complementing the patchbot. But of course it could also serve as the blueprint for a complete matrix integration test suite. Anyway, for the moment it serves as a record of the things I did locally on my system, and proofs that the editable install without SAGE_LOCAL works to a large degree.
comment:183 Changed 4 months ago by
 Dependencies changed from #30779, #30672, #30673, #30709, #30706, #30901, #30748, #29850 to #30779, #30672, #30673, #30709, #30706, #30901, #30748, #29850, #29314
After playing around with it further I was able to make most of the tests pass. There are still a few hundreds that fail (see below for details), most of which are related to issues with maxima (TypeError?: unable to start maxima: End Of File (EOF). Exception style platform.) or fpylll (blocked due to https://github.com/fplll/fpylll/issues/185). Since most test pass and there are many subpackages for which all tests pass, I'd say that's enough "proof of concept" for this ticket and would leave the rest for followup tickets.
20201117T19:31:00.7800025Z sage t randomseed=0 /__w/sage/sage/src/sage/interacts/debugger.py # 2 doctests failed 20201117T19:31:00.7801032Z sage t randomseed=0 /__w/sage/sage/src/sage/interfaces/singular.py # 5 doctests failed 20201117T19:31:00.7802028Z sage t randomseed=0 /__w/sage/sage/src/sage/interfaces/maxima_lib.py # 1 doctest failed 20201117T19:31:00.7803005Z sage t randomseed=0 /__w/sage/sage/src/sage/interfaces/quit.py # 6 doctests failed 20201117T19:31:00.7803974Z sage t randomseed=0 /__w/sage/sage/src/sage/interfaces/maxima.py # 144 doctests failed 20201117T19:31:00.7804998Z sage t randomseed=0 /__w/sage/sage/src/sage/interfaces/maxima_abstract.py # 168 doctests failed 20201117T19:31:00.7806039Z sage t randomseed=0 /__w/sage/sage/src/sage/interfaces/interface.py # 90 doctests failed 20201117T19:31:00.7807034Z sage t randomseed=0 /__w/sage/sage/src/sage/interfaces/expect.py # 6 doctests failed 20201117T19:31:00.7808001Z sage t randomseed=0 /__w/sage/sage/src/sage/interfaces/tests.py # 3 doctests failed 20201117T19:31:00.7809010Z sage t randomseed=0 /__w/sage/sage/src/sage/interfaces/gap_workspace.py # 2 doctests failed 20201117T19:31:00.7810009Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/rational.pyx # 2 doctests failed 20201117T19:31:00.7810921Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/ring.pyx # 1 doctest failed 20201117T19:31:00.7811830Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/infinity.py # 1 doctest failed 20201117T19:31:00.7812754Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/real_mpfr.pyx # 9 doctests failed 20201117T19:31:00.7813669Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/real_mpfi.pyx # 3 doctests failed 20201117T19:31:00.7814600Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/complex_mpfr.pyx # 2 doctests failed 20201117T19:31:00.7815569Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/localization.py # 1 doctest failed 20201117T19:31:00.7816785Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/tests.py # 2 doctests failed 20201117T19:31:00.7817714Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/integer_ring.pyx # 5 doctests failed 20201117T19:31:00.7818771Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/polynomial/polynomial_zmod_flint.pyx # 1 doctest failed 20201117T19:31:00.7819934Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/polynomial/polynomial_quotient_ring.py # 1 doctest failed 20201117T19:31:00.7821093Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/polynomial/polynomial_modn_dense_ntl.pyx # 6 doctests failed 20201117T19:31:00.7822245Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/polynomial/polynomial_element.pyx # 13 doctests failed 20201117T19:31:00.7823414Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/polynomial/polynomial_number_field.pyx # 19 doctests failed 20201117T19:31:00.7824560Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/polynomial/multi_polynomial_ring.py # 2 doctests failed 20201117T19:31:00.7825670Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/finite_rings/residue_field.pyx # 14 doctests failed 20201117T19:31:00.7826727Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/number_field/S_unit_solver.py # 18 doctests failed 20201117T19:31:00.7827787Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/number_field/number_field_base.pyx # 15 doctests failed 20201117T19:31:00.7828969Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/number_field/class_group.py # 2 doctests failed 20201117T19:31:00.7830020Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/number_field/number_field_ideal.py # 1 doctest failed 20201117T19:31:00.7831068Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/number_field/galois_group.py # 58 doctests failed 20201117T19:31:00.7832134Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/number_field/splitting_field.py # 28 doctests failed 20201117T19:31:00.7833172Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/number_field/order.py # 6 doctests failed 20201117T19:31:00.7834186Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/number_field/number_field.py # 33 doctests failed 20201117T19:31:00.7835200Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/number_field/unit_group.py # 1 doctest failed 20201117T19:31:00.7836234Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/number_field/number_field_rel.py # 11 doctests failed 20201117T19:31:00.7837488Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/number_field/number_field_element.pyx # 27 doctests failed 20201117T19:31:00.7838576Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/number_field/bdd_height.py # 2 doctests failed 20201117T19:31:00.7839663Z sage t randomseed=0 /__w/sage/sage/src/sage/rings/valuation/inductive_valuation.py # 4 doctests failed 20201117T19:31:00.7840725Z sage t randomseed=0 /__w/sage/sage/src/sage/cpython/dict_del_by_value.pyx # 2 doctests failed 20201117T19:31:00.7841710Z sage t randomseed=0 /__w/sage/sage/src/sage/databases/cremona.py # 2 doctests failed 20201117T19:31:00.7842673Z sage t randomseed=0 /__w/sage/sage/src/sage/groups/cubic_braid.py # 6 doctests failed 20201117T19:31:00.7843635Z sage t randomseed=0 /__w/sage/sage/src/sage/sandpiles/sandpile.py # 5 doctests failed 20201117T19:31:00.7844616Z sage t randomseed=0 /__w/sage/sage/src/sage/coding/gabidulin_code.py # 1 doctest failed 20201117T19:31:00.7845566Z sage t randomseed=0 /__w/sage/sage/src/sage/tests/cmdline.py # 59 doctests failed 20201117T19:31:00.7846502Z sage t randomseed=0 /__w/sage/sage/src/sage/tests/benchmark.py # 4 doctests failed 20201117T19:31:00.7847448Z sage t randomseed=0 /__w/sage/sage/src/sage/tests/gap_packages.py # 1 doctest failed 20201117T19:31:00.7848644Z sage t randomseed=0 /__w/sage/sage/src/sage/tests/books/judsonabstractalgebra/galoissage.py # 9 doctests failed 20201117T19:31:00.7850157Z sage t randomseed=0 /__w/sage/sage/src/sage/tests/books/judsonabstractalgebra/fieldssage.py # 17 doctests failed 20201117T19:31:00.7851577Z sage t randomseed=0 /__w/sage/sage/src/sage/tests/books/judsonabstractalgebra/domainssage.py # 5 doctests failed 20201117T19:31:00.7853321Z sage t randomseed=0 /__w/sage/sage/src/sage/tests/books/computationalmathematicswithsagemath/polynomes_doctest.py # 1 doctest failed 20201117T19:31:00.7855341Z sage t randomseed=0 /__w/sage/sage/src/sage/tests/books/computationalmathematicswithsagemath/domaines_doctest.py # 3 doctests failed 20201117T19:31:00.7857340Z sage t randomseed=0 /__w/sage/sage/src/sage/tests/books/computationalmathematicswithsagemath/combinat_doctest.py # 1 doctest failed 20201117T19:31:00.7859360Z sage t randomseed=0 /__w/sage/sage/src/sage/tests/books/computationalmathematicswithsagemath/integration_doctest.py # 1 doctest failed 20201117T19:31:00.7861360Z sage t randomseed=0 /__w/sage/sage/src/sage/tests/books/computationalmathematicswithsagemath/mpoly_doctest.py # 2 doctests failed 20201117T19:31:00.7863355Z sage t randomseed=0 /__w/sage/sage/src/sage/tests/books/computationalmathematicswithsagemath/calculus_doctest.py # 3 doctests failed 20201117T19:31:00.7865352Z sage t randomseed=0 /__w/sage/sage/src/sage/tests/books/computationalmathematicswithsagemath/sol/mpoly_doctest.py # 2 doctests failed 20201117T19:31:00.7867492Z sage t randomseed=0 /__w/sage/sage/src/sage/tests/books/computationalmathematicswithsagemath/sol/graphique_doctest.py # 8 doctests failed 20201117T19:31:00.7868993Z sage t randomseed=0 /__w/sage/sage/src/sage/repl/configuration.py # 1 doctest failed 20201117T19:31:00.7869942Z sage t randomseed=0 /__w/sage/sage/src/sage/repl/image.py # 4 doctests failed 20201117T19:31:00.7870875Z sage t randomseed=0 /__w/sage/sage/src/sage/repl/interpreter.py # 7 doctests failed 20201117T19:31:00.7871818Z sage t randomseed=0 /__w/sage/sage/src/sage/repl/preparse.py # 1 doctest failed 20201117T19:31:00.7872750Z sage t randomseed=0 /__w/sage/sage/src/sage/repl/ipython_tests.py # 1 doctest failed 20201117T19:31:00.7873726Z sage t randomseed=0 /__w/sage/sage/src/sage/repl/rich_output/pretty_print.py # 3 doctests failed 20201117T19:31:00.7874739Z sage t randomseed=0 /__w/sage/sage/src/sage/repl/ipython_kernel/install.py # 1 doctest failed 20201117T19:31:00.7875703Z sage t randomseed=0 /__w/sage/sage/src/sage/doctest/forker.py # 1 doctest failed 20201117T19:31:00.7876633Z sage t randomseed=0 /__w/sage/sage/src/sage/doctest/control.py # 2 doctests failed 20201117T19:31:00.7883133Z sage t randomseed=0 /__w/sage/sage/src/sage/doctest/test.py # 3 doctests failed 20201117T19:31:00.7884402Z sage t randomseed=0 /__w/sage/sage/src/sage/libs/singular/function.pyx # 1 doctest failed 20201117T19:31:00.7885406Z sage t randomseed=0 /__w/sage/sage/src/sage/libs/pari/convert_sage.pyx # 10 doctests failed 20201117T19:31:00.7886362Z sage t randomseed=0 /__w/sage/sage/src/sage/libs/pari/__init__.py # 12 doctests failed 20201117T19:31:00.7887265Z sage t randomseed=0 /__w/sage/sage/src/sage/libs/pari/tests.py # 9 doctests failed 20201117T19:31:00.7888179Z sage t randomseed=0 /__w/sage/sage/src/sage/libs/pynac/pynac.pyx # 1 doctest failed 20201117T19:31:00.7889240Z sage t randomseed=0 /__w/sage/sage/src/sage/dynamics/arithmetic_dynamics/generic_ds.py # 10 doctests failed 20201117T19:31:00.7890445Z sage t randomseed=0 /__w/sage/sage/src/sage/dynamics/arithmetic_dynamics/projective_ds.py # 25 doctests failed 20201117T19:31:00.7891469Z sage t randomseed=0 /__w/sage/sage/src/sage/arith/misc.py # 20 doctests failed 20201117T19:31:00.7892598Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/hyperelliptic_curves/hyperelliptic_generic.py # 3 doctests failed 20201117T19:31:00.7893789Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/product_projective/space.py # 1 doctest failed 20201117T19:31:00.7895096Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/product_projective/rational_point.py # 1 doctest failed 20201117T19:31:00.7896170Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/affine/affine_space.py # 1 doctest failed 20201117T19:31:00.7897209Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/generic/algebraic_scheme.py # 1 doctest failed 20201117T19:31:00.7898340Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/riemann_surfaces/riemann_surface.py # 31 doctests failed 20201117T19:31:00.7899522Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/projective/projective_rational_point.py # 4 doctests failed 20201117T19:31:00.7900680Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/projective/projective_space.py # 1 doctest failed 20201117T19:31:00.7901771Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/toric/toric_subscheme.py # 11 doctests failed 20201117T19:31:00.7902793Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/toric/divisor.py # 2 doctests failed 20201117T19:31:00.7903768Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/toric/points.py # 1 doctest failed 20201117T19:31:00.7904739Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/toric/variety.py # 4 doctests failed 20201117T19:31:00.7905707Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/toric/ideal.py # 43 doctests failed 20201117T19:31:00.7906825Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/elliptic_curves/heegner.py # 33 doctests failed 20201117T19:31:00.7907884Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/elliptic_curves/kraus.py # 4 doctests failed 20201117T19:31:00.7908978Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/elliptic_curves/constructor.py # 4 doctests failed 20201117T19:31:00.7910128Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/elliptic_curves/gal_reps_number_field.py # 2 doctests failed 20201117T19:31:00.7911278Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/elliptic_curves/ell_rational_field.py # 14 doctests failed 20201117T19:31:00.7912397Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/elliptic_curves/isogeny_class.py # 2 doctests failed 20201117T19:31:00.7913509Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/elliptic_curves/ell_number_field.py # 26 doctests failed 20201117T19:31:00.7914601Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/elliptic_curves/gal_reps.py # 2 doctests failed 20201117T19:31:00.7915708Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/elliptic_curves/isogeny_small_degree.py # 1 doctest failed 20201117T19:31:00.7916803Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/elliptic_curves/ell_egros.py # 4 doctests failed 20201117T19:31:00.7918998Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/elliptic_curves/ell_curve_isogeny.py # 1 doctest failed 20201117T19:31:00.7920093Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/elliptic_curves/ell_field.py # 2 doctests failed 20201117T19:31:00.7921146Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/plane_conics/con_field.py # 7 doctests failed 20201117T19:31:00.7922203Z sage t randomseed=0 /__w/sage/sage/src/sage/schemes/plane_conics/con_number_field.py # 3 doctests failed 20201117T19:31:00.7923188Z sage t randomseed=0 /__w/sage/sage/src/sage/features/__init__.py # 1 doctest failed 20201117T19:31:00.7924137Z sage t randomseed=0 /__w/sage/sage/src/sage/calculus/desolvers.py # 8 doctests failed 20201117T19:31:00.7925128Z sage t randomseed=0 /__w/sage/sage/src/sage/calculus/calculus.py # 25 doctests failed 20201117T19:31:00.7926071Z sage t randomseed=0 /__w/sage/sage/src/sage/crypto/lattice.py # 1 doctest failed 20201117T19:31:00.7927011Z sage t randomseed=0 /__w/sage/sage/src/sage/combinat/tutorial.py # 1 doctest failed 20201117T19:31:00.7927975Z sage t randomseed=0 /__w/sage/sage/src/sage/combinat/combinat.py # 5 doctests failed 20201117T19:31:00.7929092Z sage t randomseed=0 /__w/sage/sage/src/sage/combinat/degree_sequences.pyx # 1 doctest failed 20201117T19:31:00.7930148Z sage t randomseed=0 /__w/sage/sage/src/sage/categories/algebra_functor.py # 5 doctests failed 20201117T19:31:00.7931175Z sage t randomseed=0 /__w/sage/sage/src/sage/categories/number_fields.py # 1 doctest failed 20201117T19:31:00.7932168Z sage t randomseed=0 /__w/sage/sage/src/sage/categories/rings.py # 12 doctests failed 20201117T19:31:00.7933122Z sage t randomseed=0 /__w/sage/sage/src/sage/categories/primer.py # 1 doctest failed 20201117T19:31:00.7934130Z sage t randomseed=0 /__w/sage/sage/src/sage/categories/quotient_fields.py # 7 doctests failed 20201117T19:31:00.7935102Z sage t randomseed=0 /__w/sage/sage/src/sage/modular/cusps_nf.py # 1 doctest failed 20201117T19:31:00.7936063Z sage t randomseed=0 /__w/sage/sage/src/sage/modular/etaproducts.py # 10 doctests failed 20201117T19:31:00.7937057Z sage t randomseed=0 /__w/sage/sage/src/sage/modular/modsym/p1list_nf.py # 1 doctest failed 20201117T19:31:00.7938011Z sage t randomseed=0 /__w/sage/sage/src/sage/structure/formal_sum.py # 1 doctest failed 20201117T19:31:00.7938985Z sage t randomseed=0 /__w/sage/sage/src/sage/structure/element.pyx # 5 doctests failed 20201117T19:31:00.7940062Z sage t randomseed=0 /__w/sage/sage/src/sage/structure/dynamic_class.py # 1 doctest failed 20201117T19:31:00.7941022Z sage t randomseed=0 /__w/sage/sage/src/sage/misc/nested_class.pyx # 1 doctest failed 20201117T19:31:00.7941915Z sage t randomseed=0 /__w/sage/sage/src/sage/misc/trace.py # 3 doctests failed 20201117T19:31:00.7942828Z sage t randomseed=0 /__w/sage/sage/src/sage/misc/sage_ostools.pyx # 1 doctest failed 20201117T19:31:00.7943764Z sage t randomseed=0 /__w/sage/sage/src/sage/misc/functional.py # 1 doctest failed 20201117T19:31:00.7944692Z sage t randomseed=0 /__w/sage/sage/src/sage/misc/edit_module.py # 1 doctest failed 20201117T19:31:00.7945625Z sage t randomseed=0 /__w/sage/sage/src/sage/misc/temporary_file.py # 1 doctest failed 20201117T19:31:00.7946578Z sage t randomseed=0 /__w/sage/sage/src/sage/misc/lazy_attribute.pyx # 1 doctest failed 20201117T19:31:00.7947551Z sage t randomseed=0 /__w/sage/sage/src/sage/misc/sageinspect.py # 13 doctests failed 20201117T19:31:00.7948497Z sage t randomseed=0 /__w/sage/sage/src/sage/misc/sagedoc.py # 3 doctests failed 20201117T19:31:00.7949390Z sage t randomseed=0 /__w/sage/sage/src/sage/misc/compat.py # 1 doctest failed 20201117T19:31:00.7950304Z sage t randomseed=0 /__w/sage/sage/src/sage/misc/sphinxify.py # 4 doctests failed 20201117T19:31:00.7951326Z sage t randomseed=0 /__w/sage/sage/src/sage/functions/hypergeometric.py # 7 doctests failed 20201117T19:31:00.7952357Z sage t randomseed=0 /__w/sage/sage/src/sage/functions/bessel.py # 7 doctests failed 20201117T19:31:00.7953305Z sage t randomseed=0 /__w/sage/sage/src/sage/functions/gamma.py # 2 doctests failed 20201117T19:31:00.7954306Z sage t randomseed=0 /__w/sage/sage/src/sage/functions/special.py # 1 doctest failed 20201117T19:31:00.7955238Z sage t randomseed=0 /__w/sage/sage/src/sage/functions/trig.py # 1 doctest failed 20201117T19:31:00.7956168Z sage t randomseed=0 /__w/sage/sage/src/sage/functions/error.py # 7 doctests failed 20201117T19:31:00.7957101Z sage t randomseed=0 /__w/sage/sage/src/sage/functions/log.py # 6 doctests failed 20201117T19:31:00.7958207Z sage t randomseed=0 /__w/sage/sage/src/sage/functions/other.py # 15 doctests failed 20201117T19:31:00.7959216Z sage t randomseed=0 /__w/sage/sage/src/sage/functions/orthogonal_polys.py # 11 doctests failed 20201117T19:31:00.7960272Z sage t randomseed=0 /__w/sage/sage/src/sage/geometry/polyhedron/parent.py # 2 doctests failed 20201117T19:31:00.7961352Z sage t randomseed=0 /__w/sage/sage/src/sage/geometry/polyhedron/palp_database.py # 1 doctest failed 20201117T19:31:00.7962568Z sage t randomseed=0 /__w/sage/sage/src/sage/algebras/quaternion_algebra_element.py # 4 doctests failed 20201117T19:31:00.7963697Z sage t randomseed=0 /__w/sage/sage/src/sage/algebras/quatalg/quaternion_algebra.py # 6 doctests failed 20201117T19:31:00.7964873Z sage t randomseed=0 /__w/sage/sage/src/sage/algebras/quatalg/quaternion_algebra_element.pyx # 38 doctests failed 20201117T19:31:00.7965949Z sage t randomseed=0 /__w/sage/sage/src/sage/modules/free_module.py # 5 doctests failed 20201117T19:31:00.7966956Z sage t randomseed=0 /__w/sage/sage/src/sage/modules/free_module_morphism.py # 3 doctests failed 20201117T19:31:00.7967988Z sage t randomseed=0 /__w/sage/sage/src/sage/modules/free_module_integer.py # 57 doctests failed 20201117T19:31:00.7969027Z sage t randomseed=0 /__w/sage/sage/src/sage/modules/vector_space_morphism.py # 3 doctests failed 20201117T19:31:00.7970041Z sage t randomseed=0 /__w/sage/sage/src/sage/modules/fg_pid/fgp_module.py # 3 doctests failed 20201117T19:31:00.7971051Z sage t randomseed=0 /__w/sage/sage/src/sage/symbolic/expression.pyx # 17 doctests failed 20201117T19:31:00.7972071Z sage t randomseed=0 /__w/sage/sage/src/sage/symbolic/assumptions.py # 3 doctests failed 20201117T19:31:00.7973064Z sage t randomseed=0 /__w/sage/sage/src/sage/symbolic/constants_c.pyx # 1 doctest failed 20201117T19:31:00.7974140Z sage t randomseed=0 /__w/sage/sage/src/sage/symbolic/maxima_wrapper.py # 2 doctests failed 20201117T19:31:00.7975123Z sage t randomseed=0 /__w/sage/sage/src/sage/symbolic/relation.py # 4 doctests failed 20201117T19:31:00.7976102Z sage t randomseed=0 /__w/sage/sage/src/sage/symbolic/constants.py # 17 doctests failed 20201117T19:31:00.7977173Z sage t randomseed=0 /__w/sage/sage/src/sage/symbolic/expression_conversions.py # 10 doctests failed 20201117T19:31:00.7978261Z sage t randomseed=0 /__w/sage/sage/src/sage/graphs/comparability.pyx # 1 doctest failed 20201117T19:31:00.7979313Z sage t randomseed=0 /__w/sage/sage/src/sage/graphs/digraph_generators.py # 8 doctests failed 20201117T19:31:00.7980236Z sage t randomseed=0 /__w/sage/sage/src/sage/graphs/graph.py # Timed out 20201117T19:31:00.7981184Z sage t randomseed=0 /__w/sage/sage/src/sage/graphs/connectivity.pyx # 2 doctests failed 20201117T19:31:00.7982192Z sage t randomseed=0 /__w/sage/sage/src/sage/graphs/graph_generators.py # Timed out 20201117T19:31:00.7983217Z sage t randomseed=0 /__w/sage/sage/src/sage/graphs/hypergraph_generators.py # 9 doctests failed 20201117T19:31:00.7984259Z sage t randomseed=0 /__w/sage/sage/src/sage/graphs/graph_coloring.pyx # 2 doctests failed 20201117T19:31:00.7985206Z sage t randomseed=0 /__w/sage/sage/src/sage/graphs/genus.pyx # 1 doctest failed 20201117T19:31:00.7986147Z sage t randomseed=0 /__w/sage/sage/src/sage/graphs/generic_graph_pyx.pyx # 2 doctests failed 20201117T19:31:00.7987202Z sage t randomseed=0 /__w/sage/sage/src/sage/sets/set_from_iterator.py # Timed out (and interrupt failed) 20201117T19:31:00.7988200Z sage t randomseed=0 /__w/sage/sage/src/sage/matrix/matrix1.pyx # 3 doctests failed 20201117T19:31:00.7989131Z sage t randomseed=0 /__w/sage/sage/src/sage/matrix/matrix2.pyx # 3 doctests failed 20201117T19:31:00.7990123Z sage t randomseed=0 /__w/sage/sage/src/sage/matrix/matrix_rational_dense.pyx # 1 doctest failed 20201117T19:31:00.7991189Z sage t randomseed=0 /__w/sage/sage/src/sage/matrix/matrix_integer_dense.pyx # 14 doctests failed 20201117T19:31:00.7992217Z sage t randomseed=0 /__w/sage/sage/src/sage/matrix/matrix_symbolic_dense.pyx # 1 doctest failed
https://github.com/tobiasdiez/sage/runs/1413136300?check_suite_focus=true
comment:184 Changed 4 months ago by
 Commit changed from 77e1af16bcfce183bfe072cfb7324e82ee95a478 to d1341b0ab3b8bbc47a25beea2a8392632c981bfb
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
9a5a145  version bump

d4c4453  remove old workaround from #22626

ebe9780  this file should exist  leave just comments there

a507fb9  trac 29314: fix version numbers in patches for gap_packages

9836f79  move GAP's PackageManager to gap from gap_packages

435d82f  Merge branch 'u/dimpase/packages/gap411' of git://trac.sagemath.org/sage into public/build/inplace

2fc8bf5  Hangs with maxima, so remove again

f6c1d87  Try to install maxima again

13e96ad  Fix workflow syntax

d1341b0  Can we now activate cysignals again?

comment:185 Changed 4 months ago by
 Dependencies changed from #30779, #30672, #30673, #30709, #30706, #30901, #30748, #29850, #29314 to #30779, #30672, #30673, #30709, #30706, #30901, #30748, #29850, #29314, #30937, #30936, #30935, #30934
comment:186 Changed 3 months ago by
 Dependencies changed from #30779, #30672, #30673, #30709, #30706, #30901, #30748, #29850, #29314, #30937, #30936, #30935, #30934 to #30779, #30672, #30673, #30709, #30706, #30901, #30748, #29850, #29314, #30937, #30936, #30935, #30934, #30859, #30923, #30910
comment:187 Changed 3 months ago by
 Description modified (diff)
comment:188 Changed 3 months ago by
 Dependencies changed from #30779, #30672, #30673, #30709, #30706, #30901, #30748, #29850, #29314, #30937, #30936, #30935, #30934, #30859, #30923, #30910 to #30779, #30672, #30673, #30709, #30706, #30901, #30748, #29850, #29314, #30937, #30936, #30935, #30934, #30859, #30923, #30910, #30944
comment:189 Changed 3 months ago by
 Dependencies changed from #30779, #30672, #30673, #30709, #30706, #30901, #30748, #29850, #29314, #30937, #30936, #30935, #30934, #30859, #30923, #30910, #30944 to #30779, #30672, #30673, #30709, #30706, #30901, #30748, #29850, #29314, #30937, #30936, #30935, #30731 , #30859, #30923, #30910, #30944
comment:190 Changed 3 months ago by
 Keywords sd111 added
Hoping we can make progress on this ticket this week  https://wiki.sagemath.org/days111
comment:191 Changed 3 months ago by
 Status changed from needs_review to needs_work
comment:192 Changed 3 months ago by
 Commit changed from d1341b0ab3b8bbc47a25beea2a8392632c981bfb to 17e06daef346efc2f83f7b5304fc598c181079de
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
4393801  Build all of sage deps

2027ebb  Run make in parallel

405e8d2  Install dependencies on mac with tox as well

894b9d5  Fix indent

4cefc89  Always run manually to see what's going on

ee6aaeb  Don't require spkgconfigure.m4 file

8cc37e7  Cleanup mac workflow

c571a07  Fix maxima install

8ee33f2  Merge branch 'develop' of git://github.com/sagemath/sage into public/build/inplace

17e06da  Upgrade packages

comment:193 Changed 3 months ago by
From my side, this is actually ready for review. I've been using it now constantly the last few weeks and it is working. Or do you have something that I should be improving/working on?
comment:194 Changed 3 months ago by
 Description modified (diff)
comment:195 Changed 3 months ago by
 Commit changed from 17e06daef346efc2f83f7b5304fc598c181079de to 5c719fe14a0fa6a0aa966031786cf6f3c437b239
Branch pushed to git repo; I updated commit sha1. New commits:
3fcaf5f  Merge branch 'develop' of git://github.com/sagemath/sage into public/build/multiarchsimple

090e6f1  Simplify code

fa4556a  Remove _get_sage_local

7cc2005  Merge branch 'public/build/multiarchsimple' of git://trac.sagemath.org/sage into public/build/inplace

a590924  Merge branch 'develop' of git://github.com/sagemath/sage into public/build/inplace

afa0518  Make autogen more stable

5c719fe  Fix functional.py:56: DeprecationWarning: invalid escape sequence \d

comment:196 Changed 3 months ago by
 Status changed from needs_work to needs_review
Merged the latest develop branch.
comment:197 Changed 3 months ago by
 Description modified (diff)
comment:198 Changed 3 months ago by
 Commit changed from 5c719fe14a0fa6a0aa966031786cf6f3c437b239 to 7f0335de97d12b1bd5e6cc56879ab47f7423665a
comment:199 Changed 3 months ago by
 Dependencies changed from #30779, #30672, #30673, #30709, #30706, #30901, #30748, #29850, #29314, #30937, #30936, #30935, #30731 , #30859, #30923, #30910, #30944 to #30779, #30672, #30673, #30709, #30706, #30901, #31080, #29850, #29314, #30937, #30936, #30935, #30731 , #30859, #30923, #30910, #30944
comment:200 in reply to: ↑ 156 Changed 2 months ago by
Replying to mkoeppe:
It has been reported that the current
src/setup.py
(in Sage 9.2) is able to install a working sagelib on top of conda (see #28745).I'll create a test for this in #30845, and then we can test the present ticket to make sure it does not break anything.
Ensuring that the feature of installing sagelib on top of conda is not broken is a prerequisite for merging this ticket.
To move this ticket forward, could you help me with this GH Actions test? (This should be on top of #30944, which still needs review)
I think there should be one new workflow for all tests that do not build the Sage distribution (no build/pkgs
).
comment:201 followups: ↓ 202 ↓ 203 Changed 2 months ago by
Sure, I will have a look at this. Could you by extending #30845 to include in more detail how the manual workflow with conda looks like (I've no experience with conda) and why #30944 is needed.
Maybe it's a good idea to review how the GH workflows are structured and revise this if necessary. For example, focus more on the purpose vs how it done, e.g. "test that building packages in build/pgks" works, "test that sagelib installed on top of various setups works". For the latter, my hope with pipenv was actually that one can easily run it on every push on every branch as a gateway in addition to the patchbot (since the time it takes is very minimal, time of setup of < 1h, + time for doctests).
comment:202 in reply to: ↑ 201 Changed 2 months ago by
Replying to ghtobiasdiez:
Sure, I will have a look at this. Could you by extending #30845 to include in more detail how the manual workflow with conda looks like (I've no experience with conda) and
Done.
why #30944 is needed.
Well, it will be testing something that currently happens to work but is not automatically tested.
Maybe it's a good idea to review how the GH workflows are structured and revise this if necessary.
Sure, I'll be happy about more eyes on the workflows.
For example, focus more on the purpose vs how it done
All workflows test fromscratch builds of the Sage distribution. Two workflows separately test optional / experimental packages. That's really all.
comment:203 in reply to: ↑ 201 Changed 2 months ago by
Replying to ghtobiasdiez:
"test that sagelib installed on top of various setups works". For the latter, my hope with pipenv was actually that one can easily run it on every push on every branch as a gateway in addition to the patchbot (since the time it takes is very minimal, time of setup of < 1h, + time for doctests).
An incremental build setup based on Docker has already been developed, which drives the current GitLab workflow. I'd recommend to start from there (it needs attention) instead of developing another thing from scratch... Take a look at #29536
comment:204 Changed 2 months ago by
 Commit changed from 7f0335de97d12b1bd5e6cc56879ab47f7423665a to f96b923fd5c342c5401f4e8f993546b523fe2cc0
Branch pushed to git repo; I updated commit sha1. New commits:
f96b923  Use sage_conf

comment:205 Changed 2 months ago by
 Dependencies changed from #30779, #30672, #30673, #30709, #30706, #30901, #31080, #29850, #29314, #30937, #30936, #30935, #30731 , #30859, #30923, #30910, #30944 to #30779, #30672, #30673, #30709, #30706, #30901, #31080, #29850, #29314, #30937, #30935, #30731 , #30859, #30923, #30910, #30944
comment:206 Changed 2 months ago by
 Commit changed from f96b923fd5c342c5401f4e8f993546b523fe2cc0 to 8c97e7c9290c067af0e0bd974f60f4b3966a516c
Branch pushed to git repo; I updated commit sha1. New commits:
8c97e7c  Cleanup code

comment:207 Changed 2 months ago by
Since now most dependencies are merged in the develop branch, this ticket is now definitely ready for review.
comment:208 Changed 8 weeks ago by
 Commit changed from 8c97e7c9290c067af0e0bd974f60f4b3966a516c to f871fc77be3b6ec719dc3f64a033ca78e6deea67
comment:209 followups: ↓ 210 ↓ 212 Changed 8 weeks ago by
Matthias, I have a question: the current code works very well for systeminstalled packages. However, there are a few packages that are required but for which systeminstalled versions are not currently not supported (e.g. ecl). How could I use the sagebuilt versions of these packages for the built process of sagelib, i.e. for pip install e .
?
Take for example the import cdef extern from "ecl/ecl.h"
in libs/ecl.pxd
.
I guess one needs to pass the path of local/include/ecl
as argument to the C++ compiler. How is currently done?
comment:210 in reply to: ↑ 209 ; followups: ↓ 213 ↓ 228 Changed 8 weeks ago by
Replying to ghtobiasdiez:
Take for example the import
cdef extern from "ecl/ecl.h"
inlibs/ecl.pxd
. I guess one needs to pass the path oflocal/include/ecl
as argument to the C++ compiler. How is currently done?
We install all packages that are not accepted from the system into $SAGE_LOCAL, and scripts such as sageenv
and sagebuildenv
set up environment variables such as LIBRARY_PATH
and CPATH
so that compilers find stuff. Configuration for Cython is somewhat separate  you are already familiar with the cython_aliases
function.
For ECL, there's ticket #30770 to make this configuration more flexible.
comment:211 Changed 8 weeks ago by
Also, if you are able to use system packages for which we do not have system package detection yet (spkgconfigure.m4
), you can join the effort in #27330.
comment:212 in reply to: ↑ 209 ; followup: ↓ 214 Changed 8 weeks ago by
Replying to ghtobiasdiez:
How could I use the sagebuilt versions of these packages for the built process of sagelib, i.e. for
pip install e .
?
One part of the answer to this involves the sage_conf
package: Installing it into your virtual environment provides sageenvconfig
and the sage_conf
module. So you may want to add it to Pipfile
. (See also #30913).
comment:213 in reply to: ↑ 210 Changed 8 weeks ago by
comment:214 in reply to: ↑ 212 Changed 8 weeks ago by
Replying to mkoeppe:
One part of the answer to this involves the
sage_conf
package: Installing it into your virtual environment providessageenvconfig
and thesage_conf
module. So you may want to add it toPipfile
. (See also #30913).
That's already done:
+[devpackages.8c46424] +path = "./../build/pkgs/sage_conf/src" +editable = true
comment:215 Changed 8 weeks ago by
 Dependencies changed from #30779, #30672, #30673, #30709, #30706, #30901, #31080, #29850, #29314, #30937, #30935, #30731 , #30859, #30923, #30910, #30944 to #30779, #30672, #30673, #30709, #30706, #30901, #31080, #29850, #29314, #30937, #30935, #30731 , #30859, #30923, #30910, #30944, #30770
comment:216 Changed 8 weeks ago by
 Commit changed from f871fc77be3b6ec719dc3f64a033ca78e6deea67 to deeeb75c316c0e06c40e36ec65f472d4d3b90972
comment:217 Changed 8 weeks ago by
 Description modified (diff)
comment:218 Changed 8 weeks ago by
 Commit changed from deeeb75c316c0e06c40e36ec65f472d4d3b90972 to 2ea4935dbae81bf1f2c9f9dc4e29f2feeb1a733f
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
9988c5f  cicygwin*.yml: Adjust to new script packages _bootstrap, _prereq

ab19133  Merge branch 't/29124/scriptpackagesprereqtoolchainbootstrap' into t/30944/tox__improve_local_sudo_ubuntu_standard

c6ad06e  Merge branch 't/30944/tox__improve_local_sudo_ubuntu_standard' into t/31216/tox_ini__local___add_environment_variables_to_skip_system_package_installs__mechanism_for_a_local_interactive_shell

13f58fc  tox.ini (local): Run an interactive shell if posargs are bash

ace4ef6  tox.ini (local): Add option SKIP_SYSTEM_PKG_INSTALL=1

ddc4f0b  tox.ini (local): Handle environment variables SKIP_BOOTSTRAP, SKIP_CONFIGURE

bb6aec4  src/doc/en/developer/portability_testing.rst: Updates

6253bd8  src/doc/en/developer/portability_testing.rst: New section on SKIP_... environment variables

a52e899  Merge branch 'u/mkoeppe/tox_ini__local___add_environment_variables_to_skip_system_package_installs__mechanism_for_a_local_interactive_shell' of git://trac.sagemath.org/sage into public/build/inplace

2ea4935  Split workflow in install and build

comment:219 Changed 8 weeks ago by
 Dependencies changed from #30779, #30672, #30673, #30709, #30706, #30901, #31080, #29850, #29314, #30937, #30935, #30731 , #30859, #30923, #30910, #30944, #30770 to #30779, #30672, #30673, #30709, #30706, #30901, #31080, #29850, #29314, #30937, #30935, #30731 , #30859, #30923, #30910, #30944, #30770, #31216
comment:220 Changed 8 weeks ago by
 Commit changed from 2ea4935dbae81bf1f2c9f9dc4e29f2feeb1a733f to 6bcc5c34991dc4b81362b2427ff8d38b50f32c7e
Branch pushed to git repo; I updated commit sha1. New commits:
6bcc5c3  Add esac

comment:221 Changed 8 weeks ago by
 Commit changed from 6bcc5c34991dc4b81362b2427ff8d38b50f32c7e to d32b22b7091051522180ea58ea861b022e16e4d7
Branch pushed to git repo; I updated commit sha1. New commits:
d32b22b  Try with config

comment:222 Changed 8 weeks ago by
 Commit changed from d32b22b7091051522180ea58ea861b022e16e4d7 to 73c99f3b7f797a7ab61507392c40268bb50e0a00
Branch pushed to git repo; I updated commit sha1. New commits:
73c99f3  Ignore errors for now

comment:223 Changed 8 weeks ago by
 Commit changed from 73c99f3b7f797a7ab61507392c40268bb50e0a00 to 5cb58cf579b41780791b9bf0cb46563713efd1e5
Branch pushed to git repo; I updated commit sha1. New commits:
5cb58cf  Enable build as root

comment:224 Changed 8 weeks ago by
 Commit changed from 5cb58cf579b41780791b9bf0cb46563713efd1e5 to 02542220efcfc3989a682ed082092c7f6e9a1c30
Branch pushed to git repo; I updated commit sha1. New commits:
0254222  Cleanup github workflow

comment:225 Changed 8 weeks ago by
 Commit changed from 02542220efcfc3989a682ed082092c7f6e9a1c30 to 4dcd083615e7eb7218b595c61fd9b9a4754222cb
Branch pushed to git repo; I updated commit sha1. New commits:
4dcd083  Use all for the tests

comment:226 Changed 8 weeks ago by
 Commit changed from 4dcd083615e7eb7218b595c61fd9b9a4754222cb to d30fda0a1bd9682d624feed871c3c94fa53921df
Branch pushed to git repo; I updated commit sha1. New commits:
d30fda0  Also install maxima

comment:227 Changed 8 weeks ago by
 Commit changed from d30fda0a1bd9682d624feed871c3c94fa53921df to b17e6faef915267735b95135e326ffc74fd558b6
Branch pushed to git repo; I updated commit sha1. New commits:
b17e6fa  Use configure.ac as make target

comment:228 in reply to: ↑ 210 ; followup: ↓ 231 Changed 8 weeks ago by
Replying to mkoeppe:
For ECL, there's ticket #30770 to make this configuration more flexible.
This seems to work now. Thanks for you help!
Two followup questions:
 Is there a make target similar to sagelibbuilddeps that doesn't install pythonic packages such as cython.
 Why doesn't sagelibbuilddeps install maxima?
comment:229 Changed 8 weeks ago by
 Commit changed from b17e6faef915267735b95135e326ffc74fd558b6 to 5d64c5a44b44e36c260eca75c60e179093f0b67f
Branch pushed to git repo; I updated commit sha1. New commits:
5d64c5a  Install/fix other packages

comment:230 Changed 8 weeks ago by
 Commit changed from 5d64c5a44b44e36c260eca75c60e179093f0b67f to dbf2df2f53f7b1b7e7760f0fd578f86a44c9df4b
Branch pushed to git repo; I updated commit sha1. New commits:
dbf2df2  Install more dependencies manually

comment:231 in reply to: ↑ 228 Changed 8 weeks ago by
Replying to ghtobiasdiez:
 Is there a make target similar to sagelibbuilddeps that doesn't install pythonic packages such as cython.
Not yet. #30896 will go into this direction
 Why doesn't sagelibbuilddeps install maxima?
maxima is only needed at runtime of Sage, not while installing sagelib.
comment:232 Changed 8 weeks ago by
 Commit changed from dbf2df2f53f7b1b7e7760f0fd578f86a44c9df4b to b66df86359a2ee8be8ab481d0aea20b8960d0ef9
Branch pushed to git repo; I updated commit sha1. New commits:
b66df86  Run tests in parallel

comment:233 Changed 7 weeks ago by
 Commit changed from b66df86359a2ee8be8ab481d0aea20b8960d0ef9 to 1fdfe680752f2878c76234733dc34780e3107715
comment:234 Changed 7 weeks ago by
 Commit changed from 1fdfe680752f2878c76234733dc34780e3107715 to f0bd45cf7ff1ac240bfd10af8844df187c1f97f4
Branch pushed to git repo; I updated commit sha1. New commits:
f0bd45c  Install singulardata to find gftables?

comment:235 Changed 7 weeks ago by
 Commit changed from f0bd45cf7ff1ac240bfd10af8844df187c1f97f4 to 8a401efbb30b1049b05af3b4e756087d64968505
comment:236 Changed 7 weeks ago by
 Commit changed from 8a401efbb30b1049b05af3b4e756087d64968505 to 22e8fe7c5ab241996824216396b9025997c939be
comment:237 Changed 7 weeks ago by
 Dependencies changed from #30779, #30672, #30673, #30709, #30706, #30901, #31080, #29850, #29314, #30937, #30935, #30731 , #30859, #30923, #30910, #30944, #30770, #31216 to #30779, #30672, #30673, #30709, #30706, #30901, #31080, #29850, #29314, #30937, #30935, #30731 , #30859, #30923, #30910, #30944, #30770, #31216, #31029
Thanks for the help! I've now improved the github workflow and the doctest pass now (except a handful). So I would appreciate if this ticket could be review!
comment:238 Changed 7 weeks ago by
 Commit changed from 22e8fe7c5ab241996824216396b9025997c939be to 3d071819a98f2d8f8e50935223d38042e04923e1
Branch pushed to git repo; I updated commit sha1. New commits:
3d07181  Run tests in serial again

comment:239 Changed 7 weeks ago by
 Commit changed from 3d071819a98f2d8f8e50935223d38042e04923e1 to cd38e05d1b88d4879bd65aa32eb1614356569802
Branch pushed to git repo; I updated commit sha1. New commits:
cd38e05  Don't set SAGE_LOCAL in workflow

comment:240 Changed 7 weeks ago by
 Commit changed from cd38e05d1b88d4879bd65aa32eb1614356569802 to e42e1249042e31381f40889cef835e5947add799
Branch pushed to git repo; I updated commit sha1. New commits:
e42e124  Update deps

comment:241 Changed 7 weeks ago by
 Commit changed from e42e1249042e31381f40889cef835e5947add799 to d1079de570eb56f0031153286ed3ff79e273986c
Branch pushed to git repo; I updated commit sha1. New commits:
2d84b5a  tox.ini: Add missing esac

3fd16f4  Merge branch 'u/mkoeppe/tox_ini__local___add_environment_variables_to_skip_system_package_installs__mechanism_for_a_local_interactive_shell' of git://trac.sagemath.org/sage into public/build/inplace

d1079de  Merge branch 'develop' of git://github.com/sagemath/sage into public/build/inplace

comment:242 followup: ↓ 243 Changed 6 weeks ago by
Instead of introducing a new environment variable SAGE_NUM_BUILD_JOBS
, probably best to use SAGE_NUM_THREADS
, which is used already in the same source tree.
comment:243 in reply to: ↑ 242 Changed 6 weeks ago by
Replying to mkoeppe:
Instead of introducing a new environment variable
SAGE_NUM_BUILD_JOBS
, probably best to useSAGE_NUM_THREADS
, which is used already in the same source tree.
Ok, done!
comment:244 Changed 6 weeks ago by
 Commit changed from d1079de570eb56f0031153286ed3ff79e273986c to d798925a1bb48a96087c3db03950e90059c9c37b
comment:245 Changed 6 weeks ago by
 Commit changed from d798925a1bb48a96087c3db03950e90059c9c37b to cd8a8d19ea80b99357c465b4e1ff3f285a8b26b5
Branch pushed to git repo; I updated commit sha1. New commits:
cd8a8d1  Move pgkconfig to normal package

comment:246 Changed 6 weeks ago by
 Commit changed from cd8a8d19ea80b99357c465b4e1ff3f285a8b26b5 to 63c7997e078bb7b3a1ab39ef1a2c29d210376f71
Branch pushed to git repo; I updated commit sha1. New commits:
63c7997  Make cython and sage_conf also normal packages

comment:247 Changed 6 weeks ago by
 Commit changed from 63c7997e078bb7b3a1ab39ef1a2c29d210376f71 to a18b3c59eff2940159fdb930eea4af04b444e828
comment:248 Changed 6 weeks ago by
 Commit changed from a18b3c59eff2940159fdb930eea4af04b444e828 to f88cfc6a6382cf5cf35a602f83b7bedc06fc67eb
Branch pushed to git repo; I updated commit sha1. New commits:
f88cfc6  Set SAGE_LOCAL in workflow for gap

comment:249 Changed 6 weeks ago by
 Commit changed from f88cfc6a6382cf5cf35a602f83b7bedc06fc67eb to d42fbfc8fe52e06a7c6a54cf6c0a7ea94c6e3a29
Branch pushed to git repo; I updated commit sha1. New commits:
d42fbfc  See if error comes from memory problems

comment:250 Changed 6 weeks ago by
 Commit changed from d42fbfc8fe52e06a7c6a54cf6c0a7ea94c6e3a29 to 8580789c62f04a49ea893c53bf28d20a80a1aa73
Branch pushed to git repo; I updated commit sha1. New commits:
8580789  Fix indent

comment:251 Changed 6 weeks ago by
 Commit changed from 8580789c62f04a49ea893c53bf28d20a80a1aa73 to 1145360fdc51ee246c27715ce298c6005fd4b281
Branch pushed to git repo; I updated commit sha1. New commits:
1145360  Fix paths to memory usage

comment:252 Changed 6 weeks ago by
 Commit changed from 1145360fdc51ee246c27715ce298c6005fd4b281 to 4bdfef1b21da9f376cfd3c096cb9932fe5334e4e
Branch pushed to git repo; I updated commit sha1. New commits:
4bdfef1  Try with sudo

comment:253 Changed 6 weeks ago by
 Description modified (diff)
comment:254 Changed 5 weeks ago by
I think it should be done on top of #30913
comment:255 Changed 5 weeks ago by
Since the pipfile in this ticket is more uptodate and is welltested (manually and through the added github action), I would actually argue that #30913 should be on top of this ticket.
Moreover, #30770 is the only open dependency of this ticket, so I guess its also more realistic to get this in first.
comment:256 followup: ↓ 260 Changed 5 weeks ago by
This approach would also work. But I am skeptical about checking in the Pipfile.lock
. Stuff for nightmare merge conflicts for everyone. I think it would be better if this was an ignored file.
comment:257 Changed 5 weeks ago by
 Cc isuruf added
comment:258 Changed 5 weeks ago by
When the next beta comes out (hopefully soon), let's rebase it on top of it (and #30770). The commit history is a bit too wild
comment:259 Changed 5 weeks ago by
 Priority changed from major to critical
Also, could you please:
 clean up the ticket description
 move documentation items from there to the manuals (installation and/or developer's manual)
 move the remaining issues to new or existing tickets
comment:260 in reply to: ↑ 256 ; followup: ↓ 264 Changed 4 weeks ago by
Replying to mkoeppe:
But I am skeptical about checking in the
Pipfile.lock
. Stuff for nightmare merge conflicts for everyone. I think it would be better if this was an ignored file.
Without the lock file you wouldn't get a reproducible virtual environment, since pipenv would always use the latest versions (that are compatible with each other). Moreover, the lock file is normally not changed, except if you call pipenv update
.
The workflow to update Python packages would then look as follows:
 Run
pipenv update
 Test
 If everything is fine, create ticket updating pipfile.lock and create a ticket for each updated dependency that is also listed in sage's build/pgks folder.
Will have a look at the other todo's you mention.
comment:261 Changed 4 weeks ago by
 Description modified (diff)
comment:262 Changed 4 weeks ago by
Cleaned up the ticket description and added a bit of documentation. Since this is still somewhat experimental, I've only put it in the coding basics of the development doc. I guess with the recent changes and support of virtual environments, it might soon be time to update the recommended way of setting up a development platform.
comment:263 Changed 4 weeks ago by
 Commit changed from 4bdfef1b21da9f376cfd3c096cb9932fe5334e4e to 543b6d4e55879dee44f0e3d482b1ca1ebeda1d04
Branch pushed to git repo; I updated commit sha1. New commits:
543b6d4  Add basic documentation

comment:264 in reply to: ↑ 260 Changed 4 weeks ago by
Replying to ghtobiasdiez:
The workflow to update Python packages would then look as follows [...]
I would say it's not a good idea to try to change the developer workflow regarding package upgrades as part of this ticket.
comment:265 Changed 4 weeks ago by
I agree, that was more meant to illustrate possible applications of the new pipenv environment (and the lockfile for that purpose).
comment:266 Changed 4 weeks ago by
OK. I have created #31346 as a place for this.
comment:267 Changed 4 weeks ago by
Regarding the documentation: "After following the normal procedure of setting up your local development environment"  I think we should be explicit about requiring the full Sage distribution to be built first. We cannot support experimental workflows as you (still) describe them in the ticket description.
comment:268 Changed 4 weeks ago by
 Commit changed from 543b6d4e55879dee44f0e3d482b1ca1ebeda1d04 to 510a3bb5344b544e70b12cd53dd7c48e19701963
Branch pushed to git repo; I updated commit sha1. New commits:
510a3bb  Clarify point in the docs

comment:269 Changed 4 weeks ago by
Sure!
comment:270 Changed 4 weeks ago by
 Commit changed from 510a3bb5344b544e70b12cd53dd7c48e19701963 to c21cc76386b75dbf07f01ae822c120ef9f29c9e9
Branch pushed to git repo; I updated commit sha1. New commits:
c21cc76  Update deps

comment:271 Changed 4 weeks ago by
 Description modified (diff)
I've moved the part of the ticket description describing the experimental workflow on bare ubuntu to #31347.
comment:272 Changed 4 weeks ago by
9.3.beta7 is out, time for rebasing
comment:273 Changed 4 weeks ago by
 Commit changed from c21cc76386b75dbf07f01ae822c120ef9f29c9e9 to cf068dfaf42a32f6bbd1ebd55376006e56fb6d8a
Branch pushed to git repo; I updated commit sha1. New commits:
cf068df  Merge branch 'develop' of git://github.com/sagemath/sage into public/build/inplace

comment:274 Changed 4 weeks ago by
Nice, thanks for letting me know. I've merged the latest develop branch. The history of this branch still needs to be cleanup up, but let's do this after positive review.
comment:275 Changed 4 weeks ago by
Could you:
 remove the change to
tox.ini
please?  Rename
sage_build_ext
and move it to a new module ofsage_setup.command
(and document what is different to the existingsage_build_ext
)
comment:276 followups: ↓ 285 ↓ 287 Changed 4 weeks ago by
In the new CI workflows:
 For mac: I do not understand the line:
brew install pari
 For mac: Instead of setting
PKG_CONFIG_PATH
manually, perhaps use.homebrewbuildenv
because this is whatconfigure
suggests users to use  For linux: Why
make maxima conway_polynomials jmol conway_polynomials elliptic_curves combinatorial_designs polytopes_db graphs threejs
(it is not done in the mac workflow). Should we create amake
target that corresponds to what is needed?  For linux: Could you elaborate on "# Set SAGE_LOCAL (required for gap)"
Could you review and add comments to the file please?
comment:277 Changed 4 weeks ago by
Also, as #30589 has been merged, probably best to switch from python 3.8 to 3.9 here too
comment:278 Changed 4 weeks ago by
Also #30912 will bring merge conflicts that are best to resolve now
comment:279 Changed 4 weeks ago by
Another conflict from the current version of #30770.
comment:280 followup: ↓ 361 Changed 4 weeks ago by
Also create_extension
duplicates some of the code in sage_build_cython.create_extension
. In particular, it should not use numpy.get_include()
directly but rather use sage.env.sage_include_directories
(note #31333 for a related change).
comment:281 Changed 4 weeks ago by
I have tested this ticket (on top of #31335, which fixes macOSspecific problems, and resolving the mentioned merge conflicts); following the instructions from the added documentation, pipenv install
leads to ERROR: Couldn't install package: cypari2
.
The first error leading to this is:
[1/7] Cythonizing cypari2/closure.pyx Error compiling Cython file:  ... # http://www.gnu.org/licenses/ #***************************************************************************** from __future__ import absolute_import, division, print_function from cysignals.signals cimport sig_on, sig_off, sig_block, sig_unblock, sig_error ^  cypari2/closure.pyx:36:0: 'cysignals/signals.pxd' not found
comment:282 Changed 4 weeks ago by
The specific issue in cypari2
is probably https://github.com/sagemath/cypari2/issues/93 (tracked in #30922).
But overall I think it is unrealistic for these instructions to work.
Let's use more restrictive instructions, in which pipenv
is used from within sage sh
. Then one can at least assume that all libraries are available.
comment:283 Changed 4 weeks ago by
 Status changed from needs_review to needs_work
comment:284 Changed 4 weeks ago by
(Something like ../sage sh c 'PIP_FIND_LINKS=file://$(cd .. && pwd)/local/var/lib/sage/wheels pipenv install v v'
similar to what I am testing in #30913.)
comment:285 in reply to: ↑ 276 ; followup: ↓ 288 Changed 4 weeks ago by
Thanks for the review! Will address most of your points later.
Replying to mkoeppe:
 For linux: Could you elaborate on "# Set SAGE_LOCAL (required for gap)"
Without this line one gets the following error:
Traceback (most recent call last): File "/__w/sagetracmirror/sagetracmirror/src/sage/interfaces/expect.py", line 1468, in __init__ self._name = parent._create(value, name=name) File "/__w/sagetracmirror/sagetracmirror/src/sage/interfaces/interface.py", line 500, in _create self.set(name, value) File "/__w/sagetracmirror/sagetracmirror/src/sage/interfaces/gap.py", line 1421, in set self._eval_line(cmd, allow_use_file=True) File "/__w/sagetracmirror/sagetracmirror/src/sage/interfaces/gap.py", line 790, in _eval_line self._start() File "/__w/sagetracmirror/sagetracmirror/src/sage/interfaces/gap.py", line 1254, in _start gap_reset_workspace(verbose=False) File "/__w/sagetracmirror/sagetracmirror/src/sage/interfaces/gap.py", line 1587, in gap_reset_workspace g.eval('SetUserPreference("HistoryMaxLines", 30)') File "/__w/sagetracmirror/sagetracmirror/src/sage/interfaces/gap.py", line 587, in eval result = Expect.eval(self, input_line, **kwds) File "/__w/sagetracmirror/sagetracmirror/src/sage/interfaces/expect.py", line 1381, in eval return '\n'.join([self._eval_line(L, allow_use_file=allow_use_file, **kwds) File "/__w/sagetracmirror/sagetracmirror/src/sage/interfaces/expect.py", line 1381, in <listcomp> return '\n'.join([self._eval_line(L, allow_use_file=allow_use_file, **kwds) File "/__w/sagetracmirror/sagetracmirror/src/sage/interfaces/gap.py", line 790, in _eval_line self._start() File "/__w/sagetracmirror/sagetracmirror/src/sage/interfaces/gap.py", line 1259, in _start Expect._start(self, "Failed to start GAP.") File "/__w/sagetracmirror/sagetracmirror/src/sage/interfaces/expect.py", line 520, in _start raise RuntimeError("unable to start %s: %s" % (self.name(), msg)) RuntimeError: unable to start gap: End Of File (EOF). Exception style platform. Gap finished running /__w/sagetracmirror/sagetracmirror/.tox/localsudostandard/local/bin/gap r b p T E o 401m s 401m m 64m /__w/sagetracmirror/sagetracmirror/src/sage/ext_data/gap/sage.g command: /__w/sagetracmirror/sagetracmirror/.tox/localsudostandard/local/bin/gap args: ['/__w/sagetracmirror/sagetracmirror/.tox/localsudostandard/local/bin/gap', 'r', 'b', 'p', 'T', 'E', 'o', '401m', 's', '401m', 'm', '64m', '/__w/sagetracmirror/sagetracmirror/src/sage/ext_data/gap/sage.g'] buffer (last 100 chars): b'' before (last 100 chars): b'Set the environment variable SAGE_LOCAL.\r\n' after: <class 'pexpect.exceptions.EOF'> match: None match_index: None exitstatus: 1 flag_eof: True pid: 3124 child_fd: 26 closed: False timeout: None delimiter: <class 'pexpect.exceptions.EOF'> logfile: None logfile_read: None logfile_send: None maxread: 4194304 ignorecase: False searchwindowsize: None delaybeforesend: None delayafterclose: 0.1 delayafterterminate: 0.1 searcher: searcher_re: 0: re.compile(b'gap> ')
I guess the current github workflows don't see this as everything is invoked in one tox command and not in multiple steps (and enviroment variables are only active in each step).
comment:286 followup: ↓ 311 Changed 4 weeks ago by
Concerning the cypari error, I have no clue what is needed for it to compile on mac. It's also strange that it works on the github action (for mac as well as linux) but not on your system. cysignals is actually declared in the pipenv file, so it should be installed. The simplest solution would be if cypari would publish wheels on pypi.
But overall I think it is unrealistic for these instructions to work.
It's not unrealistic, it is working very well for Linux.
comment:287 in reply to: ↑ 276 ; followup: ↓ 290 Changed 4 weeks ago by
Replying to mkoeppe:
 For linux: Why
make maxima conway_polynomials jmol conway_polynomials elliptic_curves combinatorial_designs polytopes_db graphs threejs
(it is not done in the mac workflow). Should we create amake
target that corresponds to what is needed?
That's the only way I found to install these packages into SAGE_SHARE
. Will add it also to the mac workflow. I cannot judge if it would make sense to add a special make target for it, especially since you are working on making these packages normal python packages.
comment:288 in reply to: ↑ 285 Changed 4 weeks ago by
Replying to ghtobiasdiez:
Replying to mkoeppe:
 For linux: Could you elaborate on "# Set SAGE_LOCAL (required for gap)"
Without this line one gets the following error:
... Expect._start(self, "Failed to start GAP.") File "/__w/sagetracmirror/sagetracmirror/src/sage/interfaces/expect.py", line 520, in _start ...I guess the current github workflows don't see this as everything is invoked in one tox command and not in multiple steps (and enviroment variables are only active in each step).
OK, I see.
There are two ways to fix this:
 as described in #30818 ("Set environment for subprocesses invoked by Sage"). Currently subprocesses (such as the gap process invoked by the pexpectbased gap interface) rely on
sageenv
being sourced  using
pipenv run sage t
instead ofpipenv run sageruntests
, which usessageenv
.
comment:289 followup: ↓ 310 Changed 4 weeks ago by
comment:290 in reply to: ↑ 287 ; followup: ↓ 309 Changed 4 weeks ago by
Replying to ghtobiasdiez:
Replying to mkoeppe:
 For linux: Why
make maxima conway_polynomials jmol conway_polynomials elliptic_curves combinatorial_designs polytopes_db graphs threejs
(it is not done in the mac workflow). Should we create amake
target that corresponds to what is needed?That's the only way I found to install these packages into
SAGE_SHARE
. Will add it also to the mac workflow. I cannot judge if it would make sense to add a special make target for it, especially since you are working on making these packages normal python packages.
I've opened #31362 for this  which will add make allsagelocal
.
comment:291 Changed 4 weeks ago by
Thanks!
Concerning the above cysignals problem, you can try to run pipenv run pip install cysignals
and then pipenv install
again. Similarly, pipenv run pip install gmpy2==2.1.0b5
might be necessary if you have problems with the installation of pplpy. Let me know if this helps and I will add it to the documentation.
comment:292 Changed 4 weeks ago by
 Commit changed from cf068dfaf42a32f6bbd1ebd55376006e56fb6d8a to 398e6dd59ac7b8e3c49dc7917295e4fe4a9bcaba
Branch pushed to git repo; I updated commit sha1. New commits:
1f1dddc  Revert change to tox.ini

0c3f517  Move sage_build_ext

d06c84d  Run workflow for Python 3.9 as well

7fee372  Doc: Also install dev packages

a506c1f  Use sage_build command for generating sources

398e6dd  Narrow down imports to reduce install requires deps

comment:293 Changed 4 weeks ago by
 Commit changed from 398e6dd59ac7b8e3c49dc7917295e4fe4a9bcaba to 12c0a2ed96e5608f8f6c54aef0d81e8d76cdcff5
Branch pushed to git repo; I updated commit sha1. New commits:
12c0a2e  Mac: remove pari install

comment:294 Changed 4 weeks ago by
 Commit changed from 12c0a2ed96e5608f8f6c54aef0d81e8d76cdcff5 to 0157a4eac42646f49b9448afd120d07c4848750b
Branch pushed to git repo; I updated commit sha1. New commits:
1041128  Rename ecl config variable

fd05a97  Add fallback

2ac4cc8  Merge branch 'develop' of git://github.com/sagemath/sage into public/build/eclconfig

6370c34  src/sage/env.py: Do not use capture_output, which requires python 3.7

31f1bbb  Also use universal_newlines instead of text, for python 3.6 compatibility

11ad40a  Merge branch 'public/build/eclconfig' of git://trac.sagemath.org/sage into public/build/eclconfig

1a8e2e4  Merge branch 'public/build/eclconfig' of git://trac.sagemath.org/sage into public/build/inplace

0157a4e  Reuse sage_include_directories

comment:295 Changed 4 weeks ago by
 Commit changed from 0157a4eac42646f49b9448afd120d07c4848750b to a3806c58709c3b5c828668f823b37619bf1d1551
comment:296 Changed 4 weeks ago by
 Commit changed from a3806c58709c3b5c828668f823b37619bf1d1551 to f5551673591c836942a4b7d50f971ad7c910763e
Branch pushed to git repo; I updated commit sha1. New commits:
f555167  Fix workflow syntax

comment:297 Changed 4 weeks ago by
 Commit changed from f5551673591c836942a4b7d50f971ad7c910763e to bb90f76abc82987e7ff1cbfd2653dfa908fefc7b
Branch pushed to git repo; I updated commit sha1. New commits:
bb90f76  Well...maybe now

comment:298 Changed 4 weeks ago by
 Commit changed from bb90f76abc82987e7ff1cbfd2653dfa908fefc7b to d1c2a0a00f8c675ad218b502516dca12d161933d
Branch pushed to git repo; I updated commit sha1. New commits:
d1c2a0a  Try with latest version of pip

comment:299 Changed 4 weeks ago by
 Commit changed from d1c2a0a00f8c675ad218b502516dca12d161933d to f8599611360fcf04a54d28830c266183cd99ff99
Branch pushed to git repo; I updated commit sha1. New commits:
f859961  Run everything versus the correct python

comment:300 Changed 4 weeks ago by
 Commit changed from f8599611360fcf04a54d28830c266183cd99ff99 to 606966b86e35ae679edc08dd1c5fa156cff667e8
Branch pushed to git repo; I updated commit sha1. New commits:
606966b  Try to trigger macos workflow

comment:301 Changed 4 weeks ago by
 Commit changed from 606966b86e35ae679edc08dd1c5fa156cff667e8 to 541f7143df6f00116a3048fd34242e90a50ac40e
Branch pushed to git repo; I updated commit sha1. New commits:
541f714  Remove empty line?

comment:302 Changed 4 weeks ago by
 Commit changed from 541f7143df6f00116a3048fd34242e90a50ac40e to 42d25a14c00a68e58bcb904783483a5514eecb5f
Branch pushed to git repo; I updated commit sha1. New commits:
42d25a1  Still not?

comment:303 Changed 4 weeks ago by
 Commit changed from 42d25a14c00a68e58bcb904783483a5514eecb5f to f8599611360fcf04a54d28830c266183cd99ff99
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
comment:304 Changed 4 weeks ago by
 Commit changed from f8599611360fcf04a54d28830c266183cd99ff99 to f312a35f92ef30fa4a38ab54560686e2ef1bd5d6
Branch pushed to git repo; I updated commit sha1. New commits:
f312a35  Try to remove python3pip again

comment:305 Changed 4 weeks ago by
 Commit changed from f312a35f92ef30fa4a38ab54560686e2ef1bd5d6 to b01c4c98a6c590ec7526d2f87dba7a839ebee9df
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
b01c4c9  Install tox via pip

comment:306 Changed 4 weeks ago by
 Commit changed from b01c4c98a6c590ec7526d2f87dba7a839ebee9df to a1d989c3efd08d5f64612865f428102b328744da
Branch pushed to git repo; I updated commit sha1. New commits:
a1d989c  Install pip as wel of course

comment:307 Changed 4 weeks ago by
 Commit changed from a1d989c3efd08d5f64612865f428102b328744da to 442596137f517ff3cb3e365e1b8f79b41744aec2
Branch pushed to git repo; I updated commit sha1. New commits:
4425961  Strange...

comment:308 Changed 4 weeks ago by
 Commit changed from 442596137f517ff3cb3e365e1b8f79b41744aec2 to f2e436c459bad755685c6ca42a3d401b1fd29ed9
Branch pushed to git repo; I updated commit sha1. New commits:
f2e436c  Fix path of tox venv under mac

comment:309 in reply to: ↑ 290 Changed 4 weeks ago by
Replying to mkoeppe:
Replying to ghtobiasdiez:
Replying to mkoeppe:
 For linux: Why
make maxima conway_polynomials jmol conway_polynomials elliptic_curves combinatorial_designs polytopes_db graphs threejs
(it is not done in the mac workflow). Should we create amake
target that corresponds to what is needed?That's the only way I found to install these packages into
SAGE_SHARE
. Will add it also to the mac workflow. I cannot judge if it would make sense to add a special make target for it, especially since you are working on making these packages normal python packages.I've opened #31362 for this  which will add
make allsagelocal
.
OK, the toplevel target is now make buildlocal
.
comment:310 in reply to: ↑ 289 Changed 4 weeks ago by
Replying to ghtobiasdiez:
Thanks. Do you plan to work on it? Otherwise a quick fix is to use pipenv run sage t
instead of pipenv run sageruntests
as I wrote above.
comment:311 in reply to: ↑ 286 Changed 4 weeks ago by
Replying to ghtobiasdiez:
Concerning the cypari error, I have no clue what is needed for it to compile on mac.
Well it certainly needs the PARI library! When running pipenv
outside of the environment set using sage sh
, it depends on everything already installed in the system.
That's why I am saying the instructions to the users should run pipenv
within that shell.
comment:312 Changed 4 weeks ago by
 Commit changed from f2e436c459bad755685c6ca42a3d401b1fd29ed9 to a6e57808dcabbb60d9c7fdefb88ed7ca4be7caff
comment:313 Changed 4 weeks ago by
 Commit changed from a6e57808dcabbb60d9c7fdefb88ed7ca4be7caff to 85de00c2d4103d0edbcd2a41a2efcee52ea5c26e
Branch pushed to git repo; I updated commit sha1. New commits:
85de00c  Also install libffi8ubuntu1

comment:314 Changed 4 weeks ago by
 Commit changed from 85de00c2d4103d0edbcd2a41a2efcee52ea5c26e to 1760cf741ff4d392e7662075637350fe8e7894bb
Branch pushed to git repo; I updated commit sha1. New commits:
1760cf7  Or try to install libffi7

comment:315 Changed 4 weeks ago by
 Commit changed from 1760cf741ff4d392e7662075637350fe8e7894bb to efea01f09fb253083afc5c04ffa5824bdd4d731c
Branch pushed to git repo; I updated commit sha1. New commits:
efea01f  Install wget as well

comment:316 Changed 4 weeks ago by
 Commit changed from efea01f09fb253083afc5c04ffa5824bdd4d731c to 5a2cc305cf50e561d400a42d3e568cee7ea082a1
comment:317 Changed 4 weeks ago by
 Commit changed from 5a2cc305cf50e561d400a42d3e568cee7ea082a1 to 2ae43976cfacb331ae82426f25528213987225dd
Branch pushed to git repo; I updated commit sha1. New commits:
2ae4397  Mac: install pari as well

comment:318 followup: ↓ 322 Changed 4 weeks ago by
Is it actually desired that tox localhomebrew installs a fresh homebrew in the tox folder (if I'm not misreading the logs). Is there a way to use the system's homebrew?
comment:319 Changed 4 weeks ago by
 Commit changed from 2ae43976cfacb331ae82426f25528213987225dd to 88dabfa498cc3e12116d6f5f26e945ce4abd1429
comment:320 Changed 4 weeks ago by
 Commit changed from 88dabfa498cc3e12116d6f5f26e945ce4abd1429 to 8d5e37b6ab241cb2f98462de9b536937724ed977
comment:321 Changed 4 weeks ago by
 Commit changed from 8d5e37b6ab241cb2f98462de9b536937724ed977 to d9ac8df40f4eb4702795e544649a9e1830f25874
Branch pushed to git repo; I updated commit sha1. New commits:
d9ac8df  Revert changes to sage.env import

comment:322 in reply to: ↑ 318 Changed 4 weeks ago by
Replying to ghtobiasdiez:
Is it actually desired that tox localhomebrew installs a fresh homebrew in the tox folder
Yes, that's by design
Is there a way to use the system's homebrew?
From tox.ini comments:
# The variant "localhomebrewmacosusrlocal" uses the global installation in /usr/local # instead. It may install packages or update packages. It will not remove packages. # Use at your own risk.
comment:323 followup: ↓ 329 Changed 4 weeks ago by
Thanks! That is working nicely.
I think I've now addressed all your comments and suggestions.
The only thing is that since merging the latest develop branch the sage packages (installed via tox) are no longer found when running pipenv install
. At the commit https://github.com/sagemath/sagetracmirror/commit/4bdfef1b21da9f376cfd3c096cb9932fe5334e4e this was working, but stopped after I've merged the develop branch. Now packages such as linbox, singular and gap are still compiled by sage's make, but pkgconfig (in sage.env) cannot find them any longer and thus the c++ compilation stopped working. I don't have these problems locally, where I don't use tox. Do you have an idea which recent changes are the origin of this issue?
I've also tried your suggestion with sage sh pipenv
but run into several issues. I guess the problem is that pipenv is trying to reuse sages virtual environment.
comment:324 Changed 4 weeks ago by
 Commit changed from d9ac8df40f4eb4702795e544649a9e1830f25874 to 245d40fe531570a2bd3cc71e13d851a25e1ff858
Branch pushed to git repo; I updated commit sha1. New commits:
245d40f  Use correct version of fpylll for mac

comment:325 Changed 4 weeks ago by
 Commit changed from 245d40fe531570a2bd3cc71e13d851a25e1ff858 to 4b3c76f9fbe11c202c8de6aa7fe661bcfc7aeca7
Branch pushed to git repo; I updated commit sha1. New commits:
4b3c76f  Try to fix PKG_CONFIG_PATH and LD_LIBRARY_PATH

comment:326 followup: ↓ 331 Changed 4 weeks ago by
...never touch LD_LIBRARY_PATH... it's always wrong
comment:327 Changed 4 weeks ago by
Could you please take a look at #31362  it provides make buildlocal
, which I think is what you need
comment:328 followup: ↓ 335 Changed 4 weeks ago by
Thanks for making all the changes.
Right now it's hard for me to test anything from this ticket because the homebrew changes in #31335 conflict.
I would propose to do the following:
comment:329 in reply to: ↑ 323 Changed 4 weeks ago by
Replying to ghtobiasdiez:
I've also tried your suggestion with
sage sh pipenv
but run into several issues. I guess the problem is that pipenv is trying to reuse sages virtual environment.
See if you can work around this by having pipenv
use the base python that is shown in $SAGE_LOCAL/pyvenv.cfg
.
comment:330 Changed 4 weeks ago by
 Commit changed from 4b3c76f9fbe11c202c8de6aa7fe661bcfc7aeca7 to 7c89641c4f2eed5e7cac24776e5079c346d575de
Branch pushed to git repo; I updated commit sha1. New commits:
7c89641  Update lock file

comment:331 in reply to: ↑ 326 ; followup: ↓ 333 Changed 4 weeks ago by
Replying to mkoeppe:
...never touch LD_LIBRARY_PATH... it's always wrong
Ok, but what's then the correct way to do this?
comment:332 Changed 4 weeks ago by
 Dependencies changed from #30779, #30672, #30673, #30709, #30706, #30901, #31080, #29850, #29314, #30937, #30935, #30731 , #30859, #30923, #30910, #30944, #30770, #31216, #31029 to #30779, #30672, #30673, #30709, #30706, #30901, #31080, #29850, #29314, #30937, #30935, #30731 , #30859, #30923, #30910, #30944, #30770, #31216, #31029, #30912
comment:333 in reply to: ↑ 331 Changed 4 weeks ago by
Replying to ghtobiasdiez:
Replying to mkoeppe:
...never touch LD_LIBRARY_PATH... it's always wrong
Ok, but what's then the correct way to do this?
Building all libraries with rpath  see LDFLAGS setting in src/bin/sageenv
comment:334 Changed 4 weeks ago by
 Commit changed from 7c89641c4f2eed5e7cac24776e5079c346d575de to b68ce04b04b2d2104c6643700fa0f8cbe81f9f74
Branch pushed to git repo; I updated commit sha1. New commits:
2c40179  build/pkgs/sagelib/src: Update metadata

2a91862  build/pkgs/sagelib/src/README.rst: Add some text

6432727  Merge tag '9.3.beta2' into t/30912/sagelib__update_metadata_for_pypi_deployment

4a693f2  Move build/pkgs/sagelib/src/setup.cfg to SAGE_ROOT/src, replace by symlink

ea182d7  Copy changes from build/pkgs/sagelib/src to src

a1a10b9  src/VERSION.txt: New

5697335  src/setup.cfg: Add license_file=LICENSE.txt

deb9eb3  Merge tag '9.3.beta3' into t/30912/sagelib__update_metadata_for_pypi_deployment

7ad4c0e  Merge tag '9.3.beta4' into t/30912/sagelib__update_metadata_for_pypi_deployment

b68ce04  Merge branch 'u/mkoeppe/sagelib__update_metadata_for_pypi_deployment' of git://trac.sagemath.org/sage into public/build/inplace

comment:335 in reply to: ↑ 328 ; followup: ↓ 336 Changed 4 weeks ago by
Replying to mkoeppe:
I've merged the current versions of #30770 and #30912. Not sure what's the purpose of #31362. That would make the line make maxima conway_polynomials jmol conway_polynomials elliptic_curves combinatorial_designs polytopes_db graphs threejs
obsolete right? If it's just for this somewhat cosmetic improvement, I would suggest to turn around the tables and let #31362 depend on this ticket (and make the changes in the workflow there as well). Reason: I had to play a lot around with the different targets until everything works, so I'm a bit hesitant to change things again (and since unmerging a branch is not so easy, one cannot simply revert the experiment).
comment:336 in reply to: ↑ 335 ; followup: ↓ 338 Changed 4 weeks ago by
Replying to ghtobiasdiez:
Thanks.
Not sure what's the purpose of #31362. That would make the line
make maxima conway_polynomials jmol conway_polynomials elliptic_curves combinatorial_designs polytopes_db graphs threejs
obsolete right? If it's just for this somewhat cosmetic improvement
Yes, it's for making all standard nonPython packages available in SAGE_LOCAL
.
Reason: I had to play a lot around with the different targets until everything works, so I'm a bit hesitant to change things again
This does not sound good. If installing additional packages from the distribution in SAGE_LOCAL
breaks the proposed setup, then this ticket is definitely not ready.
comment:337 Changed 4 weeks ago by
Note also that #31362 does not change anything other than add two new make targets. So even if you find that these targets do not do what you need, there will be no need for unmerging it.
comment:338 in reply to: ↑ 336 Changed 4 weeks ago by
Replying to mkoeppe:
Reason: I had to play a lot around with the different targets until everything works, so I'm a bit hesitant to change things again
This does not sound good. If installing additional packages from the distribution in
SAGE_LOCAL
breaks the proposed setup, then this ticket is definitely not ready.
More in the sense, that it was a bit of tryanderror to find the minimal set of make targets that make the build pass. You could have course build every package in the sage distribution, but the point is that with pipenv only a couple of them are actually needed.
So buildlocal = sagelibbuilddeps + maxima conway_polynomials jmol conway_polynomials elliptic_curves combinatorial_designs polytopes_db graphs threejs
?
comment:339 Changed 4 weeks ago by
... minus the Python packages of sagelibbuilddeps
comment:340 Changed 4 weeks ago by
For example, it would not build numpy
.
comment:341 Changed 4 weeks ago by
Of course, there may be a number of nonPython libraries that are not needed IF you are able to pull in a wheel of the Python packages that somehow ship a bundled version of the nonPython library. For example, the numpy
wheels may come with their own BLAS  but it won't help because some of our own packages also need BLAS.
comment:342 Changed 4 weeks ago by
 Commit changed from b68ce04b04b2d2104c6643700fa0f8cbe81f9f74 to 2154a5b9f8f56ccc83bb45aa9bd10d659b015649
Branch pushed to git repo; I updated commit sha1. New commits:
bdb2237  Merge branch 't/30912/sagelib__update_metadata_for_pypi_deployment' into t/31357/fixup_src_version_txt_added_in__30912

c16651c  src/VERSION.txt: Use same format as build/pkgs/sagelib/src/VERSION.txt

2154a5b  Merge branch 'u/mkoeppe/fixup_src_version_txt_added_in__30912' of git://trac.sagemath.org/sage into public/build/inplace

comment:343 Changed 4 weeks ago by
 Dependencies changed from #30779, #30672, #30673, #30709, #30706, #30901, #31080, #29850, #29314, #30937, #30935, #30731 , #30859, #30923, #30910, #30944, #30770, #31216, #31029, #30912 to #30779, #30672, #30673, #30709, #30706, #30901, #31080, #29850, #29314, #30937, #30935, #30731 , #30859, #30923, #30910, #30944, #30770, #31216, #31029, #30912, #31357
comment:344 Changed 4 weeks ago by
 Commit changed from 2154a5b9f8f56ccc83bb45aa9bd10d659b015649 to 2773e6fd9c1868346d194227287dffd6c3642a5e
Branch pushed to git repo; I updated commit sha1. New commits:
55bc2aa  m4/sage_spkg_collect.m4: Generate install tree information for build/make/Makefile

d7dc9a2  build/make/Makefile.in: New targets allsagelocal, allsagevenv

27b589b  Merge branch 't/30912/sagelib__update_metadata_for_pypi_deployment' into t/31362/make_all_sage_local__make_all_sage_venv

8796008  Put pynac into SAGE_LOCAL, sagelib into SAGE_VENV

d6831b1  build/make/Makefile.in: Add allbuildlocal, allbuildvenv, which include dependency on toolchain

61f6ba6  Makefile: Add toplevel targets buildlocal, buildvenv

e354672  Merge branch 'u/mkoeppe/make_all_sage_local__make_all_sage_venv' of git://trac.sagemath.org/sage into public/build/inplace

2773e6f  Use buildlocal target

comment:345 Changed 4 weeks ago by
 Dependencies changed from #30779, #30672, #30673, #30709, #30706, #30901, #31080, #29850, #29314, #30937, #30935, #30731 , #30859, #30923, #30910, #30944, #30770, #31216, #31029, #30912, #31357 to #30779, #30672, #30673, #30709, #30706, #30901, #31080, #29850, #29314, #30937, #30935, #30731 , #30859, #30923, #30910, #30944, #30770, #31216, #31029, #30912, #31357, #31362
comment:346 Changed 4 weeks ago by
ok, let's see if that works
comment:347 Changed 4 weeks ago by
In any case thanks to merging these tickets, I am able to merge the tickets that fix homebrew  so I'll be able to test later today.
comment:348 Changed 3 weeks ago by
../sage sh c 'ARCHFLAGS= PIP_NO_BUILD_ISOLATION=1 pipenv install v v'
got me a bit farther. (Setting ARCHFLAGS
to the empty string helps on macOS; PIP_NO_BUILD_ISOLATION=1
helps with some of the Python packages that declare their build dependencies incorrectly (see #30922). Now it fails with this:
raise MetadataInconsistent(self._ireq, "name", dist.project_name) pip._internal.exceptions.MetadataInconsistent: Requested fpylll from https://files.pythonhosted.org/packages/ad/c5/f30f354dcce03b928702414fdbdd916d60915c6b144360c88707784385bb/fpylll0.5.4.tar.gz#egg=fpylllrecent (from r /var/folders/38/wnh4gf1552g_crsjnv2vmmww0000gp/T/pipenv7zg7bw1srequirements/pipenvb8jmkho7requirement.txt (line 1)) has different name in metadata: 'fpylll' Removed build tracker: '/private/var/folders/38/wnh4gf1552g_crsjnv2vmmww0000gp/T/pipreqtrackerz5q5pu09' ... raise exceptions.InstallError(c.dep.name, extra=err_lines) pipenv.exceptions.InstallError: ERROR: Couldn't install package: fpylllrecent Package installation failed...
comment:349 Changed 3 weeks ago by
 Commit changed from 2773e6fd9c1868346d194227287dffd6c3642a5e to e70f22ba61f2d4c1e0de5a254a15844129df7cab
comment:350 followup: ↓ 353 Changed 3 weeks ago by
This error also occurred on the github workflow. I tried to fix it by setting the name in the pipfile. If this doesn't work can you please remove the fpylllrecent in the pipfile and invoke pipenv install https://files.pythonhosted.org/packages/ad/c5/f30f354dcce03b928702414fdbdd916d60915c6b144360c88707784385bb/fpylll0.5.4.tar.gz
This should work. Could you then please post what pipenv writes to the pipfile, thanks!
buildlocal
seems to work, except that it installs some python related tools/packages (pip, setuptools_scm, wheel) and doesn't install maxima.
comment:351 Changed 3 weeks ago by
 Commit changed from e70f22ba61f2d4c1e0de5a254a15844129df7cab to 8c22f9ec046cb8253531056e0a7aff7118ca778e
Branch pushed to git repo; I updated commit sha1. New commits:
8c22f9e  Set LIBRARY_PATH

comment:352 Changed 3 weeks ago by
 Commit changed from 8c22f9ec046cb8253531056e0a7aff7118ca778e to e17019c5528c867367988d7fa99bcef76f320949
Branch pushed to git repo; I updated commit sha1. New commits:
e17019c  Install old readline as well

comment:353 in reply to: ↑ 350 Changed 3 weeks ago by
Replying to ghtobiasdiez:
buildlocal
seems to work, except that it installs some python related tools/packages (pip, setuptools_scm, wheel) and doesn't install maxima.
wheel (which pulls in the python toolchain) is fixed in #31321.
maxima  could you double check?
comment:354 Changed 3 weeks ago by
Ok good. And false alarm, maxima is installed (the output is just condensed to one line in contrast to the pages of text for make maxima
which has thrown me off).
The latest commit should hopefully fix the fpylll error.
comment:355 Changed 3 weeks ago by
 Commit changed from e17019c5528c867367988d7fa99bcef76f320949 to e2ed97f2ba3af909838802c95f4151282090c7d7
Branch pushed to git repo; I updated commit sha1. New commits:
e2ed97f  Fix fpylll

comment:356 Changed 3 weeks ago by
I will put a rebased/squashed version of the setup.py
/sage_setup
changes on #31377, because this part seems ready and I hope we can get it into 9.3 to get developers used to the inplace install idea.
comment:357 followup: ↓ 362 Changed 3 weeks ago by
(It looks to me like the venv aspects of this ticket will take more time; in particular since they seem to involve getting an uncooperative random set of wheels from PyPI to cooperate with the system.)
comment:358 Changed 3 weeks ago by
It's quite the opposite: since some of the python packages (fyplll, cypari2) don't provide wheels, system packages need to be installed which is quite hard to coordinate across different systems.
comment:359 Changed 3 weeks ago by
But the Sage distribution has already solved this problem. Just build within a sagesh.
comment:360 Changed 3 weeks ago by
Please take a look if #31377 looks agreeable to you.
comment:361 in reply to: ↑ 280 Changed 3 weeks ago by
comment:362 in reply to: ↑ 357 Changed 3 weeks ago by
Replying to mkoeppe:
(It looks to me like the venv aspects of this ticket will take more time; in particular since they seem to involve getting an uncooperative random set of wheels from PyPI to cooperate with the system.)
On this topic, see also the discussion at https://github.com/sagemath/cypari2/issues/105.
I think it is a desirable feature to be able to edit the source code in an inplace install of sage (ie what pip e
provides). This would greatly benefit all Sage developers. However, I have hard time to understand the benefit of the venv. On the other hand, I do not understand how wheels could solve the libraries/softwares dependencies. More precisely, here is a scenario (which happens to be happening in Sage)
 SageMath depends on
pylib1
andpylib2
that uses the samelibfoo
C library. Both of these libraries provide wheels that bundlelibfoo
to make some of their users happy.  SageMath also depends on
pylib3
that depends on bothpylib1
andpylib2
and uses custom Cython code that uses functions fromlibfoo
.pylib3
does not provide wheel yet. How should it be implemented?
comment:363 Changed 3 weeks ago by
Hi Vincent, thanks for joining the discussion.
For a discussion of possible distribution strategies using wheels, let's use #31251 (Metaticket: Distribution as wheels), which also has a link to a recent post by Gommers on packaging. I agree with you that it is a hard problem.
For Sage 9.3, I have a much more modest goal regarding wheels that is already fully implemented and is only waiting for review (see ticket list in #29705 for tickets by milestone). Since #29500 (already in Sage 9.2), we generate wheels in $SAGE_LOCAL/var/lib/sage/wheels
. These wheels are not distributable as they do not bundle their shared library dependencies but rather rpath into $SAGE_LOCAL
. Users can make their own venvs for development purposes where they install the prebuilt wheels from this directory. #31045 and #31279 add some missing wheels. #30913 generates dependency metadata that makes it easier to set up the venv; and adds tests using tox
.
For editable install of sagelib as part of the normal build system (no extra venv) using Tobias' new setup.py
from this ticket, see #31377.
comment:364 Changed 3 weeks ago by
 Dependencies changed from #30779, #30672, #30673, #30709, #30706, #30901, #31080, #29850, #29314, #30937, #30935, #30731 , #30859, #30923, #30910, #30944, #30770, #31216, #31029, #30912, #31357, #31362 to #30779, #30672, #30673, #30709, #30706, #30901, #31080, #29850, #29314, #30937, #30935, #30731 , #30859, #30923, #30910, #30944, #30770, #31216, #31029, #30912, #31357, #31362, #31365
comment:365 Changed 3 weeks ago by
 Commit changed from e2ed97f2ba3af909838802c95f4151282090c7d7 to 3e60c00b3cb16ae500584baf484017b922ac3988
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
804ebd7  sage_setup.dpcbuild.AllBuilder: Restrict workaround to macOS

b4ceee5  sage_setup.docbuild: In the workaround, do not go through build_many to build serially

7369a3e  Merge branch 't/31344/homebrew__docbuild_crashes__libtcl_atforkprepare' into t/31365/add_ntl_to_cython_aliases

ce03e6b  src/sage/schemes/hyperelliptic_curves/hypellfrob.pyx: Merge distutils directives

32576b4  sage.misc.cython: Add NTL aliases, cache result of cython_aliases

6e30e3a  Use variables NTL_INCDIR, NTL_LIBDIR in sage_conf, separate from NTL_PREFIX used in sagebuildenvconfig; set std=c++11 in NTL_CFLAGS

7b1e27b  Merge distutils directives for Cython files using ntl

4d2828d  Switch Cython files that use NTL to language = c++

993c35c  build/pkgs/ntl/spkgconfigure.m4: Fix up

3e60c00  Merge branch 'u/mkoeppe/add_ntl_to_cython_aliases' of git://trac.sagemath.org/sage into public/build/inplace

comment:366 Changed 3 weeks ago by
 Status changed from needs_work to needs_review
So, I think, I've found a solution to the fpylll problems. In fact, the github workflow should now also work for macos (at least the last run installed everything fine except sage, which failed due to an NTL error that is fixed with #31365). Since macos runs on the sage repo run currently really slow, you can find the latest run at https://github.com/tobiasdiez/sage/actions.
comment:367 Changed 3 weeks ago by
I've split out another ticket from here: #31384
comment:368 Changed 3 weeks ago by
 Priority changed from critical to major
 Work issues set to rebase on dependencies + #31384, #31377
comment:369 Changed 3 weeks ago by
 Dependencies changed from #30779, #30672, #30673, #30709, #30706, #30901, #31080, #29850, #29314, #30937, #30935, #30731 , #30859, #30923, #30910, #30944, #30770, #31216, #31029, #30912, #31357, #31362, #31365 to #30779, #30672, #30673, #30709, #30706, #30901, #31080, #29850, #29314, #30937, #30935, #30731 , #30859, #30923, #30910, #30944, #30770, #31216, #31029, #30912, #31357, #31362, #31365, #31377
 Work issues rebase on dependencies + #31384, #31377 deleted
comment:370 Changed 3 weeks ago by
 Commit changed from 3e60c00b3cb16ae500584baf484017b922ac3988 to 13d90ee00e330ff759d0f8fc0b4e3868a477248c
Branch pushed to git repo; I updated commit sha1. New commits:
217c675  Try to fix gmp issue on mac

25df925  For mac not linux...

90a5104  Cleanup workflow

0c4ba62  Try to only specify gmp includes

a978458  Fix a few more errors

c1ce878  Also for the manual install

13d90ee  Remove a few unnecessary changes

comment:371 Changed 3 weeks ago by
 Commit changed from 13d90ee00e330ff759d0f8fc0b4e3868a477248c to 08fef7dd62544dfb13327d4ad38fca5ab6691190
comment:372 Changed 3 weeks ago by
 Commit changed from 08fef7dd62544dfb13327d4ad38fca5ab6691190 to 8f8b61533287f67be77a4920f7ffceb5ebc39b0d
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
18df2dc  Merge branch 't/31365/add_ntl_to_cython_aliases' into t/31377/__configure___enable_editable

19ca17a  src/sage/graphs/graph_decompositions/tdlib/sage_tdlib.cpp: Move one directory level higher

f02963b  Merge branch 't/31389/remove_clash_of_tdlib__with_tdlib_pyx_in__src_sage_graphs_graph_decompositions' into t/31377/__configure___enable_editable

2b293f8  src/setup.py: Install the jupyter kernel using sage_install

b05e4ba  build/make/Makefile.in (sagelibclean): Remove generated .c and .cpp files

271d427  src/setup.py: Fix sage.misc.sageinspect by setting Cython.Compiler.Options.embed_pos_in_docstring

f6b2fc4  src/sage/repl/interpreter.py: Fix a doctest for editable installs of sagelib

9a0731a  sage.interacs.debugger: Remove deprecated module

e5d47ce  Merge branch 't/31390/remove_deprecated_sage_interacts_debugger' into t/31377/__configure___enable_editable

8f8b615  Merge branch 't/31377/__configure___enable_editable' into t/30371/public/build/inplace

comment:373 followup: ↓ 375 Changed 3 weeks ago by
Merged #31377, resolving edit conflicts.
comment:374 followup: ↓ 376 Changed 3 weeks ago by
I don't know which problem you are trying to solve with the big try/except. I don't think reducing ImportError
is the correct solution.
If you are trying to make setup.py sdist
work when libs are missing, consider using the code from build/pkgs/sagelib/src/setup.py
line 61
comment:375 in reply to: ↑ 373 ; followup: ↓ 378 Changed 3 weeks ago by
comment:376 in reply to: ↑ 374 ; followup: ↓ 379 Changed 3 weeks ago by
Replying to mkoeppe:
I don't know which problem you are trying to solve with the big try/except. I don't think reducing
ImportError
is the correct solution.If you are trying to make
setup.py sdist
work when libs are missing, consider using the code frombuild/pkgs/sagelib/src/setup.py
line 61
Yes, that was my goal. Creating the virtual environment otherwise fails as Cython is not yet installed into it. The file you refer to doesn't seem to exist.
comment:377 Changed 3 weeks ago by
 Commit changed from 8f8b61533287f67be77a4920f7ffceb5ebc39b0d to fec55234a00e46d023ad583f8c6284b970141ab0
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
fec5523  Merge branch 't/31377/__configure___enable_editable' into t/30371/public/build/inplace

comment:378 in reply to: ↑ 375 Changed 3 weeks ago by
Replying to ghtobiasdiez:
Replying to mkoeppe:
Merged #31377, resolving edit conflicts.
Thanks, but this did revert some of my changes (in particular to setup.py)....
Sorry, my mistake. I've redone the merge now. Please take a look.
comment:379 in reply to: ↑ 376 Changed 3 weeks ago by
Replying to ghtobiasdiez:
Replying to mkoeppe:
I don't know which problem you are trying to solve with the big try/except. I don't think reducing
ImportError
is the correct solution.If you are trying to make
setup.py sdist
work when libs are missing, consider using the code frombuild/pkgs/sagelib/src/setup.py
line 61Yes, that was my goal. Creating the virtual environment otherwise fails as Cython is not yet installed into it. The file you refer to doesn't seem to exist.
comment:380 Changed 2 weeks ago by
comment 374?
I've worked a bit on it. The current commit is a semiworking prototype, in the sense that all cython files are successfully compiled to C files if invoked via
python src/setup.py build_ext inplace
. The second compilation of the c files does not yet work.I had problems that none of the source cython files were correctly found on my system (wsl) and that a few dependencies in the current build files were not met (
'linbox', 'Singular', 'm4ri', 'zlib', 'cblas'
). For this reason, I started from scratch and only included what was needed so far. Any pointers to how to install these dependencies (I think they are needed for the second compilation).New commits:
Semiworking prototype for inplace compilation