#31577 closed enhancement (fixed)

Move SAGE_ROOT/build/pkgs/*/src to SAGE_ROOT/pkgs/*

Reported by: Matthias Köppe Owned by:
Priority: major Milestone: sage-9.4
Component: build Keywords:
Cc: John Palmieri, Dima Pasechnik, François Bissey Merged in:
Authors: Matthias Koeppe Reviewers: Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: 7568dc6 (Commits, GitHub, GitLab) Commit: 7568dc6dc7a2325525aaa4618ec1bc0bc417917b
Dependencies: Stopgaps:

Status badges

Description (last modified by Matthias Köppe)

In the current layout, these embedded source trees are a little difficult to find / complicated to type.

The proposed location is:

SAGE_ROOT
- pkgs
  - sage-conf
    - setup.py
    - bin/
    - sage_conf.py.in
  - sage-docbuild
    - setup.py
    - sage_docbuild   -> symlinks to SAGE_ROOT/sage_docbuild/
  - sage-sws2rst
    - setup.py
    - bin/
    - sage_sws2rst/
  - sagemath-standard
    - setup.py
    - bin             -> symlinks to SAGE_ROOT/src/bin/        
    - sage            -> symlinks to SAGE_ROOT/src/sage/
    - sage_setup      -> symlinks to SAGE_ROOT/src/sage_setup/ (removed in #29847)

that is, the new directory pkgs will be on the same level as

SAGE_ROOT
- src

the unchanged monolithic sagelib source tree, which will continue to contain subdirectories

  - bin/ 
  - doc/
  - sage/
  - sage_docbuild/      # was sage_setup/docbuild/ in Sage 9.2
  - sage_setup/ 

We add symlinks from the previous locations SAGE_ROOT/build/pkgs/sage[math]_*/src to the new locations SAGE_ROOT/pkgs/sage[math]-*

#29847 will add:

SAGE_ROOT
- pkgs
  - sage-setup
    - setup.py
    - sage_setup         -> symlinks to src/sage_setup/

and remove sage_setup from the sagemath-standard distribution.

Other modularization tickets (#29705) will add

SAGE_ROOT
- pkgs
  - sagemath-core/
  - sagemath-brial/
  - sagemath-giac/
  - sagemath-meataxe/
  - sagemath-tdlib/

etc.

#29868 will add

SAGE_ROOT
- pkgs
  - sagemath-doc-html/
  - sagemath-doc-pdf/

All files that contain Sage doctests will remain in the monolithic src/ source tree; the source trees of the distributions symlink there. This may be changed in a follow-up ticket regarding the modularization of doctesting.

Before merging this branch into a non-distclean worktree, it is useful to use

  git clean -f -x build/pkgs/*/src

Change History (92)

comment:1 Changed 21 months ago by Matthias Köppe

Alternatively, these directories could just be put into SAGE_ROOT:

SAGE_ROOT
 - sage_conf
 - sage_docbuild
 - sage_sws2rst
 - sagemath_standard

comment:2 Changed 21 months ago by Matthias Köppe

Cc: John Palmieri Dima Pasechnik added
Description: modified (diff)
Summary: Move SAGE_ROOT/build/pkgs/{sagelib,sage_conf,sage_sws2rst,sage_docbuild}/src to SAGE_ROOT/src/pkgs/...Move SAGE_ROOT/build/pkgs/*/src to SAGE_ROOT/*

(made this change in the ticket description)

Last edited 21 months ago by Matthias Köppe (previous) (diff)

comment:3 Changed 20 months ago by Matthias Köppe

Description: modified (diff)
Summary: Move SAGE_ROOT/build/pkgs/*/src to SAGE_ROOT/*Move SAGE_ROOT/build/pkgs/*/src to SAGE_ROOT/*, rename SAGE_ROOT/build to SAGE_ROOT/sage_bootstrap

comment:4 Changed 20 months ago by Matthias Köppe

Description: modified (diff)

comment:5 Changed 20 months ago by Matthias Köppe

Description: modified (diff)

comment:6 Changed 20 months ago by Matthias Köppe

Description: modified (diff)
Summary: Move SAGE_ROOT/build/pkgs/*/src to SAGE_ROOT/*, rename SAGE_ROOT/build to SAGE_ROOT/sage_bootstrapMove SAGE_ROOT/build/pkgs/*/src to SAGE_ROOT/*

comment:7 Changed 20 months ago by Matthias Köppe

Dependencies: #30913

comment:8 Changed 20 months ago by Matthias Köppe

Branch: u/mkoeppe/move_sage_root_build_pkgs___src_to_sage_root____rename_sage_root_build_to_sage_root_sage_bootstrap

comment:9 Changed 20 months ago by Matthias Köppe

Authors: Matthias Koeppe
Cc: François Bissey added
Commit: 8862b55639d36c5fd85d5c998371a657e5867762
Status: newneeds_review

Last 10 new commits:

0283da5build/make/Makefile.in: Add wheel, setuptools_wheel to PYTHON_TOOLCHAIN to make sure that PEP 517 packages have a complete build system
f720722build/pkgs/sagelib/src/tox.ini: Add factor nobuildisolation
c451b31src/setup.cfg.m4 (install_requires): Add sage_conf
6700223Merge tag '9.3.rc0' into t/30913/sagelib__add_setup_cfg__install_requires_
04da2c6build/pkgs/ipywidgets: Patch out declaring install-requires of nbformat and jupyterlab-widgets
68dc845Merge branch 't/30913/sagelib__add_setup_cfg__install_requires_' into t/31577/move_sage_root_build_pkgs___src_to_sage_root____rename_sage_root_build_to_sage_root_sage_bootstrap
9181571Move /build/pkgs/sage_conf/src to /sage_conf, leave symlink behind
4f480b4Move /build/pkgs/sage_docbuild/src to /sage_docbuild, leave symlink behind
cc04151Move /build/pkgs/sage_sws2rst/src to /sage_sws2rst, leave symlink behind
8862b55Move /build/pkgs/sagelib/src to /sagemath_standard, leave symlink behind

comment:10 Changed 20 months ago by git

Commit: 8862b55639d36c5fd85d5c998371a657e586776273d83c2403277a93e7ccc6987a3345b7caa5fc2d

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

73d83c2Move /build/pkgs/sagelib/src to /sagemath_standard, leave symlink behind

comment:11 Changed 20 months ago by git

Commit: 73d83c2403277a93e7ccc6987a3345b7caa5fc2d6ff4ba16f656b8a93942a905e130fb9181acfc48

Branch pushed to git repo; I updated commit sha1. New commits:

223e2baconfigure.ac: Use shorter path to sage_conf/
9bbb57bFixup: Add build/pkgs/sage_sws2rst/src symlink
ee0a21eFixup: Add build/pkgs/sagelib/src symlink
6ff4ba1.gitignore: Update for new source tree locations

comment:12 Changed 20 months ago by git

Commit: 6ff4ba16f656b8a93942a905e130fb9181acfc48ef2fee8943776d083ac2ab831a930d80b14720fc

Branch pushed to git repo; I updated commit sha1. New commits:

ef2fee8WIP: Fix up docker build

comment:13 Changed 20 months ago by John Palmieri

sage_docbuild   -> symlinks to SAGE_ROOT/src/sage_docbuild

Would it make sense to reverse this: put the files in SAGE_ROOT/sage_docbuild and then make SAGE_ROOT/src/sage_docbuild into a link? Is there any reason to keep SAGE_ROOT/src/sage_docbuild? If so, why also create the top-level directory?

What about SAGE_ROOT/src/sage_setup, along the same lines?

comment:14 Changed 20 months ago by Matthias Köppe

This is a great question.

There is no technical reason for keeping SAGE_ROOT/src/sage_docbuild at all. Likewise, after #29847, there will be no technical reason for keeping SAGE_ROOT/src/sage_setup.

But in #29705, I promised not to change the structure of the source tree, which I interpret as keeping the monolithic tree SAGE_ROOT/src unchanged. It is so that we don't overwhelm the developer community by making too many changes at the same time.

Of course, it could be argued that sage_docbuild is a new name anyway, and very few people will know or care about this package.

So if you think that's better, I can just move the sage_docbuild sources to SAGE_ROOT/sage_docbuild/sage_docbuild on this ticket.

(I'd rather not put reverse symlinks.)

comment:15 Changed 20 months ago by git

Commit: ef2fee8943776d083ac2ab831a930d80b14720fc12df8707ee893b54784a5f87b6cbaa9d22673c65

Branch pushed to git repo; I updated commit sha1. New commits:

12df870No more src/sage_docbuild

comment:16 Changed 20 months ago by Matthias Köppe

Description: modified (diff)

comment:17 Changed 20 months ago by Matthias Köppe

Description: modified (diff)

comment:18 Changed 20 months ago by Matthias Köppe

I will make another change, renaming the top-level directories from using underscores to using dashes. This will emphasize that these are the names of distribution packages (PyPI normalizes to dashes).

comment:19 Changed 20 months ago by Matthias Köppe

Description: modified (diff)

comment:20 Changed 20 months ago by Matthias Köppe

(This change is making the kludge that I had to add to the top-level Makefile unnecessary. The command for building make sage_conf, will no longer be fooled by the existence of a top-level directory of that name.)

comment:21 Changed 20 months ago by git

Commit: 12df8707ee893b54784a5f87b6cbaa9d22673c651fe5f3dfb768731b7e05e10380d9b70d42885ea2

Branch pushed to git repo; I updated commit sha1. New commits:

1fe5f3dSpell distribution source directories sage-conf, sage-docbuild, sage-sws2rst, sagemath-standard with dashes, not underscores

comment:22 Changed 20 months ago by Matthias Köppe

Description: modified (diff)

comment:23 Changed 20 months ago by Matthias Köppe

Description: modified (diff)

comment:24 Changed 20 months ago by git

Commit: 1fe5f3dfb768731b7e05e10380d9b70d42885ea28ad9b3e991492e6b6edd9d56ca48a4189d639545

Branch pushed to git repo; I updated commit sha1. New commits:

8ad9b3eFixup renames

comment:25 Changed 20 months ago by git

Commit: 8ad9b3e991492e6b6edd9d56ca48a4189d6395454c0d38d13e4e3789c2c8b3ea8dc178fbf414ed11

Branch pushed to git repo; I updated commit sha1. New commits:

4c0d38dAlso standardize on using dashes in distribution names in setup.cfg

comment:26 Changed 20 months ago by Matthias Köppe

Description: modified (diff)

comment:27 Changed 20 months ago by git

Commit: 4c0d38d13e4e3789c2c8b3ea8dc178fbf414ed1147b890ed00ade56e3a1d3a461a89a738cae02cf4

Branch pushed to git repo; I updated commit sha1. New commits:

47b890eMerge tag '9.3.rc3' into t/31577/move_sage_root_build_pkgs___src_to_sage_root____rename_sage_root_build_to_sage_root_sage_bootstrap

comment:28 Changed 20 months ago by John Palmieri

make ptestlong fails to run any doctests because of this error:

Using --optional=build,dochtml,homebrew,pip,sage,sage_spkg
Doctesting entire Sage library.
Traceback (most recent call last):
  File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/src/bin/sage-runtests", line 144, in <module>
    err = DC.run()
  File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/local/lib/python3.9/site-packages/sage/doctest/control.py", line 1204, in run
    self.expand_files_into_sources()
  File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/local/lib/python3.9/site-packages/sage/doctest/control.py", line 788, in expand_files_into_sources
    self.sources = [FileDocTestSource(path, self.options) for path in expand()]
  File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/local/lib/python3.9/site-packages/sage/doctest/control.py", line 788, in <listcomp>
    self.sources = [FileDocTestSource(path, self.options) for path in expand()]
  File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.3.rc2/local/lib/python3.9/site-packages/sage/doctest/sources.py", line 528, in __init__
    raise ValueError("unknown file extension %r"%ext)
ValueError: unknown file extension ''
make: *** [ptestlong] Error 1

I believe the problem is that it trying to doctest src/sage_docbuild.

comment:29 Changed 20 months ago by Matthias Köppe

OK, I guess I have to retract my claim that there is no technical reason for having src/sage_docbuild. In part that's #31352 though

comment:30 Changed 20 months ago by John Palmieri

I'm currently testing this change:

  • src/sage/doctest/control.py

    diff --git a/src/sage/doctest/control.py b/src/sage/doctest/control.py
    index afb0d6cbcf..45ec2263fb 100644
    a b class DocTestController(SageObject): 
    695695            # don't make sense to run outside a build environment
    696696            if have_git:
    697697                self.files.append(opj(SAGE_SRC, 'sage_setup'))
    698                 self.files.append(opj(SAGE_SRC, 'sage_docbuild'))
     698                self.files.append(opj(SAGE_ROOT, 'sage-docbuild'))
    699699            self.files.append(SAGE_DOC_SRC)
    700700
    701701        if self.options.all or (self.options.new and not have_git):
Last edited 20 months ago by John Palmieri (previous) (diff)

comment:31 Changed 20 months ago by Matthias Köppe

I think I'd rather undo the change in 12df870 and keep it for another ticket.

I think it's better to keep everything that is subject to Sage doctesting in one physical tree

comment:32 Changed 20 months ago by Matthias Köppe

(Modularization of doctesting is a separate issue.)

comment:33 in reply to:  30 ; Changed 20 months ago by François Bissey

Replying to jhpalmieri:

I'm currently testing this change:

  • src/sage/doctest/control.py

    diff --git a/src/sage/doctest/control.py b/src/sage/doctest/control.py
    index afb0d6cbcf..45ec2263fb 100644
    a b class DocTestController(SageObject): 
    695695            # don't make sense to run outside a build environment
    696696            if have_git:
    697697                self.files.append(opj(SAGE_SRC, 'sage_setup'))
    698                 self.files.append(opj(SAGE_SRC, 'sage_docbuild'))
     698                self.files.append(opj(SAGE_ROOT, 'sage-docbuild'))
    699699            self.files.append(SAGE_DOC_SRC)
    700700
    701701        if self.options.all or (self.options.new and not have_git):

I really don't like that kind of changes. In sage-on-distro SAGE_SRC becomes SAGE_LIB at runtime when SAGE_ROOT is undefined. That would break things there. Not to mention I am against introducing more SAGE_ROOT outside of the build system as a matter of principle. As Matthias says modularization of doctesting is a separate issue and that's what would be most appropriate there.

comment:34 in reply to:  33 Changed 20 months ago by John Palmieri

Replying to fbissey:

Replying to jhpalmieri:

I'm currently testing this change:

  • src/sage/doctest/control.py

    diff --git a/src/sage/doctest/control.py b/src/sage/doctest/control.py
    index afb0d6cbcf..45ec2263fb 100644
    a b class DocTestController(SageObject): 
    695695            # don't make sense to run outside a build environment
    696696            if have_git:
    697697                self.files.append(opj(SAGE_SRC, 'sage_setup'))
    698                 self.files.append(opj(SAGE_SRC, 'sage_docbuild'))
     698                self.files.append(opj(SAGE_ROOT, 'sage-docbuild'))
    699699            self.files.append(SAGE_DOC_SRC)
    700700
    701701        if self.options.all or (self.options.new and not have_git):

I really don't like that kind of changes. In sage-on-distro SAGE_SRC becomes SAGE_LIB at runtime when SAGE_ROOT is undefined. That would break things there. Not to mention I am against introducing more SAGE_ROOT outside of the build system as a matter of principle. As Matthias says modularization of doctesting is a separate issue and that's what would be most appropriate there.

I think that on distros, have_git will be false (it tests whether SAGE_ROOT/.git exists), so this block, and the resulting doctests, will be skipped entirely.

comment:35 Changed 20 months ago by git

Commit: 47b890ed00ade56e3a1d3a461a89a738cae02cf408b8306b9195facfc38bcfd067adeb0c7b064efa

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

a24fe5cMerge #30913
ca9935eMove /build/pkgs/sage_conf/src to /sage_conf, leave symlink behind
f81216aMove /build/pkgs/sage_docbuild/src to /sage_docbuild, leave symlink behind
b5e30f4Move /build/pkgs/sage_sws2rst/src to /sage_sws2rst, leave symlink behind
25e6d14Move /build/pkgs/sagelib/src to /sagemath_standard, leave symlink behind
e253a2b.gitignore: Update for new source tree locations
1cfeb06Spell distribution source directories sage-conf, sage-docbuild, sage-sws2rst, sagemath-standard with dashes, not underscores
8d4ea4eAlso standardize on using dashes in distribution names in setup.cfg
08b8306WIP: Fix up docker build

comment:36 Changed 20 months ago by François Bissey

You are right, it will skipped. Although technically they could be run if present.

comment:37 Changed 20 months ago by Matthias Köppe

Nevertheless I have removed that change. Too complicated for 1 ticket.

comment:38 Changed 20 months ago by Matthias Köppe

Description: modified (diff)

comment:39 in reply to:  36 ; Changed 20 months ago by John Palmieri

Replying to fbissey:

You are right, it will skipped. Although technically they could be run if present.

One of the goals of that particular method is to exclude files if in a distro. I don't know enough about distros to know if the method is well-implemented, though. (I didn't write it.)

Some related questions about this ticket: how does this new organization affect distros, and should we be doctesting any of the pieces that are being moved to SAGE_ROOT?

comment:40 in reply to:  39 Changed 20 months ago by Matthias Köppe

Replying to jhpalmieri:

should we be doctesting any of the pieces that are being moved to SAGE_ROOT?

sage-sws2rst has its own simple test script, not related to the Sage doctester.

The other packages do not have separate tests but are doctested as part of the Sage. The assumption is here that the doctester is run after all of these packages are installed. This assumption would continue to hold even as the modularization goes forward. I don't really have a plan for modularizing the doctesting, other than #30778.

comment:41 in reply to:  39 ; Changed 20 months ago by Matthias Köppe

Replying to jhpalmieri:

how does this new organization affect distros

I would hope, not at all. All paths that were valid before are still valid afterwards thanks to the symlinks build/pkgs/SOME_SPKG/src -> /SOME-SPKG

comment:42 Changed 20 months ago by Matthias Köppe

Description: modified (diff)

comment:43 Changed 20 months ago by Matthias Köppe

Description: modified (diff)

comment:44 Changed 20 months ago by Matthias Köppe

Description: modified (diff)

comment:45 in reply to:  41 Changed 20 months ago by François Bissey

Replying to mkoeppe:

Replying to jhpalmieri:

how does this new organization affect distros

I would hope, not at all. All paths that were valid before are still valid afterwards thanks to the symlinks build/pkgs/SOME_SPKG/src -> /SOME-SPKG

As far as I am concerned, anything living under build/pkgs/ should have a separate tarball and be installed as a separate package. This has been somewhat challenged by the way Matthias is conducting the modularization. But that's the goal at the end of the day. If it is there it is not supposed to be anything special anymore.

That being said sage_conf is being a pain because it is still very special as far as I can see. It still need a real SAGE_ROOT tree. But that is a different conversation and ticket.

comment:46 Changed 20 months ago by Matthias Köppe

That's right, after this ticket, SAGE_ROOT/sage-docbuild, SAGE_ROOT/sage-sws2rst, SAGE_ROOT/sagemath-standard, and SAGE_ROOT/sage-conf are standard source trees of Python distribution packages, containing nothing that is specific to Sage-the-distribution.

It is possible to build sdists from each of these source trees using setup.py sdist, and wheels using setup.py bdist_wheel.

Actually also sage-conf has the same status: You can build an sdist or a wheel from it. However, this version of sage-conf is not useful for putting it on PyPI. #29039 and #31396 create separate versions of sage-conf that are suitable for distribution.

comment:47 in reply to:  46 Changed 20 months ago by François Bissey

Replying to mkoeppe:

Actually also sage-conf has the same status: You can build an sdist or a wheel from it. However, this version of sage-conf is not useful for putting it on PyPI. #29039 and #31396 create separate versions of sage-conf that are suitable for distribution.

That's right, I had to remove a lot of stuff from the PyPI version before I could make it do the bits that would be useful to me properly as someone who build stuff from source. Considering the amount of stuff I just want to throw out, it is almost a different package altogether.

comment:48 Changed 20 months ago by Matthias Köppe

Yes, downstream packaging is encouraged to write a separate sage_conf for their purposes.

comment:49 in reply to:  48 Changed 20 months ago by François Bissey

Replying to mkoeppe:

Yes, downstream packaging is encouraged to write a separate sage_conf for their purposes.

Beurk. Anyway, we should have another meeting may be the next sage-days. I think there are things we should do to the design of "those" configuration files at large, and that could make it easier for distros in the process. I need to figure if what I think of who be easily doable python side. And it is definitely a separate conversation from this ticket.

comment:50 Changed 20 months ago by Matthias Köppe

It's really very easy:

  • You want to run our configure -> use one of our files which comes from configure.
  • You don't want to run configure -> write your sage_conf.py in which you hardcode your stuff.

comment:51 in reply to:  50 ; Changed 20 months ago by François Bissey

Replying to mkoeppe:

It's really very easy:

  • You don't want to run configure -> write your sage_conf.py in which you hardcode your stuff.

This is probably the best solution and it is also incredibly dirty once you get QA and multi-python support involved. In short this is cheap and easy if your packaging is somewhat ad hoc. Once you want this to go inside a framework with rules, there is a higher price than you may expect.

But we are getting incredibly off-topic and should stop polluting this ticket.

comment:52 in reply to:  51 Changed 20 months ago by Matthias Köppe

Yes, let's continue with this at another time and on another ticket.

comment:53 Changed 20 months ago by John Palmieri

% ./sage -t src/sage/misc/package.py 
Running doctests with ID 2021-04-12-22-39-35-af3c76ef.
Git branch: t/31577/move_sage_root_build_pkgs___src_to_sage_root____rename_sage_root_build_to_sage_root_sage_bootstrap
Using --optional=build,dochtml,homebrew,pip,sage,sage_spkg
Doctesting 1 file.
sage -t --warn-long 125.4 --random-seed=0 src/sage/misc/package.py
**********************************************************************
File "src/sage/misc/package.py", line 336, in sage.misc.package.installed_packages
Failed example:
    installed_packages()['sage_conf']  # optional - build
Expected:
    'none'
Got:
    '9.3.rc3'
**********************************************************************
1 item had failures:
   1 of   4 in sage.misc.package.installed_packages
    [50 tests, 1 failure, 2.86 s]
----------------------------------------------------------------------
sage -t --warn-long 125.4 --random-seed=0 src/sage/misc/package.py  # 1 doctest failed
----------------------------------------------------------------------

comment:54 Changed 20 months ago by git

Commit: 08b8306b9195facfc38bcfd067adeb0c7b064efa248fae72d2daf40e92acb48717f6b6d774c16781

Branch pushed to git repo; I updated commit sha1. New commits:

248fae7src/sage/misc/package.py: sage_conf now has a version

comment:55 Changed 20 months ago by Matthias Köppe

Thanks for testing!

comment:56 Changed 20 months ago by Matthias Köppe

Dependencies: #30913#30913, #31427

comment:57 Changed 20 months ago by git

Commit: 248fae72d2daf40e92acb48717f6b6d774c167819dac5c71b9f301f7746956b6bef532d5b1998133

Branch pushed to git repo; I updated commit sha1. New commits:

32c069a.gitignore: Integrate src/{doc,sage}/.gitignore here so that Docker ignores the listed files too
9dac5c7Merge #31427

comment:58 Changed 20 months ago by Matthias Köppe

Work issues: Fix up "tox -e docker..."

comment:59 Changed 20 months ago by John Palmieri

What are the files Pipfile, Pipfile.m4, etc. for? I just build Sage, and for example, sagemath-standard/Pipfile is currently a link pointing to a nonexistent target, and the same for sagemath-standard/Pipfile-dist.

comment:60 Changed 20 months ago by Matthias Köppe

If you run ./bootstrap, the targets of these symlinks should be generated.

These files were introduced in #30913. They are not used in a normal build. They are for users who want to use https://pypi.org/project/pipenv/ to set up a virtual environment.

comment:61 Changed 20 months ago by John Palmieri

I might prefer this in place of, or in addition to, the fixed installed_packages() doctest:

  • src/sage/misc/package.py

    diff --git a/src/sage/misc/package.py b/src/sage/misc/package.py
    index e402293baf..f9a3f9fc9e 100644
    a b def installed_packages(exclude_pip=True): 
    335335        '0.7.12'
    336336        sage: installed_packages()['sage_conf']  # optional - build, random
    337337        '9.3'
     338        sage: installed_packages()['sage_conf'] == sage.env.SAGE_VERSION # optional - build
     339        True
    338340
    339341    .. SEEALSO::
    340342

comment:62 Changed 20 months ago by Matthias Köppe

That's a good idea

comment:63 Changed 20 months ago by git

Commit: 9dac5c71b9f301f7746956b6bef532d5b199813358bee8e2bbe751574050aef865f9af50f426813e

Branch pushed to git repo; I updated commit sha1. New commits:

58bee8esrc/sage/misc/package.py: Better doctest for installed_packages

comment:64 Changed 20 months ago by git

Commit: 58bee8e2bbe751574050aef865f9af50f426813e16b54d18ec0c4a61c39ce9bd90be0b8d3346639f

Branch pushed to git repo; I updated commit sha1. New commits:

16b54d1build/pkgs/{sage_docbuild,sage_sws2rst}/package-version.txt: Symlink to ../sagelib/package-version.txt; do not symlink into src to make things easier for the Docker build

comment:65 Changed 20 months ago by Matthias Köppe

Unfortunately I have a bit of trouble to make this layout work well with the tox-docker build.

How about pushing the new directories down one level, to something like SAGE_ROOT/pkgs/sage-conf etc.?

comment:66 Changed 20 months ago by John Palmieri

Should the Sage source be in the same place? Or rather, should everything be in src? I don't know if there is any standard way to do this or any particular logic behind it.

comment:67 Changed 20 months ago by Matthias Köppe

I don't think there is a standard for this.

I initially had made it SAGE_ROOT/src/pkgs/sage-conf but I think it would be cleaner if src only contained the monolithic Sage library.

comment:68 Changed 20 months ago by John Palmieri

Then sure, SAGE_ROOT/pkgs/sage-conf sounds good.

comment:69 Changed 20 months ago by Matthias Köppe

Description: modified (diff)
Status: needs_reviewneeds_work
Summary: Move SAGE_ROOT/build/pkgs/*/src to SAGE_ROOT/*Move SAGE_ROOT/build/pkgs/*/src to SAGE_ROOT/pkgs/*
Work issues: Fix up "tox -e docker..."Make it SAGE_ROOT/pkgs/...

comment:70 Changed 20 months ago by Matthias Köppe

Description: modified (diff)

comment:71 Changed 20 months ago by Matthias Köppe

Description: modified (diff)

comment:72 Changed 19 months ago by Matthias Köppe

Dependencies: #30913, #31427#30913, #31427, #31825
Work issues: Make it SAGE_ROOT/pkgs/...Make it SAGE_ROOT/pkgs/..., merge #31825

comment:73 Changed 18 months ago by git

Commit: 16b54d18ec0c4a61c39ce9bd90be0b8d3346639f6ad0b8ff3b77b7d21dd95f74aa86c9aeddcbc6e8

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

9c3b887Move /build/pkgs/sage_docbuild/src to /sage_docbuild, leave symlink behind
1628441Move /build/pkgs/sage_sws2rst/src to /sage_sws2rst, leave symlink behind
a5d164bMove /build/pkgs/sagelib/src to /sagemath_standard, leave symlink behind
673de88.gitignore: Update for new source tree locations
2bdd5bdSpell distribution source directories sage-conf, sage-docbuild, sage-sws2rst, sagemath-standard with dashes, not underscores
e35a6d1Also standardize on using dashes in distribution names in setup.cfg
89c78e8WIP: Fix up docker build
fac294dbuild/pkgs/{sage_docbuild,sage_sws2rst}/package-version.txt: Symlink to ../sagelib/package-version.txt; do not symlink into src to make things easier for the Docker build
f0f5d54build/bin/write-dockerfile.sh: Adjust ADD commands
6ad0b8fWIP update .gitignore/.dockerignore

comment:74 Changed 18 months ago by Matthias Köppe

Dependencies: #30913, #31427, #31825

comment:75 Changed 18 months ago by git

Commit: 6ad0b8ff3b77b7d21dd95f74aa86c9aeddcbc6e8c58e51bdf3b979ccaddb9298a0ba039d006d7863

Branch pushed to git repo; I updated commit sha1. New commits:

c58e51bMove package source directories to pkgs/

comment:76 Changed 18 months ago by git

Commit: c58e51bdf3b979ccaddb9298a0ba039d006d78631d239235a017428a2eef7948de7ed31824371086

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

ce3c4d3Move /build/pkgs/sage_conf/src to /sage_conf, leave symlink behind
7824e0bAlso standardize on using dashes in distribution names in setup.cfg
49323d2build/pkgs/{sage_docbuild,sage_sws2rst}/package-version.txt: Symlink to ../sagelib/package-version.txt; do not symlink into src to make things easier for the Docker build
1d23923update .gitignore/.dockerignore; Move package source directories to pkgs/

comment:77 Changed 18 months ago by Matthias Köppe

Description: modified (diff)
Work issues: Make it SAGE_ROOT/pkgs/..., merge #31825

comment:78 Changed 18 months ago by git

Commit: 1d239235a017428a2eef7948de7ed318243710860cda8f17d5dd2ae80bc72024a121e6f265129943

Branch pushed to git repo; I updated commit sha1. New commits:

0cda8f1README: Update directory chart

comment:79 Changed 18 months ago by Matthias Köppe

Status: needs_workneeds_review

comment:80 Changed 18 months ago by git

Commit: 0cda8f17d5dd2ae80bc72024a121e6f265129943a7d6a3ed18a61c22b6cd65dc93b5d680701481d9

Branch pushed to git repo; I updated commit sha1. New commits:

380b42dUpdate paths in comments
a7d6a3ebuild/bin/write-dockerfile.sh: ADD pkgs

comment:81 Changed 18 months ago by Dima Pasechnik

Reviewers: Dima Pasechnik
Status: needs_reviewpositive_review

Looks good.

comment:82 Changed 18 months ago by Matthias Köppe

Thanks!

comment:83 Changed 18 months ago by Volker Braun

Status: positive_reviewneeds_work

Merge conflict

comment:84 Changed 18 months ago by Matthias Köppe

Status: needs_workpositive_review

Merges cleanly with af326a8c9a (vbraun/develop)

comment:85 Changed 18 months ago by Volker Braun

Status: positive_reviewneeds_work

then wait for the next beta

comment:86 Changed 17 months ago by git

Commit: a7d6a3ed18a61c22b6cd65dc93b5d680701481d97568dc6dc7a2325525aaa4618ec1bc0bc417917b

Branch pushed to git repo; I updated commit sha1. New commits:

7568dc6Merge tag '9.4.beta4' into t/31577/move_sage_root_build_pkgs___src_to_sage_root____rename_sage_root_build_to_sage_root_sage_bootstrap

comment:87 Changed 17 months ago by Matthias Köppe

Status: needs_workpositive_review

comment:88 Changed 17 months ago by Volker Braun

Status: positive_reviewneeds_work

whats going on here?

    STDOUT: CONFLICT (directory/file): There is a directory with name build/pkgs/sagelib/src in HEAD. Adding build/pkgs/sagelib/src as build/pkgs/sagelib/src~7568dc6dc7a2325525aaa4618ec1bc0bc417917b
    STDOUT: Removing build/pkgs/sage_sws2rst/src/sage_sws2rst/__init__.py
    STDOUT: Adding build/pkgs/sage_sws2rst/src
    STDOUT: Removing build/pkgs/sage_docbuild/src/sage_docbuild
    STDOUT: Removing build/pkgs/sage_docbuild/src/VERSION.txt
    STDOUT: CONFLICT (directory/file): There is a directory with name build/pkgs/sage_docbuild/src in HEAD. Adding build/pkgs/sage_docbuild/src as build/pkgs/sage_docbuild/src~7568dc6dc7a2325525aaa4618ec1bc0bc417917b
    STDOUT: Removing build/pkgs/sage_conf/src/bin/sage-env-config
    STDOUT: Removing build/pkgs/sage_conf/src/README.rst
    STDOUT: CONFLICT (directory/file): There is a directory with name build/pkgs/sage_conf/src in HEAD. Adding build/pkgs/sage_conf/src as build/pkgs/sage_conf/src~7568dc6dc7a2325525aaa4618ec1bc0bc417917b
    STDOUT: Auto-merging build/bin/write-dockerfile.sh
    STDOUT: Automatic merge failed; fix conflicts and then commit the result.
[release@zen Sage]$ git --version
git version 2.31.1

comment:89 Changed 17 months ago by Volker Braun

Status: needs_workpositive_review

ok those were previously not-checked in build artifacts

comment:90 Changed 17 months ago by Matthias Köppe

Description: modified (diff)

comment:91 Changed 17 months ago by Matthias Köppe

Description: modified (diff)

comment:92 Changed 17 months ago by Volker Braun

Branch: u/mkoeppe/move_sage_root_build_pkgs___src_to_sage_root____rename_sage_root_build_to_sage_root_sage_bootstrap7568dc6dc7a2325525aaa4618ec1bc0bc417917b
Resolution: fixed
Status: positive_reviewclosed
Note: See TracTickets for help on using tickets.