Opened 3 years ago
Closed 5 weeks ago
#28925 closed enhancement (fixed)
Modify find_python_sources to support modularization of sagelib by native namespace packages (PEP 420)
Reported by: | mkoeppe | Owned by: | |
---|---|---|---|
Priority: | critical | Milestone: | sage-9.7 |
Component: | build | Keywords: | sd111 |
Cc: | isuruf, embray, jdemeyer, dimpase, dcoudert, vdelecroix, vbraun, gh-tobiasdiez, fbissey | Merged in: | |
Authors: | Matthias Koeppe | Reviewers: | Kwankyu Lee, Dima Pasechnik |
Report Upstream: | N/A | Work issues: | |
Branch: | f9df1ae (Commits, GitHub, GitLab) | Commit: | f9df1ae6eef3dd955d060ca8005b9ed3f5af3974 |
Dependencies: | #33803 | Stopgaps: |
Description (last modified by )
This ticket prepares the modularization of sagelib into separate distributions (pip-installable packages) by adding support for native namespace packages (PEP 420) to sagelib's build system.
Ordinary packages (directories of modules) can be turned into namespace packages by removing the empty __init__.py
file.
Then several distribution package can share the same namespace.
Because our source discovery mechanism in sagelib's build system (setup.py
+ sage_setup
) distinguishes package directories from other directories containing data files by the presence of the __init__.py
file, the present ticket makes several adjustments to the build system - in particular to the functions find_python_sources
and find_extra_files
.
Per convention in the Sage library, namespace packages are recognized by the presence of all.py
or all__*.py
files (#33033).
In this ticket, we apply it to turn sagemath-objects and sagemath-categories (from #29865) into namespace packages,
in which __init__.py
are removed from the MANIFEST.in
files.
Because the actual files are not removed from the source tree, the normal build of the Sage distribution is not affected.
To test these two distributions:
./bootstrap && ./sage -sh -c '(cd pkgs/sagemath-objects && SAGE_NUM_THREADS=16 tox -r -v -v -v -e py39)' ./bootstrap && ./sage -sh -c '(cd pkgs/sagemath-categories && SAGE_NUM_THREADS=16 tox -r -v -v -v -e py39)'
Follow-up tickets:
- #33011 Remove the
__init__.py
files for packages designated to be namespace packages - Wishlist item: Warn or give errors if the
sage_setup: distribution
directives,namespace
andnonamespace
files indicate an inconsistent structure such as when it was forgotten to remove__init__.py
. - #30152 will make further changes to the cleaner that will allow out-of-tree namespace packages
See also:
- #29705: Meta-ticket: Modularize sagelib into separate distutils packages
- Initial discussion of namespace packages for Sage in #28175 and https://github.com/mkoeppe/sage-numerical-backends-coin/pull/4
- #21654:
src/setup.py
: Disentangle cleaning of stale installed files in build directory and in install directory
Change History (173)
comment:1 Changed 2 years ago by
- Milestone changed from sage-9.1 to sage-9.2
comment:2 Changed 2 years ago by
- Cc dimpase dcoudert added
- Description modified (diff)
comment:3 Changed 2 years ago by
- Description modified (diff)
comment:4 Changed 2 years ago by
- Description modified (diff)
- Summary changed from Modify clean_stale_files to support modularization of sagelib by namespace packages to Modify clean_stale_files to support modularization of sagelib by native namespace packages (PEP 420)
comment:5 Changed 2 years ago by
- Description modified (diff)
comment:6 Changed 2 years ago by
- Description modified (diff)
- Summary changed from Modify clean_stale_files to support modularization of sagelib by native namespace packages (PEP 420) to Modify find_python_sources, clean_stale_files to support modularization of sagelib by native namespace packages (PEP 420)
comment:7 Changed 2 years ago by
- Cc vdelecroix added
- Description modified (diff)
comment:8 Changed 2 years ago by
- Cc vbraun added
comment:9 Changed 2 years ago by
- Description modified (diff)
comment:10 Changed 2 years ago by
- Branch set to u/mkoeppe/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_
comment:11 Changed 2 years ago by
- Commit set to 8bc56623abb2579ca435b99f324a9d5eef89c857
New commits:
8e50bbe | sage.graphs.graph_decompositions: Make it a namespace package by removing __init__.py (which had unnecessary reimports only)
|
dff6e62 | Mark src/sage/ext_data as directories that are not namespace packages
|
8bc5662 | sage_setup.find.find_python_sources: Handle native namespace packages and 'nonamespace' files
|
comment:12 Changed 2 years ago by
- Description modified (diff)
comment:13 Changed 2 years ago by
- Description modified (diff)
comment:14 Changed 2 years ago by
- Description modified (diff)
comment:15 Changed 2 years ago by
- Description modified (diff)
comment:16 Changed 2 years ago by
- Commit changed from 8bc56623abb2579ca435b99f324a9d5eef89c857 to 606a509168111b4cf7306fa283e1d5f0cccd41d0
Branch pushed to git repo; I updated commit sha1. New commits:
606a509 | sage.numerical.backends: Make it a namespace package by removing __init__.py (which was empty)
|
comment:17 Changed 2 years ago by
- Description modified (diff)
comment:18 Changed 2 years ago by
Next steps: adjust the cleaner, find Cython sources
comment:19 Changed 2 years ago by
comment:20 Changed 2 years ago by
- Commit changed from 606a509168111b4cf7306fa283e1d5f0cccd41d0 to a7b2287a8b69e814781b20434849e788d00884ca
Branch pushed to git repo; I updated commit sha1. New commits:
a7b2287 | find_python_sources: New optional argunment sage_packages, filter by # distutils: sage_package = PACKAGE
|
comment:21 Changed 2 years ago by
- Description modified (diff)
comment:22 Changed 2 years ago by
- Commit changed from a7b2287a8b69e814781b20434849e788d00884ca to 194d3b430666c3bb72885a945274589ea54bd4be
Branch pushed to git repo; I updated commit sha1. New commits:
194d3b4 | sage_setup.find.find_python_sources: Also find Cython modules
|
comment:23 Changed 2 years ago by
- Commit changed from 194d3b430666c3bb72885a945274589ea54bd4be to ad3ebad7af2f4f1eea0f4a771dc029382ce60d96
comment:24 Changed 2 years ago by
- Description modified (diff)
comment:25 Changed 2 years ago by
- Commit changed from ad3ebad7af2f4f1eea0f4a771dc029382ce60d96 to 004dd5240961beb9e2c4fab328d77688df57a862
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
004dd52 | Merge tag '9.2.beta0' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_
|
comment:26 Changed 2 years ago by
- Commit changed from 004dd5240961beb9e2c4fab328d77688df57a862 to 3a68b37aba6be39b55a404874f2f66dc1ed4b0eb
Branch pushed to git repo; I updated commit sha1. New commits:
3a68b37 | src/sage_setup/find.py: Switch to parsing directive 'sage_setup: distribution = PKG'
|
comment:27 Changed 2 years ago by
- Commit changed from 3a68b37aba6be39b55a404874f2f66dc1ed4b0eb to d900139267b5aeceaa399ee6ffa190e4fd821748
Branch pushed to git repo; I updated commit sha1. New commits:
d900139 | src/module_list.py: Remove sage.graphs.graph_decompositions.* from explicit Cython module list
|
comment:28 Changed 2 years ago by
- Description modified (diff)
comment:29 Changed 2 years ago by
- Commit changed from d900139267b5aeceaa399ee6ffa190e4fd821748 to 6d71e74ac3a6ed2808e927664c593d70f11d9869
Branch pushed to git repo; I updated commit sha1. New commits:
6d71e74 | Add doctest
|
comment:30 Changed 2 years ago by
- Description modified (diff)
comment:31 Changed 2 years ago by
- Commit changed from 6d71e74ac3a6ed2808e927664c593d70f11d9869 to f670f81d1db80ab3334292b7646bc4fd034b32b0
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
0a7fb48 | Move src/fpickle_setup.py to srs/sage_setup/
|
f4a9269 | src/setup.py: Move class sage_install to new module sage_setup.command.sage_install
|
5660d8f | src/module_list.py: Move library_order to new module sage_setup.library_order
|
43d4f83 | src/setup.py: Move classes sage_build_* to new modules sage_setup.command.sage_*
|
09e3297 | src/setup.py: Move excepthook to new module sage_setup.excepthook
|
1cfed7c | build/pkgs/sagelib/spkg-install: Do not poison SAGE_LOCAL, used in script installed by spkg pkgconf
|
62c7531 | Merge branch 't/29411/make_sagelib_a_script_package' into t/29702/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup
|
b3102e9 | Merge tag '9.2.beta0' into t/29702/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup
|
5066f83 | Merge branch 't/29702/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_
|
f670f81 | WIP
|
comment:32 Changed 2 years ago by
Next: Fix find_extra_files
so that it finds the .pxd
files in sage.numerical.backends
comment:33 Changed 2 years ago by
- Dependencies set to #29702
comment:34 Changed 2 years ago by
- Commit changed from f670f81d1db80ab3334292b7646bc4fd034b32b0 to bd858270985630f780a0d367c2b4b1581c755404
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
bd85827 | is_package_or_namespace_package_dir: New, monkeypatch into Cython.Build.Dependencies
|
comment:35 Changed 2 years ago by
- Commit changed from bd858270985630f780a0d367c2b4b1581c755404 to 532bb21b12f289e579c2dc033dbc41dab47c4330
Branch pushed to git repo; I updated commit sha1. New commits:
532bb21 | is_package_or_namespace_package_dir: Make self-contained, patch into more of Cython
|
comment:36 Changed 2 years ago by
- Commit changed from 532bb21b12f289e579c2dc033dbc41dab47c4330 to 0aa36935e2656eeb0b8139f98d7352afed8221cb
Branch pushed to git repo; I updated commit sha1. New commits:
2b42029 | src/sage_setup/command/__init__.py: New
|
a830304 | src/sage_setup/command/sage_build_cython.py: Fix adding setup.py as dependency
|
d3080a5 | Merge branch 't/29702/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_
|
0aa3693 | Mark src as nonamespace
|
comment:37 Changed 2 years ago by
- Commit changed from 0aa36935e2656eeb0b8139f98d7352afed8221cb to 47cb57898d62e3d71ad18a4b27ea0a99ee3a982e
Branch pushed to git repo; I updated commit sha1. New commits:
47cb578 | is_package_or_namespace_package_dir: Recognize namespace packages by file 'namespace'
|
comment:38 Changed 2 years ago by
- Commit changed from 47cb57898d62e3d71ad18a4b27ea0a99ee3a982e to 4c51133f6e3458abf3f9303d49303e49b1b1cbb8
comment:39 Changed 2 years ago by
- Commit changed from 4c51133f6e3458abf3f9303d49303e49b1b1cbb8 to c0a2ed74a42cf8b575fa2cf26228e513bd69dfbd
Branch pushed to git repo; I updated commit sha1. New commits:
c0a2ed7 | src/module_list.py: Remove explicit listing of sage.graphs.graph_decompositions extensions
|
comment:40 Changed 2 years ago by
- Description modified (diff)
comment:41 Changed 2 years ago by
- Description modified (diff)
comment:42 Changed 2 years ago by
- Work issues set to rebase on top of #29701
comment:43 Changed 2 years ago by
- Milestone changed from sage-9.2 to sage-9.3
comment:44 Changed 2 years ago by
- Dependencies changed from #29702 to #29702, #29701
comment:45 Changed 2 years ago by
- Description modified (diff)
comment:46 Changed 2 years ago by
- Commit changed from c0a2ed74a42cf8b575fa2cf26228e513bd69dfbd to 40bc47392ab35b9a7e4059bb46a455828d5d7673
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
a43bf07 | sage.numerical.backends: Make it a namespace package by removing __init__.py (which was empty)
|
7a6d12f | sage_setup.find.find_python_sources: Handle native namespace packages, 'nonamespace' files, Cython modules, directive 'sage_setup: distribution = PKG'
|
f4a6c7d | src/module_list.py: Remove sage.graphs.graph_decompositions.* from explicit Cython module list
|
cee98e7 | Add doctest
|
7873513 | is_package_or_namespace_package_dir: New, monkeypatch into Cython
|
d01495e | Mark src as nonamespace
|
899dc32 | is_package_or_namespace_package_dir: Recognize namespace packages by file 'namespace'
|
b97b1e0 | src/module_list.py: Remove explicit list of sage.numerical.backends extensions
|
4b8181b | sage_setup/command/sage_install.py: WIP: Disable cleaning
|
40bc473 | src/module_list.py: Remove explicit listing of sage.graphs.graph_decompositions extensions
|
comment:47 Changed 2 years ago by
Squashed some commits
comment:48 Changed 2 years ago by
- Commit changed from 40bc47392ab35b9a7e4059bb46a455828d5d7673 to 975b4a0d1504887dafac780eb66120b58f204440
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
226aa5a | sage.numerical.backends: Make it a namespace package by removing __init__.py (which was empty)
|
906fb23 | Mark src and src/sage/ext_data as directories that are not namespace packages
|
4a2eee0 | sage_setup.find.find_python_sources: Handle native namespace packages, 'nonamespace' files, Cython modules, directive 'sage_setup: distribution = PKG'
|
a8b0316 | is_package_or_namespace_package_dir: New, recognize namespace packages by file 'namespace', monkeypatch into Cython
|
92bd8ae | src/module_list.py: Remove sage.graphs.graph_decompositions.* from explicit Cython module list
|
6647d09 | src/module_list.py: Remove explicit list of sage.numerical.backends extensions
|
975b4a0 | sage_setup/command/sage_install.py: WIP: Disable cleaning
|
comment:49 Changed 2 years ago by
- Commit changed from 975b4a0d1504887dafac780eb66120b58f204440 to 650c661e50e10f57da6f121942158f654de835ec
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
5d5803e | src/sage/graphs/planarity.pyx: Add forgotten distutils directive
|
a0be9b6 | Merge branch 't/29790/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_5__sage_graphs_' into t/29701/replace_use_of_module_list_optionalextension
|
d0eb84e | sage.graphs.graph_decompositions: Make it a namespace package by removing __init__.py (which had unnecessary reimports only)
|
c82fb01 | sage.numerical.backends: Make it a namespace package by removing __init__.py (which was empty)
|
d90e3b6 | Mark src and src/sage/ext_data as directories that are not namespace packages
|
f598e04 | sage_setup.find.find_python_sources: Handle native namespace packages, 'nonamespace' files
|
f522b09 | is_package_or_namespace_package_dir: New, recognize namespace packages by file 'namespace', monkeypatch into Cython
|
50df6d5 | src/module_list.py: Remove sage.graphs.graph_decompositions.* from explicit Cython module list
|
e34c028 | src/module_list.py: Remove explicit list of sage.numerical.backends extensions
|
650c661 | sage_setup/command/sage_install.py: WIP: Disable cleaning
|
comment:50 Changed 2 years ago by
- Work issues rebase on top of #29701 deleted
Rebased on top of #29701
comment:51 Changed 2 years ago by
- Dependencies changed from #29702, #29701 to #29701
comment:52 Changed 2 years ago by
- Description modified (diff)
- Work issues set to move cleaning modifications to #30152, break sage_install into several classes for the different use cases
comment:53 Changed 2 years ago by
- Commit changed from 650c661e50e10f57da6f121942158f654de835ec to 474b2007d3b68cbf14d10feb1cc3b2750bf0c02e
Branch pushed to git repo; I updated commit sha1. New commits:
14396af | Merge tag '9.2.beta2' into t/29701/replace_use_of_module_list_optionalextension
|
d62da15 | sage_setup.command.sage_build: Add the extensions to the distribution
|
06a3609 | sage_setup.find.find_python_sources: Add benchmark doctest
|
174626c | Merge branch 't/29786/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_4__sage_rings_' into t/29701/replace_use_of_module_list_optionalextension
|
034a7f7 | Merge branch 'public/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup' of git://trac.sagemath.org/sage into t/29701/replace_use_of_module_list_optionalextension
|
d3c608b | Merge tag '9.2.beta3' into t/29701/replace_use_of_module_list_optionalextension
|
d99e942 | Merge branch 't/29701/replace_use_of_module_list_optionalextension' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_
|
474b200 | Merge tag '9.2.beta5' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_
|
comment:54 Changed 2 years ago by
- Description modified (diff)
comment:55 Changed 2 years ago by
- Commit changed from 474b2007d3b68cbf14d10feb1cc3b2750bf0c02e to 2e79d33db38da4da9a87561f534f6b6ca813904a
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
eaadf77 | sage.graphs.graph_decompositions: Make it a namespace package by removing __init__.py (which had unnecessary reimports only)
|
59151ab | sage.numerical.backends: Make it a namespace package by removing __init__.py (which was empty)
|
713a838 | Mark src and src/sage/ext_data as directories that are not namespace packages
|
4faff43 | sage_setup.find.find_python_sources: Handle native namespace packages, 'nonamespace' files
|
de17c1c | is_package_or_namespace_package_dir: New, recognize namespace packages by file 'namespace', monkeypatch into Cython
|
6761b7d | src/module_list.py: Remove explicit list of sage.numerical.backends extensions
|
65f3739 | sage_setup/command/sage_install.py: WIP: Disable cleaning
|
8c20f3e | Revert "sage_setup/command/sage_install.py: WIP: Disable cleaning"
|
37bd1ca | sage_setup.command.sage_install.sage_install: Rename to sage_install_and_clean, update use it in src/setup.py; refactor through new class sage_install
|
2e79d33 | WIP
|
comment:56 Changed 2 years ago by
- Commit changed from 2e79d33db38da4da9a87561f534f6b6ca813904a to 98a1b883f8cab1fc5d3ec829a937acfa148b6347
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
8a19fe2 | build/make/Makefile.in (sagelib-clean): Clean the new build location
|
ccc67b0 | src/sage_setup: Update cythonized_dir in doctests
|
df38027 | Merge branch 't/29950/build_sagelib_using_installed_sage_setup' into t/30277/remove_src_module_list_py
|
ef4e554 | Merge branch 't/30277/remove_src_module_list_py' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_
|
6282b8d | sage.graphs.graph_decompositions: Make it a namespace package by removing __init__.py (which had unnecessary reimports only)
|
7c10680 | sage.numerical.backends: Make it a namespace package by removing __init__.py (which was empty)
|
6e68259 | Mark src and src/sage/ext_data as directories that are not namespace packages
|
b71c3a6 | sage_setup.find.find_python_sources: Handle native namespace packages, 'nonamespace' files
|
3a12146 | is_package_or_namespace_package_dir: New, recognize namespace packages by file 'namespace', monkeypatch into Cython
|
98a1b88 | sage_setup.command.sage_install.sage_install: Rename to sage_install_and_clean, update use it in src/setup.py; refactor through new class sage_install
|
comment:57 Changed 2 years ago by
- Dependencies changed from #29701 to #30277
- Work issues move cleaning modifications to #30152, break sage_install into several classes for the different use cases deleted
comment:58 Changed 2 years ago by
- Description modified (diff)
- Summary changed from Modify find_python_sources, clean_stale_files to support modularization of sagelib by native namespace packages (PEP 420) to Modify find_python_sources to support modularization of sagelib by native namespace packages (PEP 420)
comment:59 Changed 2 years ago by
- Commit changed from 98a1b883f8cab1fc5d3ec829a937acfa148b6347 to 742ab8777c8010a2e3d6435428c8baa7acdea128
comment:60 Changed 2 years ago by
- Work issues set to installed_files_by_module
comment:61 Changed 2 years ago by
- Description modified (diff)
comment:62 Changed 2 years ago by
- Cc gh-tobiasdiez added
comment:63 Changed 2 years ago by
- Commit changed from 742ab8777c8010a2e3d6435428c8baa7acdea128 to a967aca4491de98118612f30cbdf8bd75ab4efa9
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
776c979 | sage.graphs.graph_decompositions: Make it a namespace package by removing __init__.py (which had unnecessary reimports only)
|
2e61ada | sage.numerical.backends: Make it a namespace package by removing __init__.py (which was empty)
|
26b1ca0 | Mark src and src/sage/ext_data as directories that are not namespace packages
|
a733af5 | sage_setup.find.find_python_sources: Handle native namespace packages, 'nonamespace' files
|
c08c511 | is_package_or_namespace_package_dir: New, recognize namespace packages by file 'namespace', monkeypatch into Cython
|
4407e5e | sage_setup.command.sage_install.sage_install: Rename to sage_install_and_clean, update use it in src/setup.py; refactor through new class sage_install
|
a13c126 | Make sage.graphs, sage.matrix, sage.libs, sage.interfaces namespace packages
|
a967aca | WIP
|
comment:64 Changed 2 years ago by
- Dependencies #30277 deleted
comment:65 Changed 2 years ago by
- Dependencies set to #30578
comment:66 Changed 2 years ago by
- Work issues changed from installed_files_by_module to installed_files_by_module; rebase on top of #30578
comment:67 Changed 22 months ago by
- Dependencies changed from #30578 to #30780
- Work issues changed from installed_files_by_module; rebase on top of #30578 to installed_files_by_module; rebase on top of #30780
comment:68 Changed 22 months ago by
- Commit changed from a967aca4491de98118612f30cbdf8bd75ab4efa9 to a4e5683b1ecf6121bc766bf60bd90d312371b9cf
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
7980130 | Merge branch 't/30779/duplicate_src_setup_py' into t/30780/sage_setup__separate_installing_and_cleaning
|
9b95628 | sage_setup.command.sage_install.sage_install: Rename to sage_install_and_clean, update use it in build/pkgs/sagelib/src/setup.py; refactor through new class sage_install
|
886ef6d | Merge branch 't/30780/sage_setup__separate_installing_and_cleaning' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_
|
3b9559d | sage.graphs.graph_decompositions: Make it a namespace package by removing __init__.py (which had unnecessary reimports only)
|
88085b1 | sage.numerical.backends: Make it a namespace package by removing __init__.py (which was empty)
|
9c39a76 | Mark src and src/sage/ext_data as directories that are not namespace packages
|
b060462 | sage_setup.find.find_python_sources: Handle native namespace packages, 'nonamespace' files
|
fe9a7d5 | is_package_or_namespace_package_dir: New, recognize namespace packages by file 'namespace', monkeypatch into Cython
|
d2fe380 | Make sage.graphs, sage.matrix, sage.libs, sage.interfaces namespace packages
|
a4e5683 | WIP
|
comment:69 Changed 22 months ago by
- Work issues changed from installed_files_by_module; rebase on top of #30780 to installed_files_by_module
comment:70 Changed 22 months ago by
- Commit changed from a4e5683b1ecf6121bc766bf60bd90d312371b9cf to 8b37214c8c3bad2a3e046798f3b6ada33580dd31
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
d42aa12 | is_package_or_namespace_package_dir: New, recognize namespace packages by file 'namespace', monkeypatch into Cython
|
29ca24b | Make sage.graphs, sage.matrix, sage.libs, sage.interfaces namespace packages
|
1957acd | src/sage_setup/clean.py: Update comment
|
8b37214 | WIP
|
comment:71 Changed 22 months ago by
- Commit changed from 8b37214c8c3bad2a3e046798f3b6ada33580dd31 to 44a44560071ce851fce1e415000b13bc0cda3d90
comment:72 Changed 22 months ago by
- Work issues installed_files_by_module deleted
comment:73 Changed 22 months ago by
- Status changed from new to needs_review
comment:74 Changed 22 months ago by
- Commit changed from 44a44560071ce851fce1e415000b13bc0cda3d90 to 88455dc42bfe6639c6f8dfc91f3d2fca3c51fb3e
Branch pushed to git repo; I updated commit sha1. New commits:
ad9402f | sage_setup.command.sage_install: Fixup: Remove cleaning code (moved to sage_install_and_clean)
|
88455dc | Merge branch 't/30780/sage_setup__separate_installing_and_cleaning' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_
|
comment:75 Changed 22 months ago by
- Commit changed from 88455dc42bfe6639c6f8dfc91f3d2fca3c51fb3e to a9933b6b1282992d49976ce0ec586aa97e1d4c26
Branch pushed to git repo; I updated commit sha1. New commits:
a9933b6 | sage_build_cython: Pass built_distributions to find_extra_files
|
comment:76 Changed 22 months ago by
- Dependencies changed from #30780 to #30780, #30784
comment:77 Changed 22 months ago by
- Commit changed from a9933b6b1282992d49976ce0ec586aa97e1d4c26 to e4ce7207371e0d27eb1747fbe03855af108beb57
Branch pushed to git repo; I updated commit sha1. New commits:
ea0d15a | sage.matrix: Resolve circular imports without using __init__.py
|
e4ce720 | Merge branch 't/30784/sage_matrix__resolve_circular_imports_without_using___init___py' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_
|
comment:78 Changed 22 months ago by
- Description modified (diff)
comment:79 Changed 22 months ago by
- Commit changed from e4ce7207371e0d27eb1747fbe03855af108beb57 to 6aaa95887be0f4f5257208e6662cd3ca821bc0bc
Branch pushed to git repo; I updated commit sha1. New commits:
6aaa958 | sage_install_and_clean: Move all_distributions into class
|
comment:80 Changed 22 months ago by
Ready for review
comment:81 Changed 22 months ago by
- Commit changed from 6aaa95887be0f4f5257208e6662cd3ca821bc0bc to 39be6390197b4c39382420e27653c83dd99a3503
Branch pushed to git repo; I updated commit sha1. New commits:
39be639 | FileDocTestSource.in_lib: Handle namespace packages
|
comment:82 Changed 22 months ago by
- Commit changed from 39be6390197b4c39382420e27653c83dd99a3503 to 64c5120b0a03c5a907cb56f63bf9df360976bd44
Branch pushed to git repo; I updated commit sha1. New commits:
64c5120 | find_python_sources: Also collect namespace packages - so that package_data works
|
comment:83 Changed 22 months ago by
- Description modified (diff)
comment:84 follow-up: ↓ 85 Changed 22 months ago by
What's the advantage/reason to introduce the sage-specific namespace/nonamespace convention instead of following PEP 420 and keep the __init__.py
file as a marker for namespace packages together with setuptools.find_namespace_packages()
?
comment:85 in reply to: ↑ 84 Changed 22 months ago by
Replying to gh-tobiasdiez:
What's the advantage/reason to introduce the sage-specific namespace/nonamespace convention instead of following PEP 420 and keep the
__init__.py
file as a marker for namespace packages together withsetuptools.find_namespace_packages()
?
One cannot keep __init__.py
for native namespace packages. PEP 420 is quite clear about it. Sub-packages of namespace packages do have __init__.py
; and also the present ticket keeps them.
comment:86 Changed 22 months ago by
find_namespace_packages
accepts any directory as a package directory:
class PEP420PackageFinder(PackageFinder): @staticmethod def _looks_like_package(path): return True
But our source tree has some directories that should not be considered package directories. I have marked these with the nonamespace
file.
Having both nonamespace
and namespace
files results in simpler changes to the existing code.
comment:87 follow-up: ↓ 89 Changed 22 months ago by
Oh ok, sorry, I've misunderstood the ticket description. You are of course right, implicit namespace packages don't have a __init__.py
file.
Concerning the package discovery, I would still prefer using setuptools.find_namespace_packages()
, and then filter out unwanted folders based on a hard-coded list. The (no)namespace
files just add complexity (especially for new developers). They are also very similar to the explicit declaration of namespace packages proposed by PEP 382, which was rejected in favor of PEP 420.
If you do prefer to have these files, I would like to ask you to add documentation to the dev docs.
comment:88 Changed 22 months ago by
- Commit changed from 64c5120b0a03c5a907cb56f63bf9df360976bd44 to 3c7d6751b4bdc7c419ecc94a2add7b1da21be212
Branch pushed to git repo; I updated commit sha1. New commits:
3c7d675 | src/sage_setup/find.py (find_python_sources): Update doc, doctests
|
comment:89 in reply to: ↑ 87 ; follow-up: ↓ 115 Changed 22 months ago by
Replying to gh-tobiasdiez:
Concerning the package discovery, I would still prefer using
setuptools.find_namespace_packages()
, and then filter out unwanted folders based on a hard-coded list. The(no)namespace
files just add complexity (especially for new developers). They are also very similar to the explicit declaration of namespace packages proposed by PEP 382, which was rejected in favor of PEP 420.
Thanks for the input and the pointer to PEP 382. I'll try out if your suggestion can be implemented without major changes.
comment:90 Changed 22 months ago by
- Status changed from needs_review to needs_work
TBD: In incremental builds, the cleaner fails to remove previously installed __init__.py
files:
********************************************************************** File "src/sage_setup/find.py", line 343, in sage_setup.find.installed_files_by_module Failed example: files_by_module['sage.graphs.graph_decompositions'] Expected nothing Got: {'sage/graphs/graph_decompositions/__init__.py', 'sage/graphs/graph_decompositions/__pycache__/__init__.cpython-37.pyc'} **********************************************************************
comment:91 Changed 22 months ago by
- Description modified (diff)
comment:92 Changed 22 months ago by
Note that one annoying thing about find_namespace_packages
when applied to our source tree is that we have lots of directories that do not contain any python files, leading to output like this from distutils:
[sagelib-9.2.rc2] package init file 'sage/schemes/generic/notes/__init__.py' not found (or not a regular file)
The branch on this ticket handles such directories specially (no need for nonamespace
files there). If there is a solution that could silence these notes, the code could be brought closer to find_namespace_packages
behavior.
comment:93 Changed 22 months ago by
find_namespace_packages
excepts include
and exclude
directories as arguments. So one option would be to get a list of all folders not including any python files, and pass this as exclude
(or the same approach using a whitelist).
comment:94 Changed 21 months ago by
- Keywords sd111 added
comment:95 Changed 20 months ago by
- Commit changed from 3c7d6751b4bdc7c419ecc94a2add7b1da21be212 to a6e8447748b8a2b4107c43a92daa2bcef30ff27b
Branch pushed to git repo; I updated commit sha1. New commits:
a6e8447 | Merge tag '9.3.beta3' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_
|
comment:96 Changed 20 months ago by
- Dependencies #30780, #30784 deleted
- Work issues set to comment 90
comment:97 Changed 18 months ago by
- Milestone changed from sage-9.3 to sage-9.4
Setting new milestone based on a cursory review of ticket status, priority, and last modification date.
comment:98 Changed 13 months ago by
- Milestone changed from sage-9.4 to sage-9.5
Setting a new milestone for this ticket based on a cursory review.
comment:99 Changed 9 months ago by
- Commit changed from a6e8447748b8a2b4107c43a92daa2bcef30ff27b to f017b3f879f77e70425ac65174982ccf9044baf4
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
f017b3f | #28925 Modify find_python_sources to support modularization (squashed)
|
comment:100 Changed 9 months ago by
- Commit changed from f017b3f879f77e70425ac65174982ccf9044baf4 to bfaeb1488e1048c3e51a898bd6eddbbb8d9a7e13
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
bfaeb14 | 28925: Modify find_python_sources to support modularization (squashed)
|
comment:101 Changed 9 months ago by
- Dependencies set to #32927
comment:102 Changed 9 months ago by
- Commit changed from bfaeb1488e1048c3e51a898bd6eddbbb8d9a7e13 to 15813aa5f45d8cecf80ce12bbb1966ece00cace9
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
c6fc111 | Prettier diagram
|
7d6a44c | Use :mod: as markup for packages/modules
|
f83d424 | Improve ABC example
|
817a8d0 | src/doc/en/developer/packaging_sage_library.rst: More :mod: and :class: markup
|
b555735 | src/doc/en/developer/packaging_sage_library.rst: Link to pypi.org and to documentation of packaging metadata
|
81e9c9a | Merge #32899
|
ecc0886 | Merge #31386
|
00d5f59 | build/pkgs/sagelib: Invoke install cleaner explicitly, remove it from pkgs/sagemath-standard/setup.py
|
a5aab69 | Merge #32927
|
15813aa | Remove __init__.py files
|
comment:103 Changed 8 months ago by
- Dependencies changed from #32927 to #32927, #29865
comment:104 Changed 8 months ago by
- Description modified (diff)
comment:105 Changed 8 months ago by
- Commit changed from 15813aa5f45d8cecf80ce12bbb1966ece00cace9 to 864874173a842a21f1087fab2367f4bec18bd351
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
a72ee3b | src/sage/all__sagemath_objects.py: Add some more imports
|
f3ee49b | src/sage/all__sagemath_objects.py: Add definitions of Integer, RealNumber for doctesting
|
2a4a323 | Merge tag '9.5.beta7' into t/29039/pip_installable_package_sage_bootstrap
|
9ef0567 | Merge #29039
|
e0399a1 | pkgs/sagemath-objects: Install a fixed set of scripts, not all of src/bin
|
7519048 | src/bin/sage[-env]: Put SAGE_ROOT/src/bin in front of path only if run out of this directory
|
00024e6 | Merge #32933
|
bfc953b | src/sage/doctest/reporting.py: Indicate --environment if not default
|
36d6b09 | pkgs/sagemath-categories/setup.cfg.m4: Add scripts
|
8648741 | Merge #29865
|
comment:106 Changed 8 months ago by
- Commit changed from 864874173a842a21f1087fab2367f4bec18bd351 to fcfdf1de2b481d149f2d9b17cff862f997132a43
Branch pushed to git repo; I updated commit sha1. New commits:
fcfdf1d | Restore __init__.py files for #28925
|
comment:107 Changed 8 months ago by
- Commit changed from fcfdf1de2b481d149f2d9b17cff862f997132a43 to 31b9c16578aec755cad21567fa9c5812c1d85a53
Branch pushed to git repo; I updated commit sha1. New commits:
31b9c16 | src/sage_setup/command/sage_install.py (sage_clean): Do not fail if dist.py_modules is None
|
comment:108 Changed 8 months ago by
- Commit changed from 31b9c16578aec755cad21567fa9c5812c1d85a53 to 03909ae71dc00824e1e4a67f468b20a0553169d3
Branch pushed to git repo; I updated commit sha1. New commits:
f5fd073 | pkgs/sagemath-objects/MANIFEST.in: Remove __init__.py files
|
67d78e7 | pkgs/sagemath-categories/MANIFEST.in.m4: Remove src/sage/categories/__init__.py
|
4912f15 | src/sage/env.py: Get SAGE_LIB via sage.env's __file__
|
03909ae | src/sage/env.py (SAGE_LIB): Fix up
|
comment:109 Changed 8 months ago by
- Milestone changed from sage-9.5 to sage-9.6
comment:110 Changed 8 months ago by
- Dependencies changed from #32927, #29865 to #32927, #29865, #33033
comment:111 Changed 8 months ago by
- Commit changed from 03909ae71dc00824e1e4a67f468b20a0553169d3 to c7f871899760c1d9c750c712a74d667b16c37e02
Branch pushed to git repo; I updated commit sha1. New commits:
8c8c29c | WIP
|
c3f6bcb | src/sage/misc/namespace_package.py (is_package_or_sage_namespace_package_dir): New
|
ab23a14 | src/sage/doctest/sources.py: Use is_package_or_sage_namespace_package_dir
|
0f05856 | src/sage/doctest/sources.py: Remove unnecessary conversion to bool
|
d8aa37c | src/sage/misc/namespace_package.py: In doctests, use 'directory' instead of the single-letter variable name
|
2cf8d8d | Merge #33033
|
c7f8718 | src/sage_setup/find.py: Use is_package_or_sage_namespace_package_dir
|
comment:112 Changed 8 months ago by
- Commit changed from c7f871899760c1d9c750c712a74d667b16c37e02 to f3764bc2c73149b9a369027fc0c001f8243918f9
Branch pushed to git repo; I updated commit sha1. New commits:
f3764bc | src/sage_setup/find.py: Get rid of use of 'nonamespace' files
|
comment:113 Changed 8 months ago by
- Commit changed from f3764bc2c73149b9a369027fc0c001f8243918f9 to 6089ae892c72a41033524c137a6e6cecc853b4b6
comment:114 Changed 8 months ago by
- Description modified (diff)
comment:115 in reply to: ↑ 89 ; follow-up: ↓ 122 Changed 8 months ago by
Replying to mkoeppe:
Replying to gh-tobiasdiez:
Concerning the package discovery, I would still prefer using
setuptools.find_namespace_packages()
, and then filter out unwanted folders based on a hard-coded list. The(no)namespace
files just add complexity (especially for new developers). They are also very similar to the explicit declaration of namespace packages proposed by PEP 382, which was rejected in favor of PEP 420.Thanks for the input and the pointer to PEP 382. I'll try out if your suggestion can be implemented without major changes.
Done now via #33033.
comment:116 Changed 8 months ago by
- Work issues comment 90 deleted
comment:117 Changed 8 months ago by
- Description modified (diff)
- Status changed from needs_work to needs_review
comment:118 Changed 8 months ago by
- Commit changed from 6089ae892c72a41033524c137a6e6cecc853b4b6 to 37a957915d1bac88f011cd185f400b9483a455b6
comment:119 Changed 8 months ago by
- Commit changed from 37a957915d1bac88f011cd185f400b9483a455b6 to 2f6d47334df875115f6d65611b83a00af21314d5
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
55e258d | pkgs/sagemath-categories/MANIFEST.in.m4: Remove src/sage/categories/__init__.py
|
07ffddb | src/sage/env.py: Get SAGE_LIB via sage.env's __file__
|
320f2e0 | src/sage/env.py (SAGE_LIB): Fix up
|
85ef02e | WIP
|
95a7205 | src/sage_setup/find.py: Use is_package_or_sage_namespace_package_dir
|
6ab1bd7 | src/sage_setup/find.py: Get rid of use of 'nonamespace' files
|
0b33605 | src/sage_setup/find.py: Add doctest result
|
2e995dd | src/sage/{arith,ext,libs,rings,sets}/all__sagemath_objects.py
|
f244b57 | pkgs/sagemath-objects/MANIFEST.in: Exclude generated files
|
2f6d473 | pkgs/sagemath-categories/MANIFEST.in.m4: Exclude generated files
|
comment:120 Changed 8 months ago by
- Dependencies changed from #32927, #29865, #33033 to #29865, #33033
- Description modified (diff)
Rebased away from #32927.
comment:121 Changed 8 months ago by
- Commit changed from 2f6d47334df875115f6d65611b83a00af21314d5 to f5addb34e5dc66d461a757df1f9620dbe5184230
comment:122 in reply to: ↑ 115 ; follow-ups: ↓ 123 ↓ 124 Changed 8 months ago by
Replying to mkoeppe:
Replying to mkoeppe:
Replying to gh-tobiasdiez:
Concerning the package discovery, I would still prefer using
setuptools.find_namespace_packages()
, and then filter out unwanted folders based on a hard-coded list. The(no)namespace
files just add complexity (especially for new developers). They are also very similar to the explicit declaration of namespace packages proposed by PEP 382, which was rejected in favor of PEP 420.Thanks for the input and the pointer to PEP 382. I'll try out if your suggestion can be implemented without major changes.
Done now via #33033.
Probably I'm missing something here. The new mechanism seems to still rely on indicator/marker files, just that the are now named all__
. Except for the different name this seems to be exactly the approach of the rejected PEP 382.
The point of my proposal was that only the setup file should know/decide on which packages it want to include and exclude. Then the general helper in #33033 would like
is_namespace_package(dir): return dir in setuptools.find_namespace_packages(include=..., exclude=...)
And the setup.cfg would use https://setuptools.pypa.io/en/latest/userguide/package_discovery.html.
comment:123 in reply to: ↑ 122 Changed 8 months ago by
Replying to gh-tobiasdiez:
The new mechanism seems to still rely on indicator/marker files, just that the are now named
all__
. Except for the different name this seems to be exactly the approach of the rejected PEP 382.
That's certainly true, but at least the Sage source tree already has these files
comment:124 in reply to: ↑ 122 Changed 8 months ago by
Replying to gh-tobiasdiez:
The point of my proposal was that only the setup file should know/decide on which packages it want to include and exclude. Then the general helper in #33033 would like
is_namespace_package(dir): return dir in setuptools.find_namespace_packages(include=..., exclude=...)
The helper function is added in #33033 for runtime use (doctesting). Doctesting needs to be able to work on (a) Sage source trees, (b) installed Sage packages (this is important, for example, for the downstream packagers), (c) arbitrary files. Hardcoding inclusions and exclusions in the helper function seems a bit unmodular
comment:125 Changed 8 months ago by
In the end it comes down to who should have the control:
- Should the doctest code decide what to test, and the setup.py/cfg file what to distribute and the sage startup script what to expose
- or should the module/package be responsible to declare that it should be tested, distributed and exposed
Currently, this ticket (and the related ones) go in the direction of the second approach by using all.py
. For me, using inversion of control and following the first approach seems more natural and flexible. Since you have a better overview though, I trust your judgement.
comment:126 Changed 7 months ago by
- Commit changed from f5addb34e5dc66d461a757df1f9620dbe5184230 to 178d111960e751d5d34bf3097ddd39c65f24c42d
comment:127 Changed 6 months ago by
- Dependencies #29865, #33033 deleted
comment:128 Changed 6 months ago by
- Commit changed from 178d111960e751d5d34bf3097ddd39c65f24c42d to 5c873e0611693ecf766945a76ea7e60af0507636
Branch pushed to git repo; I updated commit sha1. New commits:
5c873e0 | Merge tag '9.6.beta1' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_
|
comment:129 Changed 6 months ago by
- Commit changed from 5c873e0611693ecf766945a76ea7e60af0507636 to 5546c6358845073440ea2b92a7e6b8f1cfc8c76b
comment:130 Changed 6 months ago by
All dependencies are merged
comment:131 Changed 6 months ago by
- Commit changed from 5546c6358845073440ea2b92a7e6b8f1cfc8c76b to f961a7d372809b232620c0224928e8341a09e99a
Branch pushed to git repo; I updated commit sha1. New commits:
f961a7d | src/setup.py: Support namespace packages in Cython 0.x
|
comment:132 Changed 6 months ago by
- Priority changed from major to critical
comment:133 Changed 5 months ago by
- Commit changed from f961a7d372809b232620c0224928e8341a09e99a to d2b529398016451fbf4766ecc2df3829a0efc1bb
Branch pushed to git repo; I updated commit sha1. New commits:
d2b5293 | Merge tag '9.6.beta5' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_
|
comment:134 Changed 5 months ago by
- Commit changed from d2b529398016451fbf4766ecc2df3829a0efc1bb to aac736c54ecd074b3d653f9c658e9fb9f29fd011
Branch pushed to git repo; I updated commit sha1. New commits:
aac736c | build/pkgs/sage_setup/dependencies: Add all sage_setup source files
|
comment:135 Changed 4 months ago by
- Milestone changed from sage-9.6 to sage-9.7
comment:136 Changed 4 months ago by
- Commit changed from aac736c54ecd074b3d653f9c658e9fb9f29fd011 to 6258b79f1a3aa980bbc73d3257b404b320f133ec
Branch pushed to git repo; I updated commit sha1. New commits:
6258b79 | Merge tag '9.6.rc2' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_
|
comment:137 Changed 3 months ago by
- Dependencies set to #33803
comment:138 Changed 3 months ago by
- Commit changed from 6258b79f1a3aa980bbc73d3257b404b320f133ec to e8a98b13d16e29812321562983a34b7edb97973b
Branch pushed to git repo; I updated commit sha1. New commits:
37f5cb5 | .github/workflows/sdist.yml: Build wheels too
|
b273985 | sage.env.cython_aliases: Handle ecl via default_required_modules
|
6ac30fd | pkgs/sagemath-objects/MANIFEST.in: Add missing sage.misc.package_dir
|
762dd70 | pkgs/sagemath-objects/MANIFEST.in: Add sage.misc.sagedoc
|
33a1540 | pkgs/sagemath-{objects,categories}/README.rst: Mention dependencies
|
e8a98b1 | Merge #33803
|
comment:139 Changed 3 months ago by
- Commit changed from e8a98b13d16e29812321562983a34b7edb97973b to 06eec85b4f5fe564d56e3317ef9c22425903cc6e
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
32c831c | sage.env.cython_aliases: Handle ecl via default_required_modules
|
4d75ff7 | pkgs/sagemath-objects/MANIFEST.in: Add missing sage.misc.package_dir
|
078d72d | pkgs/sagemath-objects/MANIFEST.in: Add sage.misc.sagedoc
|
4da43c5 | pkgs/sagemath-{objects,categories}/README.rst: Mention dependencies
|
06eec85 | Merge #33803
|
comment:140 Changed 3 months ago by
- Commit changed from 06eec85b4f5fe564d56e3317ef9c22425903cc6e to 4e95de9b16dd4b1629847e9ac7837ea291bb9cb5
comment:141 Changed 3 months ago by
The failure in src/sage/interfaces/expect.py
shown by the patchbot is unrelated.
Ready for review
comment:142 Changed 3 months ago by
should one exclude
not only *.so, but *.dylib, too?
comment:143 Changed 3 months ago by
On macOS, the compiled python modules are also .so, not .dylib.
comment:144 Changed 3 months ago by
- Cc fbissey added
comment:145 Changed 3 months ago by
- Commit changed from 4e95de9b16dd4b1629847e9ac7837ea291bb9cb5 to a92db463d340bff6819a10ca05c9c9ec1c2b95ad
Branch pushed to git repo; I updated commit sha1. New commits:
a92db46 | Merge tag '9.7.beta1' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_
|
comment:146 Changed 2 months ago by
- Commit changed from a92db463d340bff6819a10ca05c9c9ec1c2b95ad to 79822e41a32907476352becb078e10f921c0eba9
Branch pushed to git repo; I updated commit sha1. New commits:
a15f173 | pkgs/sagemath-categories/tox.ini: Fix instructions in comment
|
8587d47 | src/sage/misc/package_dir.py: New context manager cython_namespace_package_support
|
bc0992d | src/sage/misc/cython.py: Activate cython_namespace_package_support
|
7c98c36 | src/setup.py: Fix up use of cython_namespace_package_support
|
79822e4 | pkgs/sagemath-objects/setup.py: Monkey-patching cython is now done by the sage_build_cython command
|
comment:147 Changed 2 months ago by
- Commit changed from 79822e41a32907476352becb078e10f921c0eba9 to 39272c1c5c8e33072f29a80baea95af40f7f6ad1
Branch pushed to git repo; I updated commit sha1. New commits:
39272c1 | src/sage_setup/command/sage_build_cython.py: Reduce verbosity
|
comment:148 Changed 2 months ago by
- Commit changed from 39272c1c5c8e33072f29a80baea95af40f7f6ad1 to 81237e15f8a7c035486811f51e365664b84ecae9
Branch pushed to git repo; I updated commit sha1. New commits:
81237e1 | Merge tag '9.7.beta2' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_
|
comment:149 Changed 8 weeks ago by
Sorry for my ignorance, but I am unable to merge this ticket:
% git clone -c core.symlinks=true --branch develop --tags https://github.com/sagemath/sage.git [...] % cd sage % git-trac-merge 28925 #28925 remote branch: u/mkoeppe/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_ From git://trac.sagemath.org/sage * branch u/mkoeppe/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_ -> FETCH_HEAD * [new branch] u/mkoeppe/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_ -> trac/u/mkoeppe/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_ Auto-merging build/pkgs/sage_setup/dependencies CONFLICT (content): Merge conflict in build/pkgs/sage_setup/dependencies Automatic merge failed; fix conflicts and then commit the result. %
comment:150 Changed 7 weeks ago by
- Commit changed from 81237e15f8a7c035486811f51e365664b84ecae9 to dd2c525c3c191b9c72fda24e0cf99f9654ed7dfe
Branch pushed to git repo; I updated commit sha1. New commits:
dd2c525 | Merge tag '9.7.beta3' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_
|
comment:151 Changed 7 weeks ago by
I've resolved the merge conflict
comment:152 Changed 6 weeks ago by
Not used import?
-
pkgs/sagemath-objects/setup.py
diff --git a/pkgs/sagemath-objects/setup.py b/pkgs/sagemath-objects/setup.py index 124f0cb218..907cabef83 100644
a b if sdist: 42 42 python_modules = [] 43 43 cython_modules = [] 44 44 else: 45 from sage_setup.find import find_python_sources , is_package_or_namespace_package_dir45 from sage_setup.find import find_python_sources 46 46 python_packages, python_modules, cython_modules = find_python_sources( 47 47 '.', ['sage']) # for now, we do the filtering using MANIFEST
comment:153 Changed 6 weeks ago by
- Commit changed from dd2c525c3c191b9c72fda24e0cf99f9654ed7dfe to cd4d8e9aa0c9954019fef9dc29e43d26a891f460
comment:154 Changed 6 weeks ago by
Thanks!
And I have applied the change that left that import unused also to another file.
comment:155 Changed 6 weeks ago by
Thanks.
Another comment: clean_install_dir
and find_extra_files
have a new argument distributions
, but the docstrings do not explain it.
comment:156 Changed 6 weeks ago by
- Commit changed from cd4d8e9aa0c9954019fef9dc29e43d26a891f460 to 9ab3200f0c30e6502f915a08136f636371a0e995
comment:157 Changed 6 weeks ago by
- Reviewers set to Kwankyu Lee, ...
Thank you for the update.
I have been testing (using) this ticket and and the sequel #33011 for a while. They works well. I checked the code as far as I can (not very far), and it looks good to me.
If we don't have further comments on this ticket, then it would better be tested by more developers in the wild by merging this now, so that we can fix problems before the next release.
I will wait for your (not Matthias) meta comments, and proceed to "positive review" in a couple of days.
comment:158 follow-up: ↓ 159 Changed 6 weeks ago by
Is this
./sage -sh -c '(cd pkgs/sagemath-categories && tox -v -v -v)'
broken? It doesn't work with (and without) this ticket.
comment:159 in reply to: ↑ 158 ; follow-ups: ↓ 160 ↓ 165 Changed 6 weeks ago by
Replying to klee:
Is this
./sage -sh -c '(cd pkgs/sagemath-categories && tox -v -v -v)'broken? It doesn't work with (and without) this ticket.
works for me (after make build
from a clean state done)
comment:160 in reply to: ↑ 159 ; follow-up: ↓ 166 Changed 6 weeks ago by
Replying to dimpase:
Replying to klee:
Is this
./sage -sh -c '(cd pkgs/sagemath-categories && tox -v -v -v)'broken? It doesn't work with (and without) this ticket.
works for me (after
make build
from a clean state done)
I did make distclean; make -j8; ./sage -sh -c '(cd pkgs/sagemath-categories && tox -v -v -v)'
. So this is clean.
This is the initial part that goes wrong:
sage -t --random-seed=251639358214975434990568781743714015030 --environment=sage.all__sagemath_categories /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories/.tox/python/lib/python3.10/site-packages/sage/structure/coerce_actions.pyx ********************************************************************** File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories/.tox/python/lib/python3.10/site-packages/sage/structure/coerce_actions.pyx", line 58, in sage.structure.coerce_actions.GenericAction.__init__ Failed example: M = MatrixSpace(ZZ,2) Exception raised: Traceback (most recent call last): File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories/.tox/python/lib/python3.10/site-packages/sage/doctest/forker.py", line 695, in _run self.compile_and_execute(example, compiler, test.globs) File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories/.tox/python/lib/python3.10/site-packages/sage/doctest/forker.py", line 1093, in compile_and_execute exec(compiled, globs) File "<doctest sage.structure.coerce_actions.GenericAction.__init__[0]>", line 1, in <module> M = MatrixSpace(ZZ,Integer(2)) NameError: name 'MatrixSpace' is not defined
Any clue? I am on Mac.
comment:161 follow-up: ↓ 162 Changed 6 weeks ago by
- Reviewers changed from Kwankyu Lee, ... to Kwankyu Lee, Dima Pasechnik
- Status changed from needs_review to positive_review
I start from git clean -fdx
- which cleans much more.
You'll probably want to move upstream/
outside and after git clean
make a symbolic link to it, otherwise it will re-download things.
So it's positive review from me.
comment:162 in reply to: ↑ 161 ; follow-up: ↓ 167 Changed 5 weeks ago by
Replying to dimpase:
I start from
git clean -fdx
- which cleans much more.
Thanks. But it didn't help. As far as I understand, the doctest failures are expected. The real problem is:
$ pkgs/sagemath-categories/.tox/py39/bin/python3.9 Python 3.9.13 (main, May 24 2022, 21:28:31) [Clang 13.1.6 (clang-1316.0.21.2)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from sage.categories.category import Category Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories/.tox/py39/lib/python3.9/site-packages/sage/categories/category.py", line 107, in <module> from sage.misc.c3_controlled import _cmp_key, _cmp_key_named, C3_sorted_merge File "sage/misc/c3_controlled.pyx", line 364, in init sage.misc.c3_controlled (build/cythonized/sage/misc/c3_controlled.c:11819) File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories/.tox/py39/lib/python3.9/site-packages/sage/structure/__init__.py", line 2, in <module> import sage.structure.element File "sage/structure/element.pyx", line 1, in init sage.structure.element (build/cythonized/sage/structure/element.c:36204) File "sage/structure/category_object.pyx", line 60, in init sage.structure.category_object (build/cythonized/sage/structure/category_object.c:9982) ImportError: cannot import name Category
Strangely this works
Hera:sage-dev$ pkgs/sagemath-categories/.tox/py39/bin/python3.9 Python 3.9.13 (main, May 24 2022, 21:28:31) [Clang 13.1.6 (clang-1316.0.21.2)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from sage.structure.sage_object import SageObject >>> from sage.categories.category import Category >>>
Anyway, this is not a problem of this ticket. The situation is the same without this ticket.
Hence my positive review to this ticket!
comment:163 follow-up: ↓ 164 Changed 5 weeks ago by
hmm, do you have the current directory, i.e. .
, in your PATH
? IMHO this might mess things up.
comment:164 in reply to: ↑ 163 Changed 5 weeks ago by
Replying to dimpase:
hmm, do you have the current directory, i.e.
.
, in yourPATH
? IMHO this might mess things up.
Yes. But even after removing it from the path, it's the same.
./configure
shows python3-3.10.3: using system package; SPKG will not be installed
, and
$ pkgs/sagemath-categories/.tox/py39/bin/python3.9 Python 3.9.13 (main, May 24 2022, 21:28:31) [Clang 13.1.6 (clang-1316.0.21.2)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.path ['', '/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python39.zip', '/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9', '/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload', '/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories/.tox/py39/lib/python3.9/site-packages']
$ pkgs/sagemath-categories/.tox/python/bin/python Python 3.10.4 (main, Apr 26 2022, 19:42:59) [Clang 13.1.6 (clang-1316.0.21.2)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.path ['', '/usr/local/Cellar/python@3.10/3.10.4/Frameworks/Python.framework/Versions/3.10/lib/python310.zip', '/usr/local/Cellar/python@3.10/3.10.4/Frameworks/Python.framework/Versions/3.10/lib/python3.10', '/usr/local/Cellar/python@3.10/3.10.4/Frameworks/Python.framework/Versions/3.10/lib/python3.10/lib-dynload', '/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories/.tox/python/lib/python3.10/site-packages']
comment:165 in reply to: ↑ 159 Changed 5 weeks ago by
Replying to dimpase:
Replying to klee:
Is this
./sage -sh -c '(cd pkgs/sagemath-categories && tox -v -v -v)'broken? It doesn't work with (and without) this ticket.
works for me (after
make build
from a clean state done)
sagemath-categories (in contrast to sagemath-standard) does not need make build
because all of its build and runtime dependencies can be found on PyPI.
comment:166 in reply to: ↑ 160 ; follow-up: ↓ 169 Changed 5 weeks ago by
Replying to klee:
This is the initial part that goes wrong:
sage -t --random-seed=251639358214975434990568781743714015030 --environment=sage.all__sagemath_categories /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories/.tox/python/lib/python3.10/site-packages/sage/structure/coerce_actions.pyx ********************************************************************** File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories/.tox/python/lib/python3.10/site-packages/sage/structure/coerce_actions.pyx", line 58, in sage.structure.coerce_actions.GenericAction.__init__ Failed example: M = MatrixSpace(ZZ,2) Exception raised: Traceback (most recent call last): File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories/.tox/python/lib/python3.10/site-packages/sage/doctest/forker.py", line 695, in _run self.compile_and_execute(example, compiler, test.globs) File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories/.tox/python/lib/python3.10/site-packages/sage/doctest/forker.py", line 1093, in compile_and_execute exec(compiled, globs) File "<doctest sage.structure.coerce_actions.GenericAction.__init__[0]>", line 1, in <module> M = MatrixSpace(ZZ,Integer(2)) NameError: name 'MatrixSpace' is not defined
sagemath-categories as of this ticket does not contain src/sage/matrix/matrix_space.py (see pkgs/sagemath-categories/MANIFEST.in
), so MatrixSpace
is not available in sage.all__sagemath_categories
.
Note that the tox
run ends with the message "lots of doctest failures are expected". This is one of them.
comment:167 in reply to: ↑ 162 Changed 5 weeks ago by
Replying to klee:
The real problem is:
$ pkgs/sagemath-categories/.tox/py39/bin/python3.9 Python 3.9.13 (main, May 24 2022, 21:28:31) [Clang 13.1.6 (clang-1316.0.21.2)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from sage.categories.category import Category Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories/.tox/py39/lib/python3.9/site-packages/sage/categories/category.py", line 107, in <module> from sage.misc.c3_controlled import _cmp_key, _cmp_key_named, C3_sorted_merge File "sage/misc/c3_controlled.pyx", line 364, in init sage.misc.c3_controlled (build/cythonized/sage/misc/c3_controlled.c:11819) File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories/.tox/py39/lib/python3.9/site-packages/sage/structure/__init__.py", line 2, in <module> import sage.structure.element File "sage/structure/element.pyx", line 1, in init sage.structure.element (build/cythonized/sage/structure/element.c:36204) File "sage/structure/category_object.pyx", line 60, in init sage.structure.category_object (build/cythonized/sage/structure/category_object.c:9982) ImportError: cannot import name CategoryStrangely this works
Hera:sage-dev$ pkgs/sagemath-categories/.tox/py39/bin/python3.9 Python 3.9.13 (main, May 24 2022, 21:28:31) [Clang 13.1.6 (clang-1316.0.21.2)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from sage.structure.sage_object import SageObject >>> from sage.categories.category import Category >>>
The Sage library contains many cyclic imports. If one starts at the wrong place in an import cycle, one can run into problems with partially initialized modules. See #33580 (Meta-ticket: make all modules importable without sage.all)
comment:168 Changed 5 weeks ago by
Thanks both for the review!
comment:169 in reply to: ↑ 166 Changed 5 weeks ago by
Replying to mkoeppe:
Note that the
tox
run ends with the message "lots of doctest failures are expected".
Yes, it's there. It would perhaps better be colored red or green...
comment:170 Changed 5 weeks ago by
- Status changed from positive_review to needs_work
Merge failure on top of:
843eb03e7e Updated SageMath version to 9.7.beta4
merge was not clean: conflicts in pkgs/sagemath-categories/tox.ini
comment:171 Changed 5 weeks ago by
- Commit changed from 9ab3200f0c30e6502f915a08136f636371a0e995 to f9df1ae6eef3dd955d060ca8005b9ed3f5af3974
Branch pushed to git repo; I updated commit sha1. New commits:
f9df1ae | Merge tag '9.7.beta4' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_
|
comment:172 Changed 5 weeks ago by
- Status changed from needs_work to positive_review
comment:173 Changed 5 weeks ago by
- Branch changed from u/mkoeppe/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_ to f9df1ae6eef3dd955d060ca8005b9ed3f5af3974
- Resolution set to fixed
- Status changed from positive_review to closed
pushing these forward to 9.2