Opened 2 years ago

Last modified 3 months ago

#30151 needs_work enhancement

Modularization of sagelib: Break out a separate package sagemath-meataxe

Reported by: mkoeppe Owned by:
Priority: major Milestone: sage-9.7
Component: packages: optional Keywords: sd111
Cc: SimonKing, fbissey, jhpalmieri, dimpase Merged in:
Authors: Matthias Koeppe Reviewers:
Report Upstream: N/A Work issues: Rebase, rename "sage-..." -> "sagemath-..."
Branch: u/mkoeppe/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe (Commits, GitHub, GitLab) Commit: ebe1dd6c5721d24d7d496c7da0b632c1e12b9be4
Dependencies: #28925 Stopgaps:

Status badges

Description (last modified by mkoeppe)

... in analogy of #29864 (sage-tdlib).

To try: make sagemath_meataxe

(or ./configure --enable-sagemath_meataxe && make build)

This is part of #29705: Meta-ticket: Modularize sagelib into separate distributions (distutils packages)

Change History (39)

comment:1 Changed 21 months ago by mkoeppe

  • Branch set to u/mkoeppe/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe

comment:2 Changed 21 months ago by git

  • Commit set to b04074667037521ccc628dde3bf27203c3fd4312

Branch pushed to git repo; I updated commit sha1. New commits:

16c60casrc/setup.py: Remove meataxe - now taken care of by package sage_meataxe
b040746fixup

comment:3 Changed 21 months ago by SimonKing

Can you elaborate on what is supposed to be done to use meataxe --- both as a C library, as collection of executables, and as optional Cython extension module?

comment:4 follow-up: Changed 21 months ago by mkoeppe

spkg meataxe stays as is. It installs the C library and executables. No dependency on anything from Sage.

spkg sagelib will no longer conditionally build the Cython extension module.

Instead, the new spkg sage_meataxe will build the Cython extension module. It declares both sagelib and meataxe as a dependency.

sage_meataxe, in the implementation of this ticket, does not have a source tarball (note there is no checksums.ini). Instead, it is a "script" package, whose sources lie in build/pkgs/sage_meataxe/src. It has the structure of a Python package (distribution), with setup.py doing the build and installation. The actual Cython sources remain in the src/sage tree (using symlinks). (It is possible to build a pip-installable source distribution tarball as well, using the spkg-src script. Eventually, we will upload it to PyPI.)

comment:5 in reply to: ↑ 4 ; follow-up: Changed 21 months ago by SimonKing

Replying to mkoeppe:

spkg sagelib will no longer conditionally build the Cython extension module.

Instead, the new spkg sage_meataxe will build the Cython extension module. It declares both sagelib and meataxe as a dependency.

So, in order to use it, one has to do make sage_meataxe, right? It would automatically install its dependency meataxe, would build the corresponding Cython extension module, AND would also build the documentation of the Cython extension module? And sage -testall would run the doctests from the corresponding Cython extension module if and only if sage_meataxe is installed, so that there is no need to mark all its tests as "optional"?

If the documentation and test suite would be taken care of, it would indeed be very helpful.

comment:6 in reply to: ↑ 5 Changed 21 months ago by mkoeppe

Replying to SimonKing:

Replying to mkoeppe:

spkg sagelib will no longer conditionally build the Cython extension module.

Instead, the new spkg sage_meataxe will build the Cython extension module. It declares both sagelib and meataxe as a dependency.

So, in order to use it, one has to do make sage_meataxe, right? It would automatically install its dependency meataxe, would build the corresponding Cython extension module, AND would also build the documentation of the Cython extension module?

Yes.

And sage -testall would run the doctests from the corresponding Cython extension module if and only if sage_meataxe is installed, so that there is no need to mark all its tests as "optional"?

Yes, I would hope so. I have just created #30778 for this idea.

comment:7 Changed 21 months ago by git

  • Commit changed from b04074667037521ccc628dde3bf27203c3fd4312 to 1e03fef8e80260a729bf5942ef3d76065e4b666c

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

b060462sage_setup.find.find_python_sources: Handle native namespace packages, 'nonamespace' files
d42aa12is_package_or_namespace_package_dir: New, recognize namespace packages by file 'namespace', monkeypatch into Cython
29ca24bMake sage.graphs, sage.matrix, sage.libs, sage.interfaces namespace packages
1957acdsrc/sage_setup/clean.py: Update comment
8b37214WIP
92f1145sage_install_and_clean: If all_distributions is a list, do not clean files that do not belong to this list
44a4456src/sage_setup/command/sage_install.py: Also handle modules in namespace packages
706bf59Merge branch 't/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_' into t/30151/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe
f096870build/pkgs/sage_meataxe: New
1e03fefsrc/setup.py: Remove meataxe - now taken care of by package sage_meataxe

comment:8 Changed 21 months ago by git

  • Commit changed from 1e03fef8e80260a729bf5942ef3d76065e4b666c to aacec429b46ebdeb4322a39173996023f38d10a6

Branch pushed to git repo; I updated commit sha1. New commits:

ad9402fsage_setup.command.sage_install: Fixup: Remove cleaning code (moved to sage_install_and_clean)
88455dcMerge branch 't/30780/sage_setup__separate_installing_and_cleaning' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_
aacec42Merge branch 't/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_' into t/30151/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe

comment:9 Changed 21 months ago by git

  • Commit changed from aacec429b46ebdeb4322a39173996023f38d10a6 to ac1038915bdd1f228d5d44a425b8e30f011e3dbb

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

800f740Merge branch 't/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_' into t/30151/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe
3823197build/pkgs/sage_meataxe: New
ac10389build/pkgs/sagelib/src/setup.py: Remove meataxe - now taken care of by package sage_meataxe

comment:10 Changed 21 months ago by git

  • Commit changed from ac1038915bdd1f228d5d44a425b8e30f011e3dbb to 12b22d95ad1f020567f79a03d253ed50882f61b7

Branch pushed to git repo; I updated commit sha1. New commits:

12b22d9fixup

comment:11 Changed 21 months ago by git

  • Commit changed from 12b22d95ad1f020567f79a03d253ed50882f61b7 to 500635d3fa50483f9f57cac865b035b4b857ff44

Branch pushed to git repo; I updated commit sha1. New commits:

500635dfixup

comment:12 Changed 21 months ago by git

  • Commit changed from 500635d3fa50483f9f57cac865b035b4b857ff44 to 3d3cc4285ac4cd68adee2a0693ab867fbccd2ead

Branch pushed to git repo; I updated commit sha1. New commits:

3d3cc42build/pkgs/sage_meataxe/spkg-install: Install via bdist_wheel

comment:13 Changed 21 months ago by git

  • Commit changed from 3d3cc4285ac4cd68adee2a0693ab867fbccd2ead to 10e2432ccb7aca7410519fc4eb742c37aa3c3040

Branch pushed to git repo; I updated commit sha1. New commits:

99f3cc7build/pkgs/sage_meataxe/src/setup.py: Set built_distributions
a9933b6sage_build_cython: Pass built_distributions to find_extra_files
10e2432Merge branch 't/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_' into t/30151/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe

comment:14 Changed 21 months ago by mkoeppe

  • Authors set to Matthias Koeppe

The installation seems to work, but Sage cannot start because of a circular import related from sage.matrix.args. This needs a bit more work.

comment:15 Changed 21 months ago by mkoeppe

  • Description modified (diff)

comment:16 Changed 21 months ago by mkoeppe

  • Description modified (diff)

comment:17 Changed 21 months ago by SimonKing

  • Dependencies changed from #28925 to #28925 #28711

#28711 previously depended on this ticket. However, because of giving a talk on p_group_cohomology at the next Sage Days, I am interested in getting p_group_cohomology up and running ASAP.

Therefore, I suggest to make this ticket depend on #28711, rather than the other way around. As soon as the circular import problem is solved, one can modify SAGE_ROOT/build/pkgs/p_group_cohomology/dependencies (same for any other package depending on meataxe, if there is any); I believe this change logically belongs here, because it is THIS ticket that makes the change in the dependencies files possible and necessary.

comment:18 Changed 21 months ago by mkoeppe

No objection here. This ticket certainly won't make it into Sage 9.2. (Not sure if #28711 can make it - it looks like Volker does not intend to merge non-"blocker" tickets.)

comment:19 Changed 21 months ago by mkoeppe

  • Work issues set to Merge #28711 when ready

comment:20 Changed 21 months ago by git

  • Commit changed from 10e2432ccb7aca7410519fc4eb742c37aa3c3040 to 202914a5de774b08af0856c47a10694e3d9e2a87

Branch pushed to git repo; I updated commit sha1. New commits:

ea0d15asage.matrix: Resolve circular imports without using __init__.py
e4ce720Merge branch 't/30784/sage_matrix__resolve_circular_imports_without_using___init___py' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_
202914aMerge branch 't/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_' into t/30151/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe

comment:21 Changed 21 months ago by git

  • Commit changed from 202914a5de774b08af0856c47a10694e3d9e2a87 to 33e0eca8b24e62a259c375a2b246726a35b388a0

Branch pushed to git repo; I updated commit sha1. New commits:

33e0ecabuild/pkgs/sage_meataxe: Add metadata

comment:22 Changed 21 months ago by mkoeppe

  • Status changed from new to needs_review

Ready for review

comment:23 Changed 21 months ago by git

  • Commit changed from 33e0eca8b24e62a259c375a2b246726a35b388a0 to 98abe7ef8e05189b9b9f87ba7181c7adef3ce8bf

Branch pushed to git repo; I updated commit sha1. New commits:

6c408b5build/pkgs/sagelib/src/setup.py: Do not clean files that do not belong to us
6aaa958sage_install_and_clean: Move all_distributions into class
98abe7eMerge branch 't/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_' into t/30151/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe

comment:24 Changed 21 months ago by git

  • Commit changed from 98abe7ef8e05189b9b9f87ba7181c7adef3ce8bf to fb5637845624c447a51829bc4ab0765f7926a46d

Branch pushed to git repo; I updated commit sha1. New commits:

39be639FileDocTestSource.in_lib: Handle namespace packages
fb56378Merge branch 't/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_' into t/30151/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe

comment:25 Changed 21 months ago by git

  • Commit changed from fb5637845624c447a51829bc4ab0765f7926a46d to ebd5ae7ad1c3eef6af7ab69874c8a4aefb75b3d5

Branch pushed to git repo; I updated commit sha1. New commits:

64c5120find_python_sources: Also collect namespace packages - so that package_data works
ebd5ae7Merge branch 't/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_' into t/30151/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe

comment:26 Changed 21 months ago by git

  • Commit changed from ebd5ae7ad1c3eef6af7ab69874c8a4aefb75b3d5 to 6cbcbeca52ae309c24a429177ddefdeffa28408e

Branch pushed to git repo; I updated commit sha1. New commits:

4f5a810Upgrade p_group_cohomology to version 3.3.2
de8adfaRevert a change to "dependencies" of p_group_cohomology
cf60884Add upstream_url to p_group_cohomology
ae50157p_group_cohomology's upstream changed by some new commits
bd34c47build/pkgs/p_group_cohomology/checksums.ini: Fix upstream_url
245560dFix docbuild of p_group_cohomology
f4e5f6aFix a doctest in sage/tests/modular_cohomology.py
649ba8dMerge branch 'u/SimonKing/fix_spkg_install_of_p_group_cohomology' of git://trac.sagemath.org/sage into t/30151/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe
6cbcbecbuild/pkgs/sage_meataxe: Move dependencies here from build/pkgs/p_group_cohomology/dependencies, add sage_meataxe there

comment:27 Changed 21 months ago by git

  • Commit changed from 6cbcbeca52ae309c24a429177ddefdeffa28408e to b984eeb7f3a868f9c16ea97a9753238127e2f2cd

Branch pushed to git repo; I updated commit sha1. New commits:

b984eebbuild/pkgs/p_group_cohomology/spkg-install.in: Remove check for sage-meataxe modules

comment:28 Changed 21 months ago by mkoeppe

  • Cc dimpase added

comment:29 Changed 21 months ago by git

  • Commit changed from b984eeb7f3a868f9c16ea97a9753238127e2f2cd to 11637a68d0a5691329ec068a5f6a8a910438f5a0

Branch pushed to git repo; I updated commit sha1. New commits:

3c7d675src/sage_setup/find.py (find_python_sources): Update doc, doctests
11637a6Merge branch 't/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_' into t/30151/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe

comment:30 Changed 21 months ago by git

  • Commit changed from 11637a68d0a5691329ec068a5f6a8a910438f5a0 to 7cedb3d932a4591f28fae9dcbbf7aeb26ca98dd2

Branch pushed to git repo; I updated commit sha1. New commits:

7cedb3dbuild/pkgs/sagelib/src/setup.py: Fix up all_distributions

comment:31 Changed 19 months ago by mkoeppe

  • Keywords sd111 added

comment:32 Changed 19 months ago by mkoeppe

  • Dependencies changed from #28925 #28711 to #28925
  • Work issues changed from Merge #28711 when ready to Rebase

comment:33 Changed 19 months ago by mkoeppe

  • Status changed from needs_review to needs_work

comment:34 Changed 19 months ago by mkoeppe

  • Description modified (diff)
  • Summary changed from Modularization of sagelib: Break out a separate package sage-meataxe to Modularization of sagelib: Break out a separate package sagemath-meataxe
  • Work issues changed from Rebase to Rebase, rename "sage-..." -> "sagemath-..."

comment:35 Changed 19 months ago by git

  • Commit changed from 7cedb3d932a4591f28fae9dcbbf7aeb26ca98dd2 to ebe1dd6c5721d24d7d496c7da0b632c1e12b9be4

Branch pushed to git repo; I updated commit sha1. New commits:

a6e8447Merge tag '9.3.beta3' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_
ebe1dd6Merge branch 't/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_' into t/30151/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe

comment:36 Changed 17 months ago by mkoeppe

  • Milestone changed from sage-9.3 to sage-9.4

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

comment:37 Changed 12 months ago by mkoeppe

  • Milestone changed from sage-9.4 to sage-9.5

Setting a new milestone for this ticket based on a cursory review.

comment:38 Changed 7 months ago by mkoeppe

  • Milestone changed from sage-9.5 to sage-9.6

comment:39 Changed 3 months ago by mkoeppe

  • Milestone changed from sage-9.6 to sage-9.7
Note: See TracTickets for help on using tickets.