Opened 2 years ago
Last modified 8 months ago
#29854 new enhancement
Install sage_conf using flit instead of setuptools
Reported by:  Matthias Köppe  Owned by:  

Priority:  major  Milestone:  sagewishlist 
Component:  build  Keywords:  
Cc:  Merged in:  
Authors:  Reviewers:  
Report Upstream:  N/A  Work issues:  
Branch:  u/mkoeppe/install_sage_conf_using_flit (Commits, GitHub, GitLab)  Commit:  7ea5fabac7e4f2485a89e93f2e5931c0f9639935 
Dependencies:  #29846  Stopgaps: 
Description (last modified by )
So far this is just an illustration of what can be done with flit (#29846) and what cannot:
 Both nonsrc and src layouts are supported (for the latter, see https://github.com/takluyver/flit/pull/260).
 Symlinks of directories or files to somewhere outside of the root are not supported.
Not intended for merging at this point.
Change History (18)
comment:1 Changed 2 years ago by
Branch:  → u/mkoeppe/install_sage_conf_using_flit 

comment:2 Changed 2 years ago by
Commit:  → fd83992029e0eacadd60f2d5bf6c53150c8ce46c 

Work issues:  → Rebase so it is not on top of #29850 
comment:3 Changed 2 years ago by
This branch does not work because it is trying to use a symlink for sage_conf.py into the SAGE_ROOT/src directory.
[sage_confnone] Processing /Users/mkoeppe/s/sage/sagerebasing/build/pkgs/sage_conf/src [sage_confnone] Created temporary directory: /private/var/folders/38/wnh4gf1552g_crsjnv2vmmww0000gp/T/pipreqbuildw7fopzwn [sage_confnone] Added file:///Users/mkoeppe/s/sage/sagerebasing/build/pkgs/sage_conf/src to build tracker '/private/var/folders/38/wnh4gf1552g_crsjnv2vmmww0000gp/T/pipreqtrackerjmerw27d' [sage_confnone] Created temporary directory: /private/var/folders/38/wnh4gf1552g_crsjnv2vmmww0000gp/T/pipmodernmetadatadxzef0bb [sage_confnone] Preparing wheel metadata: started [sage_confnone] Running command /Users/mkoeppe/s/sage/sagerebasing/local/bin/python3 /Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.7/sitepackages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /var/folders/38/wnh4gf1552g_crsjnv2vmmww0000gp/T/tmpb7arkmg5 [sage_confnone] Traceback (most recent call last): [sage_confnone] File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.7/sitepackages/pip/_vendor/pep517/_in_process.py", line 280, in <module> [sage_confnone] main() [sage_confnone] File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.7/sitepackages/pip/_vendor/pep517/_in_process.py", line 263, in main [sage_confnone] json_out['return_val'] = hook(**hook_input['kwargs']) [sage_confnone] File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.7/sitepackages/pip/_vendor/pep517/_in_process.py", line 133, in prepare_metadata_for_build_wheel [sage_confnone] return hook(metadata_directory, config_settings) [sage_confnone] File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.7/sitepackages/flit_core/buildapi.py", line 28, in prepare_metadata_for_build_wheel [sage_confnone] module = Module(ini_info.module, os.getcwd()) [sage_confnone] File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.7/sitepackages/flit_core/common.py", line 55, in __init__ [sage_confnone] raise ValueError("No file/folder found for module {}".format(name)) [sage_confnone] ValueError: No file/folder found for module sage_conf [sage_confnone] Preparing wheel metadata: finished with status 'error' [sage_confnone] ERROR: Command errored out with exit status 1: /Users/mkoeppe/s/sage/sagerebasing/local/bin/python3 /Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.7/sitepackages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /var/folders/38/wnh4gf1552g_crsjnv2vmmww0000gp/T/tmpb7arkmg5 Check the logs for full command output.
comment:4 Changed 2 years ago by
Commit:  fd83992029e0eacadd60f2d5bf6c53150c8ce46c → e537d48dec34f383c3b66ec48f24b09cc142ae94 

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
e537d48  build/pkgs/sage_conf: Install with flit instead of setuptools

comment:5 Changed 2 years ago by
Dependencies:  → #29846 

Work issues:  Rebase so it is not on top of #29850 
comment:7 Changed 2 years ago by
Commit:  e537d48dec34f383c3b66ec48f24b09cc142ae94 → 4d20d4a7c1ed3403644cdd45c7e0afcc4b8c8f2d 

comment:8 Changed 2 years ago by
Description:  modified (diff) 

Summary:  Install sage_conf using flit → Install sage_conf using flit instead of setuptools 
comment:9 Changed 2 years ago by
Description:  modified (diff) 

comment:10 Changed 2 years ago by
With the version in 27cf73b, also:
flit build Fetching list of valid trove classifiers Iflit.validate Version number normalised: '9.2.beta0' > '9.2b0' (see PEP 440) Wflit_core.versionno Found 4 files tracked in git Iflit.sdist Traceback (most recent call last): File "/Users/mkoeppe/s/sage/sagerebasing/local/bin/flit", line 8, in <module> sys.exit(main()) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.7/sitepackages/flit/__init__.py", line 171, in main gen_setup_py=args.setup_py) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.7/sitepackages/flit/build.py", line 46, in main sdist_file = sb.build(dist_dir, gen_setup_py=gen_setup_py) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.7/sitepackages/flit/sdist.py", line 223, in build return Path(super().build(str(target_dir), gen_setup_py=gen_setup_py)) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.7/sitepackages/flit_core/sdist.py", line 181, in build files_to_add = self.apply_includes_excludes(self.select_files()) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.7/sitepackages/flit_core/sdist.py", line 155, in apply_includes_excludes .format(", ".join(missing_crucial))) Exception: Crucial files were excluded from the sdist: src/sage_conf.py
comment:11 Changed 2 years ago by
Commit:  4d20d4a7c1ed3403644cdd45c7e0afcc4b8c8f2d → 7ea5fabac7e4f2485a89e93f2e5931c0f9639935 

Branch pushed to git repo; I updated commit sha1. New commits:
7ea5fab  build/pkgs/sage_conf/src/pyproject.toml: Add tool.flit.sdist section

comment:12 Changed 2 years ago by
This fixes the sdist.
Same change on top of 27cf73b creates a tar file with a packaged symlink:
lrwxrxrx 0/0 0 sage_conf9.2b0/src > ../../../../src
and gives an error afterwards:
Built sdist: dist/sage_conf9.2b0.tar.gz Iflit_core.sdist Traceback (most recent call last): File "/Users/mkoeppe/s/sage/sagerebasing/local/bin/flit", line 8, in <module> sys.exit(main()) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.7/sitepackages/flit/__init__.py", line 171, in main gen_setup_py=args.setup_py) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.7/sitepackages/flit/build.py", line 51, in main with unpacked_tarball(sdist_file) as tmpdir: File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py", line 112, in __enter__ return next(self.gen) File "/Users/mkoeppe/s/sage/sagerebasing/local/lib/python3.7/sitepackages/flit/build.py", line 24, in unpacked_tarball tf.extractall(tmpdir) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/tarfile.py", line 2000, in extractall numeric_owner=numeric_owner) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/tarfile.py", line 2042, in extract numeric_owner=numeric_owner) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/tarfile.py", line 2104, in _extract_member os.makedirs(upperdirs) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/os.py", line 223, in makedirs mkdir(name, mode) FileExistsError: [Errno 17] File exists: '/var/folders/38/wnh4gf1552g_crsjnv2vmmww0000gp/T/tmp6jcs8l04/sage_conf9.2b0/src'
comment:13 Changed 2 years ago by
A big part of the issues with symlinks comes, of course, from the behavior of pip, which copies the local source directory to a temporary location, breaking symlinks in the process. Various unresolved pip issues talk about finding a solution for local builds, most recently https://github.com/pypa/pip/issues/7555 (earlier issues: https://github.com/pypa/pip/issues/2195 and within, https://github.com/pypa/pip/pull/7882).
comment:14 Changed 2 years ago by
Milestone:  sage9.2 → sage9.3 

comment:15 Changed 20 months ago by
Milestone:  sage9.3 → sage9.4 

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.
comment:16 Changed 14 months ago by
Milestone:  sage9.4 → sage9.5 

comment:17 Changed 10 months ago by
Milestone:  sage9.5 → sage9.6 

comment:18 Changed 8 months ago by
Milestone:  sage9.6 → sagewishlist 

Last 10 new commits:
WIP: Install sageenvconfig with sage_conf
Fixup
Merge branch 't/21559/changesrcbininstallation' into t/29850/install_sage_env_config_with_sage_conf
build/pkgs/pytoml: New
build/pkgs/flit, flit_core: New
Add dependencies
build/bin/sagepipinstall: For PEP 517 packages, do not try to uninstall first
build/pkgs/pytoml/spkginstall.in: New
Merge branch 't/29846/add_packages_flit_and_dependencies_flit_core__pytoml' into t/29850/install_sage_env_config_with_sage_conf
build/pkgs/sage_conf: Install with flit instead of setuptools