Opened 9 months ago

Closed 9 months ago

Last modified 8 months ago

#32707 closed enhancement (fixed)

flit_core: Update to 3.4.0; setuptools_scm: Update to 6.3.2

Reported by: mkoeppe Owned by:
Priority: critical Milestone: sage-9.5
Component: packages: standard Keywords:
Cc: vbraun, dimpase, jhpalmieri Merged in:
Authors: Matthias Koeppe Reviewers: Volker Braun
Report Upstream: N/A Work issues:
Branch: 9eea50c (Commits, GitHub, GitLab) Commit:
Dependencies: #32698 Stopgaps:

Status badges

Description (last modified by mkoeppe)

https://flit.readthedocs.io/en/latest/history.html#version-3-4

This simplifies the dependencies and removes a "circular dependency" warning from the output during build.

Change History (28)

comment:1 Changed 9 months ago by mkoeppe

  • Authors set to Matthias Koeppe

comment:2 Changed 9 months ago by mkoeppe

  • Dependencies set to #32698

comment:3 Changed 9 months ago by mkoeppe

  • Dependencies changed from #32698 to #32698, #32492

comment:4 Changed 9 months ago by mkoeppe

  • Branch set to u/mkoeppe/flit_core__update_to_3_4_0

comment:5 Changed 9 months ago by mkoeppe

  • Commit set to ed0b42b704f26e17059fdc1ef407e8be2778cdde
  • Dependencies changed from #32698, #32492 to #32698
  • Summary changed from flit_core: Update to 3.4.0 to flit_core: Update to 3.4.0; setuptools_scm: Update to 6.3.2; remove package toml

Last 10 new commits:

8bcbf1dMakefile (distclean): Remove convenience symlinks prefix, venv
dcb4a08configure.ac: Indicate default in help string for --with-sage-venv
e33bc89Merge tag '9.5.beta3' into t/32442/__configure__make___with_sage_venv_the_default
2101b8cbuild/pkgs/python3/spkg-build.in: Make sure that python finds sqlite3 when determining which extension modules to build
3bbc5d8build/pkgs/python3/spkg-build.in: Set rpath
99f0a5fMerge #32698
7100660build/pkgs/backcall/dependencies: flit_core now uses tomli
0f5158fbuild/pkgs/setuptools_scm: Update to 6.3.2
e053097build/pkgs/importlib_metadata/dependencies: setuptools_scm[toml] uses tomli now
ed0b42bbuild/pkgs/toml: Remove

comment:6 Changed 9 months ago by git

  • Commit changed from ed0b42b704f26e17059fdc1ef407e8be2778cdde to 4720666f8525cfc6923c1a290e51a49016c6127f

Branch pushed to git repo; I updated commit sha1. New commits:

4720666build/pkgs/tomli/spkg-install.in: Set up PYTHONPATH

comment:7 Changed 9 months ago by mkoeppe

  • Status changed from new to needs_review

comment:8 follow-up: Changed 9 months ago by vbraun

Is this the expected output?

ERROR: Could not find a version that satisfies the requirement tomli (from flit-core) (from versions: none)
ERROR: No matching distribution found for tomli
Warning: installing with "python3 -m pip install --verbose --no-index --find-links=/home/release/Sage/local/var/lib/sage/venv-python3.9/var/lib/sage/wheels --disable-pip-version-check --isolated --no-cache-dir" failed. Retrying, adding "--no-deps --ignore-installed --ignore-requires-python"
Using pip 21.2.4 from /home/release/Sage/local/var/lib/sage/venv-python3.9/lib64/python3.9/site-packages/pip (python 3.9)
Looking in links: /home/release/Sage/local/var/lib/sage/venv-python3.9/var/lib/sage/wheels
Processing /home/release/Sage/local/var/lib/sage/venv-python3.9/var/tmp/sage/build/flit_core-3.4.0/inst/home/release/Sage/local/var/lib/sage/venv-python3.9/var/lib/sage/wheels/flit_core-3.4.0-py3-none-any.whl
Installing collected packages: flit-core
Successfully installed flit-core-3.4.0
Warning: The installation needed to use "--no-deps --ignore-installed --ignore-requires-python" to succeed. This means that a dependencies file in build/pkgs/ needs to be updated. Please report this to sage-devel@googlegroups.com, including the build log of this package.

comment:9 follow-up: Changed 9 months ago by vbraun

tox still depends on toml:

build/pkgs/tox/dependencies:$(PYTHON) packaging six filelock pluggy py toml virtualenv importlib_metadata | $(PYTHON_TOOLCHAIN)

comment:10 in reply to: ↑ 8 Changed 9 months ago by mkoeppe

Replying to vbraun:

Is this the expected output?

Warning: The installation needed to use "--no-deps --ignore-installed --ignore-requires-python" to succeed. This means that a dependencies file in build/pkgs/ needs to be updated. Please report this to sage-devel@googlegroups.com, including the build log of this package.

Yes

comment:11 Changed 9 months ago by git

  • Commit changed from 4720666f8525cfc6923c1a290e51a49016c6127f to 9eea50ccf4bf2db2f824c89b21a387334b1e01b4

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

fa561b2build/pkgs/tomli/spkg-install.in: Set up PYTHONPATH
9eea50cbuild/pkgs/toml/spkg-configure.m4: If system tox is used, mark as not required

comment:12 in reply to: ↑ 9 Changed 9 months ago by mkoeppe

Replying to vbraun:

tox still depends on toml:

build/pkgs/tox/dependencies:$(PYTHON) packaging six filelock pluggy py toml virtualenv importlib_metadata | $(PYTHON_TOOLCHAIN)

Thanks for catching this

comment:13 Changed 9 months ago by mkoeppe

  • Summary changed from flit_core: Update to 3.4.0; setuptools_scm: Update to 6.3.2; remove package toml to flit_core: Update to 3.4.0; setuptools_scm: Update to 6.3.2

comment:14 Changed 9 months ago by mkoeppe

  • Cc jhpalmieri added
  • Description modified (diff)

comment:15 follow-up: Changed 9 months ago by vbraun

Ok its a bit sub-optimal to tell people to complain on sage-devel upon successful installation, but hopefully nobody looks into the log.

However, building the source tarball fails because tox still depends on toml:

sage -sdist
[...]
[threejs-r122.p0] Using cached file /home/release/Sage/upstream/threejs-sage-r122.tar.gz
make[1]: Nothing to be done for 'tomli'.
make[1]: Nothing to be done for 'tornado'.
make[1]: *** No rule to make target 'toml', needed by '/home/release/Sage/local/var/lib/sage/venv-python3.9/var/lib/sage/installed/tox-3.24.3'.  Stop.
make: *** [Makefile:2336: download-for-sdist] Error 2

comment:16 Changed 9 months ago by mkoeppe

Did you run bootstrap?

comment:17 follow-ups: Changed 9 months ago by vbraun

Yes, did run bootstrap. The content of the source tarball hopefully doesn't depend on which system libraries are available.

Also just got this while trying the latest version

****************************************************
Uninstalling existing 'pyparsing'
Warning: File '/home/release/Sage/local/var/lib/sage/wheels/pyparsing-2.4.7-py2.py3-none-any.whl' not found
Warning: Directory '/home/release/Sage/local/var/lib/sage/wheels' not found
Running pip-uninstall script for 'pyparsing'
Found existing installation: pyparsing 2.4.7
Uninstalling pyparsing-2.4.7:
  Successfully uninstalled pyparsing-2.4.7
Removing stamp file '/home/release/Sage/local/var/lib/sage/venv-python3.9/var/lib/sage/installed/pyparsing-2.4.7'
Installing pyparsing-2.4.7
Processing /home/release/Sage/local/var/lib/sage/venv-python3.9/var/tmp/sage/build/pyparsing-2.4.7/src
    Running command python setup.py egg_info
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/home/release/Sage/local/var/lib/sage/venv-python3.9/var/tmp/sage/build/pyparsing-2.4.7/src/setup.py", line 18, in <module>
        setup(# Distribution meta-data
      File "/home/release/Sage/local/var/lib/sage/venv-python3.9/lib64/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
        return distutils.core.setup(**attrs)
      File "/home/release/Sage/local/var/lib/sage/venv-python3.9/lib64/python3.9/site-packages/setuptools/_distutils/core.py", line 108, in setup
        _setup_distribution = dist = klass(attrs)
      File "/home/release/Sage/local/var/lib/sage/venv-python3.9/lib64/python3.9/site-packages/setuptools/dist.py", line 450, in __init__
        _Distribution.__init__(
      File "/home/release/Sage/local/var/lib/sage/venv-python3.9/lib64/python3.9/site-packages/setuptools/_distutils/dist.py", line 293, in __init__
        self.finalize_options()
      File "/home/release/Sage/local/var/lib/sage/venv-python3.9/lib64/python3.9/site-packages/setuptools/dist.py", line 827, in finalize_options
        for ep in sorted(loaded, key=by_order):
      File "/home/release/Sage/local/var/lib/sage/venv-python3.9/lib64/python3.9/site-packages/setuptools/dist.py", line 826, in <lambda>
        loaded = map(lambda e: e.load(), filtered)
      File "/home/release/Sage/local/var/lib/sage/venv-python3.9/lib64/python3.9/site-packages/pkg_resources/__init__.py", line 2449, in load
        self.require(*args, **kwargs)
      File "/home/release/Sage/local/var/lib/sage/venv-python3.9/lib64/python3.9/site-packages/pkg_resources/__init__.py", line 2472, in require
        items = working_set.resolve(reqs, env, installer, extras=self.extras)
      File "/home/release/Sage/local/var/lib/sage/venv-python3.9/lib64/python3.9/site-packages/pkg_resources/__init__.py", line 772, in resolve
        raise DistributionNotFound(req, requirers)
    pkg_resources.DistributionNotFound: The 'tomli>=1.0.0' distribution was not found and is required by the application
WARNING: Discarding file:///home/release/Sage/local/var/lib/sage/venv-python3.9/var/tmp/sage/build/pyparsing-2.4.7/src. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
**************************************************************************************************************************************************************
Error building a wheel for pyparsing-2.4.7
**************************************************************************************************************************************************************

comment:18 in reply to: ↑ 15 Changed 9 months ago by mkoeppe

Replying to vbraun:

Ok its a bit sub-optimal to tell people to complain on sage-devel upon successful installation, but hopefully nobody looks into the log.

Yes, we can address this in #32715.

comment:19 in reply to: ↑ 17 Changed 9 months ago by mkoeppe

Replying to vbraun:

Yes, did run bootstrap. The content of the source tarball hopefully doesn't depend on which system libraries are available.

Asking because it looks like what a make run would look like after the toml package has been removed.

comment:20 Changed 9 months ago by mkoeppe

I recommend to use make dist instead of directly using ./sage --sdist, which takes care of regenerating the Makefile.

Last edited 9 months ago by mkoeppe (previous) (diff)

comment:21 in reply to: ↑ 17 Changed 9 months ago by mkoeppe

Replying to vbraun:

Also just got this while trying the latest version

****************************************************
Uninstalling existing 'pyparsing'
Warning: File '/home/release/Sage/local/var/lib/sage/wheels/pyparsing-2.4.7-py2.py3-none-any.whl' not found
Warning: Directory '/home/release/Sage/local/var/lib/sage/wheels' not found
Running pip-uninstall script for 'pyparsing'
Found existing installation: pyparsing 2.4.7

Best to get rid of this messed up worktree

comment:22 Changed 9 months ago by vbraun

The pyparsing error says that tomli is required, but its not listed as a dependency

[release@zen Sage]$ cat build/pkgs/pyparsing/dependencies 
$(PYTHON) | setuptools pip wheel

I think thats a real dependency error

comment:23 Changed 9 months ago by mkoeppe

There's nothing in pyparsing that refers to tomli.

I think in your worktree, you have for some reason a broken installation record of pip packages.

Can you reproduce this in a from scratch build?

comment:24 Changed 9 months ago by vbraun

  • Reviewers set to Volker Braun
  • Status changed from needs_review to positive_review

comment:25 Changed 9 months ago by vbraun

  • Branch changed from u/mkoeppe/flit_core__update_to_3_4_0 to 9eea50ccf4bf2db2f824c89b21a387334b1e01b4
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:26 Changed 9 months ago by vbraun

  • Commit 9eea50ccf4bf2db2f824c89b21a387334b1e01b4 deleted

I'm still getting very frequent pkg_resources.DistributionNotFound: The 'xxx>=20.0' distribution was not found and is required by the application errors with incremental builds with some packaging-related pip packages. Basically have to do a make distclean every day now. Something is messed up there...

comment:27 Changed 9 months ago by vbraun

I've created #32751 for this issue

comment:28 Changed 8 months ago by vbraun

The issue is not fixed, I still see very frequent pkg_resources.DistributionNotFound: The 'xxx>=20.0' distribution was not found and is required by the application errors during incremental builds

Note: See TracTickets for help on using tickets.