Opened 2 years ago

Closed 2 years ago

#30974 closed enhancement (fixed)

sage --package create PACKAGE --pypi

Reported by: Matthias Köppe Owned by:
Priority: major Milestone: sage-9.3
Component: build Keywords: sd111
Cc: Sébastien Labbé, John Palmieri, Martin Albrecht Merged in:
Authors: Matthias Koeppe Reviewers: Sébastien Labbé
Report Upstream: N/A Work issues:
Branch: 0c42490 (Commits, GitHub, GitLab) Commit: 0c4249072d1ab781a83a69e5503d4a88aa2f9db2
Dependencies: Stopgaps:

Status badges

Description (last modified by Matthias Köppe)

We add a simple command that creates a normal package from PyPI.

$ ./sage -package create scikit_spatial --pypi
Downloading tarball to /Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/upstream/scikit-spatial-5.0.0.tar.gz
[......................................................................]
$ ls build/pkgs/scikit_spatial/
SPKG.rst		dependencies		package-version.txt	type
checksums.ini		install-requires.txt	spkg-install.in
$ cat build/pkgs/scikit_spatial/SPKG.rst 
scikit_spatial
==============

Description
-----------

Spatial objects and computations based on NumPy arrays.

License
-------

BSD license

Upstream Contact
----------------

https://pypi.org/project/scikit-spatial/

Additionally, we fix the following error that occurs in sage --package create when --type is not set:

./sage -package create jupyter_jsmol --tarball jupyter_jsmol-VERSION.tar.gz --url https://pypi.io/packages/source/j/jupyter-jsmol/jupyter_jsmol-VERSION.tar.gz --version 0.2.4
  File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/build/bin/../sage_bootstrap/package.py", line 291, in _init_type
    with open(os.path.join(self.path, 'type')) as f:
IOError: [Errno 2] No such file or directory: '/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/build/pkgs/jupyter_jsmol/type'

The fix just uses type optional in this situation, which is a good default.

Change History (19)

comment:1 Changed 2 years ago by Matthias Köppe

Branch: u/mkoeppe/improve_sage___package_create

comment:2 Changed 2 years ago by Matthias Köppe

Commit: c2f578100bbb49bb5b12d764c71c35ed9fa3111e

Help on this ticket is welcome


New commits:

c2f5781WIP

comment:3 Changed 2 years ago by git

Commit: c2f578100bbb49bb5b12d764c71c35ed9fa3111e879b042a260001fb7b351095edc247e8fc9660cb

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

3ff85e7sage_bootstrap.pypi.PyPiVersion: Add properties tarball, package_url, license, summary
c81ddf6sage_bootstrap.creator.PackageCreator.set_description: New
879b042sage --package create: Add options to set the contents of SPKG.rst, and to get data from PyPI

comment:4 Changed 2 years ago by git

Commit: 879b042a260001fb7b351095edc247e8fc9660cb0622e205672b00538af05ebdddd3c1828e4a9ba3

Branch pushed to git repo; I updated commit sha1. New commits:

0622e20sage --package create: If --pypi is given, also write default dependencies, spkg-install.in, and install-requires.txt

comment:5 Changed 2 years ago by git

Commit: 0622e205672b00538af05ebdddd3c1828e4a9ba3f0102d3bb859b0bd243532336938552fb7fca781

Branch pushed to git repo; I updated commit sha1. New commits:

f0102d3sage --package create --pypi: Use the canonical package name in install-requires.txt

comment:6 Changed 2 years ago by git

Commit: f0102d3bb859b0bd243532336938552fb7fca7813ebccc256992f716175707ba41d722046da0816d

Branch pushed to git repo; I updated commit sha1. New commits:

c54a33fsage --package create: If a tarball is set, provide a default to 'type'
3ebccc2sage --package cerate: Refuse to create packages with dashes in their name

comment:7 Changed 2 years ago by git

Commit: 3ebccc256992f716175707ba41d722046da0816de82fe48bca9295256e00053145787fc8b99725ff

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

e82fe48sage --package create: Refuse to create packages with dashes in their name

comment:8 Changed 2 years ago by Matthias Köppe

Authors: Matthias Koeppe
Cc: John Palmieri added
Description: modified (diff)
Status: newneeds_review
Summary: Improve sage --package createsage --package create PACKAGE --pypi

comment:9 Changed 2 years ago by git

Commit: e82fe48bca9295256e00053145787fc8b99725ffe904464680eecebba2235c2f9fdc3729432f922a

Branch pushed to git repo; I updated commit sha1. New commits:

8d07bb5src/doc/en/developer/packaging.rst: Mention upstream_url
86e048fsrc/doc/en/developer/packaging.rst: Trimming mirror list
401ca7cMerge commit '86e048fc1ec5c2bf04f7eb46319f53b283a079fe' of git://trac.sagemath.org/sage into t/30974/improve_sage___package_create
e904464src/doc/en/developer/packaging.rst: Document sage --package create --pypi

comment:10 Changed 2 years ago by git

Commit: e904464680eecebba2235c2f9fdc3729432f922a0c4249072d1ab781a83a69e5503d4a88aa2f9db2

Branch pushed to git repo; I updated commit sha1. New commits:

0c42490sage --package create PACKAGE --pypi --source=pip

comment:11 Changed 2 years ago by Matthias Köppe

Keywords: sd111 added

Hoping we can make progress on this ticket this week - https://wiki.sagemath.org/days111

comment:12 Changed 2 years ago by Matthias Köppe

Cc: Martin Albrecht added

comment:13 Changed 2 years ago by Sébastien Labbé

Status: needs_reviewneeds_info

I tested it with my own optional package which I should include in the list of sage optional package at some point...

It looks good:

$ sage -package create slabbe --pypi --type optional
Downloading tarball to /home/slabbe/GitBox/sage/upstream/slabbe-0.6.1.tar.gz
[......................................................................]
$ git status
Sur la branche 30974
Fichiers non suivis:
  (utilisez "git add <fichier>..." pour inclure dans ce qui sera validé)

	build/pkgs/slabbe/

aucune modification ajoutée à la validation mais des fichiers non suivis sont présents (utilisez "git add" pour les suivre)
slabbe@miami sage $ cd build/pkgs/slabbe/
slabbe@miami slabbe $ ls
checksums.ini  install-requires.txt  spkg-install.in  type
dependencies   package-version.txt   SPKG.rst

The content of the following is:

$ cat install-requires.txt 
slabbe

But, my package does depend on two other packages:

$ grep requires ~/GitBox/slabbe/setup.py 
    install_requires=['pytimeparse', 'roman'],
    #install_requires=['cython','cysignals'], # this causes update of cysignals

I guess we do not want the create command to be recursive, but should the content of install-requires.txt depend on the content of the install_requires variable of the setup.py of the package?

comment:14 Changed 2 years ago by Sébastien Labbé

Now testing as a pip package rather than a normal package:

$ mv slabbe slabbe-1
$ cd ../..
$ sage -package create slabbe --pypi --source pip --type optional
$ git status
Sur la branche 30974
Fichiers non suivis:
  (utilisez "git add <fichier>..." pour inclure dans ce qui sera validé)

	build/pkgs/slabbe-1/
	build/pkgs/slabbe/

$ cd build/pkgs/slabbe
$ ls
dependencies  requirements.txt  SPKG.rst  type
$ cat dependencies 
$(PYTHON) | $(PYTHON_TOOLCHAIN)

----------
All lines of this file are ignored except the first.
$ cat requirements.txt 
slabbe
$ cat type 
optional

Again, should install_requires=['pytimeparse', 'roman'], be considered in the content of the above?

comment:15 Changed 2 years ago by Matthias Köppe

No, I think it's out of the scope of the ticket to determine dependencies by looking into the tarball

comment:16 Changed 2 years ago by Matthias Köppe

Status: needs_infoneeds_review

comment:17 Changed 2 years ago by Sébastien Labbé

Reviewers: Sébastien Labbé
Status: needs_reviewpositive_review

Ok.

I tested the creation (2 ways: spkg + pip). I chacked that installation (sage -i) works in both cases.

comment:18 Changed 2 years ago by Matthias Köppe

Thank you!

comment:19 Changed 2 years ago by Volker Braun

Branch: u/mkoeppe/improve_sage___package_create0c4249072d1ab781a83a69e5503d4a88aa2f9db2
Resolution: fixed
Status: positive_reviewclosed
Note: See TracTickets for help on using tickets.