Opened 3 years ago

Closed 11 months ago

#29919 closed enhancement (fixed)

Restore "huge" package type

Reported by: mkoeppe Owned by:
Priority: major Milestone: sage-9.6
Component: build Keywords:
Cc: jhpalmieri, dimpase, mjo, slabbe, etn40ff, gh-tobiasdiez Merged in:
Authors: Matthias Koeppe Reviewers: John Palmieri
Report Upstream: N/A Work issues:
Branch: 15cc20c (Commits, GitHub, GitLab) Commit: 15cc20c4a35db2b2b3a0bb52e444bb6d67268c32
Dependencies: Stopgaps:

Status badges

Description (last modified by mkoeppe)

(from #29901)

... for packages like database_stein_watkins and polytopes_db_4d

Parts of sage_bootstrap know about this package type, but other scripts don't.

Our GH Actions scripts filter out these packages by name. This can be refactored.

With the present ticket, we can now say sage --package list :optional: :experimental: --no-file huge to filter out the huge packages.

Huge packages are now marked by the presence of the file huge; so this is actually orthogonal to the package type.

We also add files named has_nonfree_dependencies to some other packages so that they can be filtered out by the same mechanism.

Change History (25)

comment:1 Changed 3 years ago by mkoeppe

Cc: mjo added

comment:2 Changed 2 years ago by mkoeppe

Milestone: sage-9.2sage-9.3

comment:3 Changed 2 years ago by mkoeppe

Cc: slabbe added
Description: modified (diff)

comment:4 Changed 2 years ago by mkoeppe

Cc: etn40ff added

comment:5 Changed 2 years ago by mkoeppe

Milestone: sage-9.3sage-9.4

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

comment:6 Changed 19 months ago by mkoeppe

Milestone: sage-9.4sage-9.5

comment:7 Changed 14 months ago by mkoeppe

Milestone: sage-9.5sage-9.6

comment:8 Changed 12 months ago by mkoeppe

Cc: gh-tobiasdiez added
Description: modified (diff)

comment:9 Changed 12 months ago by mkoeppe

Authors: Matthias Koeppe
Description: modified (diff)
Status: newneeds_review

comment:10 Changed 12 months ago by mkoeppe

Branch: u/mkoeppe/restore__huge__package_type

comment:11 Changed 12 months ago by mkoeppe

Commit: ce1924c14776c96eb7f4ab3bbaeb624480411fb6
Dependencies: #29901, #20104

New commits:

e6b82e8build/sage_bootstrap: Remove :huge: package class
74ca883sage -package list: Add option --no-file
ce1924cbuild/pkgs/{database_stein_watkins, polytopes_db_4d}: Mark as huge packages

comment:12 Changed 12 months ago by mkoeppe

Description: modified (diff)

comment:13 Changed 12 months ago by git

Commit: ce1924c14776c96eb7f4ab3bbaeb624480411fb6554ad59f12f3970722315193cbfc537db2b634f0

Branch pushed to git repo; I updated commit sha1. New commits:

554ad59build/pkgs/{scipoptsuite,sage_numerical_backends_cplex,sage_numerical_backends_gurobi}/has_nonfree_dependencies: New

comment:14 Changed 11 months ago by jhpalmieri

Can you document both huge and has_nonfree_dependencies in the developer's guide?

comment:15 Changed 11 months ago by mkoeppe

Good idea

comment:16 Changed 11 months ago by mjo

I think we should have a cleaner way to add "tags" to packages, but in lieu of general solution this gets the job done.

The nonfree-dependencies thing is a bit of a legal mess. Users who install such a package lose the right to distribute their copy of sage. Entities such as universities may not want to wade into that sort of grey area. Anyway, that's orthogonal to this ticket.

comment:17 Changed 11 months ago by git

Commit: 554ad59f12f3970722315193cbfc537db2b634f0b401649578c412275b9e79fbac235c4421837688

Branch pushed to git repo; I updated commit sha1. New commits:

b401649src/doc/en/developer/packaging.rst: Explain package tags has_nonfree_dependencies, huge

comment:18 Changed 11 months ago by git

Commit: b401649578c412275b9e79fbac235c4421837688bf0c9238cea8a7b90bcb1a7a39cfcd2a9dd21ef5

Branch pushed to git repo; I updated commit sha1. New commits:

bf0c923src/doc/en/developer/packaging.rst: Add section title

comment:19 Changed 11 months ago by git

Commit: bf0c9238cea8a7b90bcb1a7a39cfcd2a9dd21ef515cc20c4a35db2b2b3a0bb52e444bb6d67268c32

Branch pushed to git repo; I updated commit sha1. New commits:

15cc20csrc/doc/en/developer/packaging.rst: Typo

comment:20 Changed 11 months ago by jhpalmieri

Would

if any(not pkg.has_file(filename) for filename in filenames):

be more efficient than the current

if not all(pkg.has_file(filename) for filename in filenames):

and similarly a few lines later with not any(...)? Or does Python do this sort of optimization automatically?

comment:21 Changed 11 months ago by jhpalmieri

Never mind, I guess it doesn't matter.

comment:22 Changed 11 months ago by mkoeppe

There's no difference in efficiency - both any and all shortcircuit. And I think there is no difference in clarity

comment:23 Changed 11 months ago by jhpalmieri

Reviewers: John Palmieri
Status: needs_reviewpositive_review

I got confused and thought that the two might short-circuit at different points, but they will short-circuit at exactly the same point, so it makes no difference.

This is not your fault and should not be fixed on this ticket, but I wish that the code in expand_class.py were documented. Anyway, let's move forward with this.

comment:24 Changed 11 months ago by mkoeppe

Thanks!

comment:25 Changed 11 months ago by vbraun

Branch: u/mkoeppe/restore__huge__package_type15cc20c4a35db2b2b3a0bb52e444bb6d67268c32
Resolution: fixed
Status: positive_reviewclosed
Note: See TracTickets for help on using tickets.