Opened 10 years ago

Closed 16 hours ago

#11729 closed enhancement (invalid)

Cache module import locations

Reported by: vbraun Owned by: jason
Priority: major Milestone: sage-duplicate/invalid/wontfix
Component: misc Keywords: sd32
Cc: was, robertwb, ohanar, SimonKing Merged in:
Authors: Volker Braun Reviewers: Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description (last modified by vbraun)

This ticket implements a cached module importer for Python. It compiles a list of all available files during build (when sage -b is run), hooks into the Python import framework and finds modules by the predetermined file list.

Building the Sage library (sage -b) automatically creates the cache and the sitecustomize.py that is used to load the importer into the Python session. The cache is a standard Python dictionary in sage/misc/modules_cache.py.

[vbraun@volker-laptop-two hg]$ strace -e file -f sage -c quit |& grep ENOENT | wc -l
26327
[vbraun@volker-laptop-two hg]$ sage -b
...
[vbraun@volker-laptop-two hg]$ strace -e file -f sage -c quit |& grep ENOENT | wc -l
3785

If you encounter any problems while testing this patch, you have to

rm $SAGE_LOCAL/lib/python/site-packages/sitecustomize.py*

to not load the importer during Python startup.

Apply trac_11729_cached_importer.patch to the sage library and trac_11729_import_cache_during_build.patch to the sage_scripts ($AGE_LOCAL/bin) repository.

Attachments (2)

trac_11729_import_cache_during_build.patch (1.3 KB) - added by vbraun 10 years ago.
Updated patch
trac_11729_cached_importer.patch (18.9 KB) - added by vbraun 10 years ago.
Updated patch

Download all attachments as: .zip

Change History (19)

comment:1 Changed 10 years ago by vbraun

  • Description modified (diff)

comment:2 Changed 10 years ago by vbraun

  • Cc was added

comment:3 Changed 10 years ago by robertwb

  • Cc robertwb added

comment:4 Changed 10 years ago by was

  • Keywords sd32 added

comment:5 Changed 10 years ago by vbraun

  • Description modified (diff)

comment:6 Changed 10 years ago by was

NOTE: If you try to test this, watch out because of these lines:

	113	cat <<EOF > "$SAGE_LOCAL/lib/python/site-packages/sitecustomize.py" 
 	114	from sage.misc.importer import sage_importer 
 	115	sage_importer.register() 
 	116	EOF 

They can make it so your Sage install is totally broken. To fix, just remove the contents of sitecustomize.py. I spent 15 minutes confused by this.

comment:7 Changed 10 years ago by vbraun

You have to delete both $SAGE_LOCAL/lib/python/site-packages/sitecustomize.py and $SAGE_LOCAL/lib/python/site-packages/sitecustomize.pyc, though the latter might not exist. The final version of the patch will test that sage starts up successfully and otherwise delete the sitecustomize.py automatically.

Also, if you run into problems (presumably some ImportError), please post what is causing it.

Changed 10 years ago by vbraun

Updated patch

Changed 10 years ago by vbraun

Updated patch

comment:8 Changed 10 years ago by vbraun

  • Description modified (diff)

I've added another workaround for PIL.Image, now make ptest passes! I would say that the patch is now feature-complete, though it lacks documentation and doctests. Though writing doctests is tricky without breaking the module importing. Somebody should test it on OSX :-)

comment:9 Changed 9 years ago by ohanar

  • Cc ohanar added

comment:10 Changed 9 years ago by SimonKing

  • Cc SimonKing added

comment:11 Changed 8 years ago by jdemeyer

  • Milestone changed from sage-5.11 to sage-5.12

comment:12 Changed 8 years ago by vbraun_spam

  • Milestone changed from sage-6.1 to sage-6.2

comment:13 Changed 8 years ago by vbraun_spam

  • Milestone changed from sage-6.2 to sage-6.3

comment:14 Changed 7 years ago by vbraun_spam

  • Milestone changed from sage-6.3 to sage-6.4

comment:15 Changed 2 days ago by mkoeppe

  • Milestone changed from sage-6.4 to sage-duplicate/invalid/wontfix
  • Status changed from new to needs_review

outdated, should close

comment:16 Changed 25 hours ago by dimpase

  • Reviewers set to Dima Pasechnik
  • Status changed from needs_review to positive_review

comment:17 Changed 16 hours ago by mkoeppe

  • Resolution set to invalid
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.