Opened 9 years ago
Last modified 9 years ago
#11845 new enhancement
Pickle python functions
Reported by: | nbruin | Owned by: | was |
---|---|---|---|
Priority: | major | Milestone: | sage-feature |
Component: | pickling | Keywords: | |
Cc: | Merged in: | ||
Authors: | Reviewers: | ||
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description
Simon King pointed out that, although python function objects are normally not guaranteed to be picklable, the marshal module is happy to serialize code objects (although portability of the result is not guaranteed across versions), see
Since sage pickles are supposed to be highly portable, we should probably not enable the feature by default, but the feature can be extremely useful when, for instance, distributing jobs across multiple worker nodes.
Some work is needed to work properly with closures, so having a debugged and well-maintained interface for pickling python functions is desirable. I have no idea whether it would be appropriate to stuff somewhere in the sage library, so for now this ticket can serve as a repository.
Attachments (1)
Change History (3)
Changed 9 years ago by
comment:1 follow-up: ↓ 2 Changed 9 years ago by
Note that there is already some code in the library for this in sage.misc.fpickle
. It is used at least in @parallel
and pickling of symbolic functions with user defined evaluation, derivation, etc. methods. Now that I read the thread on sage-devel, I realize that we might have problems unpickling these symbolic functions if this pickle format is not compatible between Python versions.
comment:2 in reply to: ↑ 1 Changed 9 years ago by
Replying to burcin:
Note that there is already some code in the library for this in
sage.misc.fpickle
. It is used at least in@parallel
and pickling of symbolic functions with user defined evaluation, derivation, etc. methods.
Are those pickles doctested? In that case, sage's picklejar would already be tainted. Incidentally, if @parallel uses this to distribute code & data over workers, then supporting closures is probably a very welcome improvement. Creating a closure for each job is a virtually ideal way of wrapping code&data together for distribution.
implementation of a PickleableFunction? class (not a patch!)