Opened 9 years ago

Last modified 9 years ago

#11845 new enhancement

Pickle python functions

Reported by: nbruin Owned by: was
Priority: major Milestone: sage-feature
Component: pickling Keywords:
Cc: Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description

Simon King pointed out that, although python function objects are normally not guaranteed to be picklable, the marshal module is happy to serialize code objects (although portability of the result is not guaranteed across versions), see

sage-devel thread

Since sage pickles are supposed to be highly portable, we should probably not enable the feature by default, but the feature can be extremely useful when, for instance, distributing jobs across multiple worker nodes.

Some work is needed to work properly with closures, so having a debugged and well-maintained interface for pickling python functions is desirable. I have no idea whether it would be appropriate to stuff somewhere in the sage library, so for now this ticket can serve as a repository.


Attachments (1)

picklefunc.py (4.5 KB) - added by nbruin 9 years ago.
implementation of a PickleableFunction? class (not a patch!)

Download all attachments as: .zip

Change History (3)

Changed 9 years ago by nbruin

implementation of a PickleableFunction? class (not a patch!)

comment:1 follow-up: Changed 9 years ago by burcin

Note that there is already some code in the library for this in sage.misc.fpickle. It is used at least in @parallel and pickling of symbolic functions with user defined evaluation, derivation, etc. methods. Now that I read the thread on sage-devel, I realize that we might have problems unpickling these symbolic functions if this pickle format is not compatible between Python versions.

comment:2 in reply to: ↑ 1 Changed 9 years ago by nbruin

Replying to burcin:

Note that there is already some code in the library for this in sage.misc.fpickle. It is used at least in @parallel and pickling of symbolic functions with user defined evaluation, derivation, etc. methods.

Are those pickles doctested? In that case, sage's picklejar would already be tainted. Incidentally, if @parallel uses this to distribute code & data over workers, then supporting closures is probably a very welcome improvement. Creating a closure for each job is a virtually ideal way of wrapping code&data together for distribution.

Note: See TracTickets for help on using tickets.