Opened 5 years ago

Last modified 5 weeks ago

#25499 new defect

list of projective points fails to pickle in p_iter_fork call

Reported by: bhutz Owned by:
Priority: minor Milestone:
Component: pickling Keywords:
Cc: raghukul01, saraedum, tscrim Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges


The following is a toy example showing that a list of projective points is failing to pickle. Replace the point return with just the list of coordinates works just fine.

def func(Z, p):
    Xp = Z.change_ring(GF(p))
    return Xp.rational_points()

from sage.parallel.ncpus import ncpus
from sage.parallel.use_fork import p_iter_fork


normalized_input = []
for q in primes(1,10):
    normalized_input.append(((Y, q, ), {}))

p_iter = p_iter_fork(ncpus(), 10)
points_pair = list(p_iter(func, normalized_input))

Attachments (2)

pickle.sage (2.4 KB) - added by saraedum 5 years ago.
a hacky script to produce the problematic pickle graph (9.9 KB) - added by saraedum 5 years ago.
the pickle graph

Download all attachments as: .zip

Change History (9)

Changed 5 years ago by saraedum

Attachment: pickle.sage added

a hacky script to produce the problematic pickle graph

Changed 5 years ago by saraedum

Attachment: added

the pickle graph

comment:1 Changed 5 years ago by saraedum

So, I wrote a little script (attached) that creates a graph of objects that fail to pickle here. You can look at the attached dot-file with xdot. Each node is an object that fails to pickle. Each edge points to another node that is part of that object and that also fails to pickle. (more specifically, it points to the items in __reduce_ex__(2) that fail to pickle.) Nodes that have edges going out of them are usually not problematic. They most probably fail to pickle because something inside them fails to pickle. The problem are most likely the sinks, i.e., the nodes with zero out-degree at the bottom of the graph. From my experience these are likely decorated functions/methods or creative caching constructions.

comment:2 Changed 5 years ago by bhutz

I installed xdot and looked at the .dot file. I see the 4 sinks. They are all named fairly obscure things like <type Type>. Is there a way to figure out what those actually are?

comment:3 Changed 5 years ago by bhutz

It almost looks like it is the defining polynomials of the subscheme that is failing to pickle not the points. And if you change the example to have

return Xp.defining_ideal()

you get the same pickling error.

comment:4 Changed 5 years ago by saraedum

It's a bit tricky to find out what these are. You can run my script and add arbitrary print statements where the sinks are printed. It might be possible to connect a debugger somehow, but it's complicated because it's in a forked process. I think that all four are methods of some kind.

Btw, if the code complains about some package not being found, you can install it with sage -pip PACKAGE.

comment:5 Changed 5 years ago by saraedum

Sorry, sage -pip install PACKAGE.

comment:6 Changed 5 years ago by raghukul01

This bug is not specific to projective points, if we change P to AffineSpace, error is same.

comment:7 Changed 5 weeks ago by mkoeppe

Milestone: sage-8.3
Note: See TracTickets for help on using tickets.