make @parallel work with class/instance methods

[niles]

The @parallel decorator currently cannot be used with methods that implicitly prepend their arguments with a class instance, because @parallel requires that the first argument be a list (or iterable) of arguments to pass to the function.

More details can be found at this  asksage question.

---

Apply:

• Patch at #11462
• trac_11461_rebase_3.patch .

[niles]

this patch breaks @parallel, but prints information about how it's broken

[niles]

I wish I could tell buildbot not to apply parallel.patch -- I put it here only to share what I've learned about how @parallel works: The object that gets passed to Parallel is no longer a method, but only a plain function. Here's how I'm testing this:

First, the following goes in a separate file

@parallel
def func(n):
    sleep(.2)
    return prime_pi(n)

class PTest(SageObject):
    attr = 'red'
    def meth0(self,n):
        return n
    @parallel
    def meth1(self,n):
        "long and complicated class method"
        sleep(.5)
        return [prime_pi(n),self.attr]

N = PTest()
M = [1000,2000,3000,4000,5000]
L = [(N,x) for x in [1000,2000,3000,4000,5000]]

Then I apply the patch and do this:

sage: N.meth1(100)
args: (<class '__main__.PTest'>, 100)
branch 2.5.1
...
NotImplementedError: this branch not finished

sage: func(100)
args: (100,)
branch 2.5.1
...
NotImplementedError: this branch not finished

So the function and method are being treated the same by Parallel. None of the ideas for detecting the difference: .__self__, ismethod, or isfunction seem to work. So I guess this means something earlier in the application of @parallel needs to be fixed.

[mhansen]

I put up an initial patch which shows how this sort of thing should be done. The key is the get method which gets the bound/class/static method when it's accessed as an attribute. I still need to add documentation / tests / etc. but for those who need it now, there's a patch.

[niles]

Thanks! This seems to be working well. There are some failures applying to 4.7.1.alpha2, which seem related to ticket #9976 (improving introspection for decorators). I simply dropped those two hunks of the patch and it is working fine (with introspection!) in simple examples.

[mhansen]

I posted a patch which applies on top of #9976.

[mhansen]

Apply trac_11461.patch

[kcrisman]

Formatting things:

• This needs very minor rebasing with respect to #11467.
• It should also have at least one example in the ParallelFunction class definition (there should be a docstring) so that something shows up in the reference manual.
• The end of the call docstring seems to have the triple quote misindented.
• except AtrributeError: this isn't really a formatting error, but just an error. There should be a doctest for this?
• anqd is not a word

It looks good, though. I don't feel quite comfortable enough with the parallel stuff to review it completely, but if the doctests work then this makes sense, and the getdoc etc. certainly is right.

I'm cc:ing Niles in because he originally asked about this, so hopefully he can positively review the rest.

[niles]

Replying to kcrisman:

I'm cc:ing Niles in because he originally asked about this, so hopefully he can positively review the rest.

I've been looking at this, and it's working well for me. I hope to have time for a thorough review soon. One minor issue (from #9976): the author section has "8 apr 2011" which should be "8 Apr 2011".

[niles]

rebased to 4.7.1

[niles]

Passes all long tests on 4.7.1.

Patchbot: apply trac_11461_rebase.patch

[niles]

rebased to 4.7.2

[niles]

Rebased again; comments of Karl have also been addressed; anyone else who could review this?

Patchbot: apply trac_11461_rebase.patch

[niles]

I meant to say:

Patchbot: apply trac_11461_rebase_2.patch

[kcrisman]

See also #11462. My guess is that that has priority due to the prior positive review...

Niles, my sense is that you only rebased, but give positive review to Mike's patch. What would still have to be reviewed?

[niles]

Indeed, I see no problems with Mike's patch. The descriptor protocol is the right way to fix this, and the implementation of sage.misc.sageinspect methods is straightforward. Rebase 3 fixes a minor issue with the documentation: Positive review!

Patchbot: apply trac_11461_rebase_3.patch

[niles]

rebased to 4.7.2, minor typos fixed, now depends on #11462

[niles]

Sorry for the many updates; this patch is now compatible with (depends on) #11462.

[niles]

Patchbot: apply #11462 and trac_11461_rebase_3.patch