p_group_cohomology upgrade

[SimonKing]

The optional package p_group_cohomology-2.1.4 does not install on sage-6.1.b4. First problem: It tries to include interrupt.pxi, but apparently these things have moved.

Also, it is using the old spkg layout. With a little help from an expert of the new conventions after switching to git, this could be fixed as well.

A package version that installs both on sage-5.12, sage-5.13 and sage-6.1.beta4 is available at ​http://sage.math.washington.edu/home/SimonKing/Cohomology/p_group_cohomology-2.1.4.p1.spkg

[SimonKing]

In an unpublished version, I have solved it to the extent that the package installs both with sage-5.13.beta4 and sage-6.1.beta4, and passes almost all tests with sage-6.1.beta4.

So, this needs to be addressed as well. But the failures seem relatively harmless:

• Some results have a different sign than expected. However, the given result is mathematically correct. So, I should think how to modify the tests so that they do not rely on a choice of a representative.
• Apparently there have been changes in the Singular interface. In two corner cases, tests fail while it is attempted to create some ideal in the Singular interface.

[SimonKing]

I guess I should Cc a couple of people. I can modify my spkg so that it installs fine (in spite of the new source layout of Sage) both in sage-5.13.beta4 and sage-6.1.beta4, and it passes most tests, except a corner case in using the Singular interface and except some unexpected results that are mathematically equivalent.

However, I need some help in the transition to using git on spkgs. If I understood correctly, one should provide a pointer to "upstream" code that will then be downloaded and installed. It could be that this is not so straight forward:

• Some code is Python, Cython, Gap and Singular code that I wrote exclusively for the spkg: In a way, the spkg *is* upstream.
• Some code is C-code written by David Green, and available only in the spkg. Again, in a way, the spkg is upstream.
• Some code is a fork of an old version of C-MeatAxe. Why "old"? Well, the new version of C-MeatAxe is not faster than the old, but the function names have thoroughly changed. Switching to the new version would have no advantage for the spkg, but would require a rewrite of David Green's code. And why "fork"? Well, the spkg applies quite some patch to the original sources of the C-MeatAxe. At some point, I have send my patch to the authors of C-MeatAxe, but I don't think they are using it now (and it does not seem to be a very active project).

So, given this setting, what is and how to access "upstream"?

[vbraun]

Make your own tarball for "upstream sources". You are upstream for any code that is not available anywhere else or for forks that you are maintaining. Publish the tarball, ideally tracking its contents in one of your own repositories.

PS: it goes without saying, but it would be nice if your meataxe fork were to not conflict with the new meataxe.

[SimonKing]

As a first step, I was fixing the most urgent problem: The package did not build on the most recent Sage versions. Also, the expected output of some doctests changed (old and new result are mathematically equivalent).

A colleague of mine had problems to load cohomology rings that he computed and saved with previous versions of the spkg. The underlying problem was that the address of the group in the small groups library is supposed to be a pair of integers, but the package did not complain if is was given by two integers defined in the GAP pexpect interface. If you then saved the cohomology ring, the pexpect element would be part of the pickle---but can not be unpickled, because the element is defined in terms of a GAP session that is not available when one reloads the data.

So, I needed to do two or three more changes:

• If a saved cohomology ring is loaded, then "custom" attributes depending on pexpect elements are simply ignored. If the loaded ring's attribute _key (which is important for the cache of cohomology rings) depends on pexpect (which is the case in the scenario sketched above), then it is attempted to reconstruct it from other available data. This was enough to rescue the data of my colleague, but admittedly it involves some trickery.
• The small groups library address is now always transformed into a pair of (Sage) integers, even if it was originally provided by GAP-via-pexpect.

So I think there already is something that can be reviewed.

What is not done yet (and therefore it is 2.1.4.p1 and not 2.1.5):

• Updated SPKG.txt
• Transition to git. Here I'd really need some guidance how to "track its contents in my own repository": How to set up a repository at university of Washington (I think there is the "official" home of the package: public documentation, download site and database), so that people can pull from it? How to set it up such that I can push to it from Jena?

What will not be done: Moving my fork to more recent MeatAxe. Reasons:

• My current project is to replace the algorithm for computing minimal projective resolutions by something faster (using a non-commutative F5 algorithm). This, I plan to do in the Sage library, relying on the matrices available there (e.g., Linbox). Hence, MeatAxe is not involved in my current project.
• Rebasing my patches on top of the new version would involve a lot of work.
• David Green's code for computing minimal projective resolutions uses MeatAxe for matrix arithmetic, and it was written using the old MeatAxe version. Hence, I would need to change a substantial amount of C-code that was not authored by me. Not a nice job.

So, I would only consider to switch to the latest MeatAxe if there would be a good reason. However, the matrix arithmetic in the latest MeatAxe version is not superior to the old version (MeatAxe is much more than matrix arithmetic, by the way, but I am using it, since MeatAxe's matrix arithmetic non-prime fields of odd order <255 is substantially faster than the matrix implementation that is currently used in Sage).

[dimpase]

Ready for review?

[SimonKing]

I have different things on my plate at the moment, so that I would like to postpone the switch to the new spkg layout. So, from my perspective, making it work again has priority. All tests should pass on both pre and post git-switch Sage.

So, if you don't mind, I think one can review it, but of course I should document the switch from v2.1.4 to v2.1.4.p1 in SPKG.txt.

[dimpase]

Replying to SimonKing:

• Transition to git. Here I'd really need some guidance how to "track its contents in my own repository": How to set up a repository at university of Washington (I think there is the "official" home of the package: public documentation, download site and database), so that people can pull from it? How to set it up such that I can push to it from Jena?

Why don't you put it on github (or bitbucket - you can host git repos there too) ? This is really easy...

[vbraun]

Harald, can you mirror this? whats the process for old-style spkgs, just manually put in the right folder on boxen?

[schilly]

ok, spkg is mirrored