Conversion from Singular to Sage

[SimonKing]

On ​sage-devel, Francisco Botana complained about some shortcomings of the conversion from Singular (pexpect interface) to Sage.

I think the conversions provided by this patch are quite thorough.

First of all, the patch provides a conversion of base rings, even with minpoly, with complicated block, matrix and weighted orders (note that one needs #11316) and even quotient rings:

sage: singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')
'ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp);'
sage: R = singular('r1').sage_basering()
sage: R
Multivariate Polynomial Ring in a, b, c, d, e, f over Finite Field in x of size 3^2
sage: R.term_order()
Block term order with blocks:
(Matrix term order with matrix
[1 2]
[3 0],
 Weighted degree reverse lexicographic term order with weights (2, 3),
 Lexicographic term order of length 2)

sage: singular.eval('ring r3 = (3,z),(a,b,c),dp')
'ring r3 = (3,z),(a,b,c),dp;'
sage: singular.eval('minpoly = 1+z+z2+z3+z4')
'minpoly = 1+z+z2+z3+z4;'
sage: singular('r3').sage_basering()
Multivariate Polynomial Ring in a, b, c over Univariate Quotient Polynomial Ring in z over Finite Field of size 3 with modulus z^4 + z^3 + z^2 + z + 1

sage: singular.eval('ring r5 = (9,a), (x,y,z),lp') 
'ring r5 = (9,a), (x,y,z),lp;'
sage: Q = singular('std(ideal(x^2,x+y^2+z^3))', type='qring')
sage: Q.sage_basering()
Quotient of Multivariate Polynomial Ring in x, y, z over Finite Field in a of size 3^2 by the ideal (y^4 - y^2*z^3 + z^6, x + y^2 + z^3)

By consequence, it is now straight forward to convert polynomials or ideals to sage:

sage: singular.eval('ring R = integer, (x,y,z),lp') 
'// ** You are using coefficient rings which are not fields...'
sage: I = singular.ideal(['x^2','y*z','z+x']) 
sage: I.sage()  # indirect doctest
Ideal (x^2, y*z, x + z) of Multivariate Polynomial Ring in x, y, z over Integer Ring

# Note that conversion of a Singular string to a Sage string was missing
sage: singular('ringlist(basering)').sage()
[['integer'], ['x', 'y', 'z'], [['lp', (1, 1, 1)], ['C', (0)]], Ideal (0) of Multivariate Polynomial Ring in x, y, z over Integer Ring]

sage: singular.eval('ring r10 = (9,a), (x,y,z),lp')
'ring r10 = (9,a), (x,y,z),lp;'
sage: Q = singular('std(ideal(x^2,x+y^2+z^3))', type='qring')
sage: Q.sage()
Quotient of Multivariate Polynomial Ring in x, y, z over Finite Field in a of size 3^2 by the ideal (y^4 - y^2*z^3 + z^6, x + y^2 + z^3)
sage: singular('x^2+y').sage()
x^2 + y
sage: singular('x^2+y').sage().parent()
Quotient of Multivariate Polynomial Ring in x, y, z over Finite Field in a of size 3^2 by the ideal (y^4 - y^2*z^3 + z^6, x + y^2 + z^3)

---

Apply

1. trac11431_singular_sage_conversion.patch​
2. trac11431_singular_sage_documentation.patch​

to the Sage library.

[SimonKing]

Note to the reviewer: I did not check whether the doc strings look nice in html.

[SimonKing]

Conversion Singular -> Sage

[SimonKing]

So far, the method sage() has not been mentioned at all in the documentation of sage.interfaces.singular! Therefore I added a second patch, providing some use cases of .sage() in the short tutorial of sage.interfaces.singular.

The examples that are visible in the html version of the documentation still do not cover all features (quotient rings etc.), but I guess if a user knows about the existence of sage() (s)he will guess its correct usage.

[malb]

• can we rename sage_basering to sage_global_ring. In Sage base_ring has a special meaning which is not meant in your patch. I think global ring captures it.
• don't real fields also have a precision in Singular?
• Previously, we enforced and assumed not short, but it's good you're taking it into account.
• Can you also fix "If it is provided, then you *must* take care OFF it."?
• shouldn't TermOrder_from_Singular be lower-case since it's a function? Or were you aiming for upper-case because it's a constructor?
• In "Invalid termorder in Singular" shouldn't it be term order, i.e. with a space?

[SimonKing]

Hi Martin!

Replying to malb:

• can we rename sage_basering to sage_global_ring. In Sage base_ring has a special meaning which is not meant in your patch. I think global ring captures it.

Sounds fine to me. Obviously I took the name sage_basering from Singular, but you are right that it might be confusing for people who know the notion base ring only from Sage.

• don't real fields also have a precision in Singular?

Right, that should be taken care of as well.

• Can you also fix "If it is provided, then you *must* take care OFF it."?

Why "off"? With two f?

Wouldn't it be correct English to say "If it is provided, then you must take care that it matches the current ring in Singular"? (i.e., I forgot the "that" in my patch)

• shouldn't TermOrder_from_Singular be lower-case since it's a function? Or were you aiming for upper-case because it's a constructor?

I made it upper case because it returns a TermOrder (that is how the class is called). But actually I was ignorant to the convention that functions are lower-case. So I'll change it.

• In "Invalid termorder in Singular" shouldn't it be term order, i.e. with a space?

Yep!

[SimonKing]

Replying to SimonKing:

...

• Can you also fix "If it is provided, then you *must* take care OFF it."?

Why "off"? With two f?

Wouldn't it be correct English to say "If it is provided, then you must take care that it matches the current ring in Singular"? (i.e., I forgot the "that" in my patch)

Perhaps not. But "If it is provided, then you have to make sure that it matches..." is fine, isn't it?

• shouldn't TermOrder_from_Singular be lower-case since it's a function? Or were you aiming for upper-case because it's a constructor?

I made it upper case because it returns a TermOrder (that is how the class is called). But actually I was ignorant to the convention that functions are lower-case. So I'll change it.

• In "Invalid termorder in Singular" shouldn't it be term order, i.e. with a space?

Yep!

[SimonKing]

Show examples of .sage() to the singular interface tutorial. Fix some details of the previous patch.

[SimonKing]

I've put my changes into the second patch, so, please replace that when you test it.

Names: I changed sage_basering into sage_global_ring and TermOrder_from_Singular into termorder_from_singular.

Precision: Singular does not tell the precision of a real field in charstr(), it does so only for complex fields. But the precision both of real and complex fields is part of ringlist(). So, both in the real and the complex case, I take the precision from there, not from charstr().

Grammar: It is now "Invalid term order" (not "termorder") and "If it is provided, then you have to make sure that it matches..." (not "If it is provided, then you *must* take care it matches...")

Tests: Of course I added a test showing that the precision of real fields is taken care of. And it turned out that one doctest needed a fix, because when I created it I used a different term order (so that the same polynomial was printed in a different way).

[malb]

I'll give this a positive review: I agree with Simon's fixes (and the patchbot can take care of any doctest failures etc.)

[SimonKing]

Thank you! And the patchbot actually says that the doctests pass...

[jdemeyer]

There are some issues on Solaris. On the "marks" and "t2" buildbots, I get the following error (identical on both systems):

sage -t -long  -force_lib devel/sage/sage/rings/polynomial/term_order.py
**********************************************************************
File "/scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.1.alpha3/devel/sage-main/sage/rings/polynomial/term_order.py", line 1878:
    sage: singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')
Exception raised:
    Traceback (most recent call last):
      File "/scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.1.alpha3/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.1.alpha3/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.1.alpha3/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_64[2]>", line 1, in <module>
        singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')###line 1878:
    sage: singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')
      File "/scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.1.alpha3/local/lib/python/site-packages/sage/interfaces/singular.py", line 567, in eval
        raise RuntimeError, 'Singular error:\n%s'%s
    RuntimeError: Singular error:
       ? cannot open `gftables/9`
       ? cannot make ring
       ? error occurred in or before STDIN line 33: `ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp);`
       ? expected ring-expression. type 'help ring;'
**********************************************************************
File "/scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.1.alpha3/devel/sage-main/sage/rings/polynomial/term_order.py", line 1881:
    sage: termorder_from_singular(singular)
Expected:
    Block term order with blocks:
    (Matrix term order with matrix
    [1 2]
    [3 0],
     Weighted degree reverse lexicographic term order with weights (2, 3),
     Lexicographic term order of length 2)
Got:
    Block term order with blocks:
    (Lexicographic term order of length 3,
     Degree lexicographic term order of length 5,
     Lexicographic term order of length 2)
**********************************************************************

[SimonKing]

Hi Jeroen,

Replying to jdemeyer:

sage: singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')

...

RuntimeError?: Singular error:

? cannot open gftables/9 ? cannot make ring ? error occurred in or before STDIN line 33: ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp); ? expected ring-expression. type 'help ring;'

That is very strange. I can not imagine that my patch introduced that bug: It changes the conversion from Singular to Sage, but it should not change Singular itself (and what you get in your example is an error from Singular, not from the interface).

Hence, I wonder whether my patch has introduced or uncovered that bug.

Can you test on marks and t2 with unpatched Sage whether the line

sage: singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')

fails?

[jdemeyer]

Replying to SimonKing:

Can you test on marks and t2 with unpatched Sage whether the line

sage: singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')

fails?

Unfortunately, I cannot. I am not able to compile Sage on marks and I do not have an account on t2.

[SimonKing]

I was able to build sage-4.7.1.rc1 on mark (which is supposed to be identical with mark2). It turns out that the bug was not introduced but revealed by my patch.

With the freshly built Sage-on-mark, I get

sage: singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)

/home/simonking/SAGE/sage-4.7.1.rc1mark/<ipython console> in <module>()

/home/simonking/SAGE/sage-4.7.1.rc1mark/local/lib/python2.6/site-packages/sage/interfaces/singular.pyc in eval(self, x, allow_semicolon, strip, **kwds)
    548
    549         if s.find("error") != -1 or s.find("Segment fault") != -1:
--> 550             raise RuntimeError, 'Singular error:\n%s'%s
    551
    552         if get_verbose() > 0:

RuntimeError: Singular error:
   ? cannot open `gftables/9`
   ? cannot make ring
   ? error occurred in or before STDIN line 25: `ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp);`
   ? expected ring-expression. type 'help ring;'

The question is: How shall we proceed? Removing that particular test would be a shame, since it shows that my patch can manage to parse pretty complicated stuff. Perhaps it is possible to simplify it, so that it is still complicated, but works on mark and t2.

In either case, I will report on sage-solaris, and probably will file a bug report to the Singular team as well.

[SimonKing]

For the record: I reported the problem on the ​singular trac.

[SimonKing]

Replying to SimonKing:

For the record: I reported the problem on the ​singular trac.

... and I opened #11645.

[SimonKing]

As I have indicated on #11645, there is a potential work around for the missing gftables: If one has a polynomial ring over a finite non-prime field in Sage and converts it into Singular, then the missing gftable is created. Hence, one could ensure that the problematic doc test passes by repending that test with another one that converts a ring from Sage into Singular.

That would be a nasty work-around, but at least it would mean that my patch could be used.

[SimonKing]

Work around a bug on Solaris revealed by the first patch

[SimonKing]

I provided a third patch that uses - as a workaround - the fact that libsingular is able to create missing gftables files.

On Singular trac, Hannes said "No, gftables are parts of the sources and will be copied during installation, but usually not computed (the program for that, a part of factory, will not be build during a standard compilation and installation).

So, I wonder why the files aren't copied on Solaris and how libsingular is able to create them. But I think the third patch (that isn't tested, yet) should suffice to make the doc tests of the first two patches work on solaris.

Back to "needs review" - I hope that someone can doctest on Solaris.

[SimonKing]

Note that #11645 also fixes the problem with the new doc test.

So, if someone would like to review my patches, I see two ways to make things work:

Either

1. get #11645 and trac11431_singular_sage_conversion.patch​ and trac11431_singular_sage_documentation.patch​

or

2. get trac11431_singular_sage_conversion.patch​, trac11431_singular_sage_documentation.patch​ and trac11431_solaris_workaround.patch​.

[jdemeyer]

What's the status of this ticket now? Since #11645 has been merged in sage-4.7.1, we can assume that as prerequisite. Which patches should be applied then?

If somebody can give these patches a "tentative positive review" (without testing on Solaris), I can test them on the buildbot machines, including Solaris.

[jdemeyer]

Replying to jdemeyer:

If somebody can give these patches a "tentative positive review" (without testing on Solaris), I can test them on the buildbot machines, including Solaris.

To the reviewer: if you do give such a "tentative positive review", you can set the status to "positive review" (even if it has not been tested properly).

[malb]

Simon, can you rebase your patches to 4.7.1?

[SimonKing]

Replying to malb:

Simon, can you rebase your patches to 4.7.1?

Since #11645 got fixed, I have only applied the first two patches, to sage-4.7.2.alpha1. They applied without fuzz. Why is it needed to rebase them?

[malb]

Hi, I get this:

SAGE_ROOT=/mnt/usb1/scratch/malb/sage-4.7.2.alpha0
(sage subshell) sage:sage malb$ hg qimport http://trac.sagemath.org/sage_trac/raw-attachment/ticket/11431/trac11431_singular_sage_conversion.patch && hg qpush
adding trac11431_singular_sage_conversion.patch to series file
applying trac11431_singular_sage_conversion.patch
patching file sage/rings/polynomial/term_order.py
Hunk #1 FAILED at 1857
1 out of 1 hunks FAILED -- saving rejects to file sage/rings/polynomial/term_order.py.rej
patch failed, unable to continue (try -v)
patch failed, rejects left in working dir
errors during apply, please fix and refresh trac11431_singular_sage_conversion.patch
SAGE_ROOT=/mnt/usb1/scratch/malb/sage-4.7.2.alpha0

[SimonKing]

Hi Martin,

you forgot the other dependency: #11316 was only merged in sage-4.7.2.alpha1, but you started with alpha0.

[malb]

Right, now it applies!

[malb]

applies & doctests pass. I read the patch a while ago so positive review it is.

[leif]

Simon, are you ok with dropping the (IMHO obsolete) Solaris patch?

[SimonKing]

Replying to leif:

Simon, are you ok with dropping the (IMHO obsolete) Solaris patch?

Yes, it can be dropped, since that problem has been successfully dealt with on a different ticket.