Opened 8 years ago

Closed 8 years ago

#11431 closed defect (fixed)

Conversion from Singular to Sage

Reported by: SimonKing Owned by: was
Priority: major Milestone: sage-4.7.2
Component: interfaces Keywords:
Cc: malb Merged in: sage-4.7.2.alpha3
Authors: Simon King Reviewers: Martin Albrecht
Report Upstream: None of the above - read trac for reasoning. Work issues:
Branch: Commit:
Dependencies: #11316 #11645 Stopgaps:

Description (last modified by leif)

On sage-devel, Francisco Botana complained about some shortcomings of the conversion from Singular (pexpect interface) to Sage.

I think the conversions provided by this patch are quite thorough.

First of all, the patch provides a conversion of base rings, even with minpoly, with complicated block, matrix and weighted orders (note that one needs #11316) and even quotient rings:

sage: singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')
'ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp);'
sage: R = singular('r1').sage_basering()
sage: R
Multivariate Polynomial Ring in a, b, c, d, e, f over Finite Field in x of size 3^2
sage: R.term_order()
Block term order with blocks:
(Matrix term order with matrix
[1 2]
[3 0],
 Weighted degree reverse lexicographic term order with weights (2, 3),
 Lexicographic term order of length 2)

sage: singular.eval('ring r3 = (3,z),(a,b,c),dp')
'ring r3 = (3,z),(a,b,c),dp;'
sage: singular.eval('minpoly = 1+z+z2+z3+z4')
'minpoly = 1+z+z2+z3+z4;'
sage: singular('r3').sage_basering()
Multivariate Polynomial Ring in a, b, c over Univariate Quotient Polynomial Ring in z over Finite Field of size 3 with modulus z^4 + z^3 + z^2 + z + 1

sage: singular.eval('ring r5 = (9,a), (x,y,z),lp') 
'ring r5 = (9,a), (x,y,z),lp;'
sage: Q = singular('std(ideal(x^2,x+y^2+z^3))', type='qring')
sage: Q.sage_basering()
Quotient of Multivariate Polynomial Ring in x, y, z over Finite Field in a of size 3^2 by the ideal (y^4 - y^2*z^3 + z^6, x + y^2 + z^3)

By consequence, it is now straight forward to convert polynomials or ideals to sage:

sage: singular.eval('ring R = integer, (x,y,z),lp') 
'// ** You are using coefficient rings which are not fields...'
sage: I = singular.ideal(['x^2','y*z','z+x']) 
sage: I.sage()  # indirect doctest
Ideal (x^2, y*z, x + z) of Multivariate Polynomial Ring in x, y, z over Integer Ring

# Note that conversion of a Singular string to a Sage string was missing
sage: singular('ringlist(basering)').sage()
[['integer'], ['x', 'y', 'z'], [['lp', (1, 1, 1)], ['C', (0)]], Ideal (0) of Multivariate Polynomial Ring in x, y, z over Integer Ring]

sage: singular.eval('ring r10 = (9,a), (x,y,z),lp')
'ring r10 = (9,a), (x,y,z),lp;'
sage: Q = singular('std(ideal(x^2,x+y^2+z^3))', type='qring')
sage: Q.sage()
Quotient of Multivariate Polynomial Ring in x, y, z over Finite Field in a of size 3^2 by the ideal (y^4 - y^2*z^3 + z^6, x + y^2 + z^3)
sage: singular('x^2+y').sage()
x^2 + y
sage: singular('x^2+y').sage().parent()
Quotient of Multivariate Polynomial Ring in x, y, z over Finite Field in a of size 3^2 by the ideal (y^4 - y^2*z^3 + z^6, x + y^2 + z^3)

Apply

  1. trac11431_singular_sage_conversion.patch
  2. trac11431_singular_sage_documentation.patch

to the Sage library.

Attachments (3)

trac11431_singular_sage_conversion.patch (15.8 KB) - added by SimonKing 8 years ago.
Conversion Singular -> Sage
trac11431_singular_sage_documentation.patch (11.1 KB) - added by SimonKing 8 years ago.
Show examples of .sage() to the singular interface tutorial. Fix some details of the previous patch.
trac11431_solaris_workaround.patch (2.7 KB) - added by SimonKing 8 years ago.
Work around a bug on Solaris revealed by the first patch

Download all attachments as: .zip

Change History (33)

comment:1 Changed 8 years ago by SimonKing

  • Status changed from new to needs_review

Note to the reviewer: I did not check whether the doc strings look nice in html.

Changed 8 years ago by SimonKing

Conversion Singular -> Sage

comment:2 Changed 8 years ago by SimonKing

So far, the method sage() has not been mentioned at all in the documentation of sage.interfaces.singular! Therefore I added a second patch, providing some use cases of .sage() in the short tutorial of sage.interfaces.singular.

The examples that are visible in the html version of the documentation still do not cover all features (quotient rings etc.), but I guess if a user knows about the existence of sage() (s)he will guess its correct usage.

comment:3 follow-up: Changed 8 years ago by malb

  • Reviewers set to Martin Albrecht
  • Status changed from needs_review to needs_work
  • can we rename sage_basering to sage_global_ring. In Sage base_ring has a special meaning which is not meant in your patch. I think global ring captures it.
  • don't real fields also have a precision in Singular?
  • Previously, we enforced and assumed not short, but it's good you're taking it into account.
  • Can you also fix "If it is provided, then you *must* take care OFF it."?
  • shouldn't TermOrder_from_Singular be lower-case since it's a function? Or were you aiming for upper-case because it's a constructor?
  • In "Invalid termorder in Singular" shouldn't it be term order, i.e. with a space?

comment:4 in reply to: ↑ 3 ; follow-up: Changed 8 years ago by SimonKing

Hi Martin!

Replying to malb:

  • can we rename sage_basering to sage_global_ring. In Sage base_ring has a special meaning which is not meant in your patch. I think global ring captures it.

Sounds fine to me. Obviously I took the name sage_basering from Singular, but you are right that it might be confusing for people who know the notion base ring only from Sage.

  • don't real fields also have a precision in Singular?

Right, that should be taken care of as well.

  • Can you also fix "If it is provided, then you *must* take care OFF it."?

Why "off"? With two f?

Wouldn't it be correct English to say "If it is provided, then you must take care that it matches the current ring in Singular"? (i.e., I forgot the "that" in my patch)

  • shouldn't TermOrder_from_Singular be lower-case since it's a function? Or were you aiming for upper-case because it's a constructor?

I made it upper case because it returns a TermOrder (that is how the class is called). But actually I was ignorant to the convention that functions are lower-case. So I'll change it.

  • In "Invalid termorder in Singular" shouldn't it be term order, i.e. with a space?

Yep!

comment:5 in reply to: ↑ 4 Changed 8 years ago by SimonKing

Replying to SimonKing:

...

  • Can you also fix "If it is provided, then you *must* take care OFF it."?

Why "off"? With two f?

Wouldn't it be correct English to say "If it is provided, then you must take care that it matches the current ring in Singular"? (i.e., I forgot the "that" in my patch)

Perhaps not. But "If it is provided, then you have to make sure that it matches..." is fine, isn't it?

  • shouldn't TermOrder_from_Singular be lower-case since it's a function? Or were you aiming for upper-case because it's a constructor?

I made it upper case because it returns a TermOrder (that is how the class is called). But actually I was ignorant to the convention that functions are lower-case. So I'll change it.

  • In "Invalid termorder in Singular" shouldn't it be term order, i.e. with a space?

Yep!

Changed 8 years ago by SimonKing

Show examples of .sage() to the singular interface tutorial. Fix some details of the previous patch.

comment:6 Changed 8 years ago by SimonKing

  • Status changed from needs_work to needs_review

I've put my changes into the second patch, so, please replace that when you test it.

Names: I changed sage_basering into sage_global_ring and TermOrder_from_Singular into termorder_from_singular.

Precision: Singular does not tell the precision of a real field in charstr(), it does so only for complex fields. But the precision both of real and complex fields is part of ringlist(). So, both in the real and the complex case, I take the precision from there, not from charstr().

Grammar: It is now "Invalid term order" (not "termorder") and "If it is provided, then you have to make sure that it matches..." (not "If it is provided, then you *must* take care it matches...")

Tests: Of course I added a test showing that the precision of real fields is taken care of. And it turned out that one doctest needed a fix, because when I created it I used a different term order (so that the same polynomial was printed in a different way).

comment:7 Changed 8 years ago by malb

  • Status changed from needs_review to positive_review

I'll give this a positive review: I agree with Simon's fixes (and the patchbot can take care of any doctest failures etc.)

comment:8 Changed 8 years ago by SimonKing

Thank you! And the patchbot actually says that the doctests pass...

comment:9 Changed 8 years ago by jdemeyer

  • Merged in set to sage-4.7.1.alpha3
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:10 follow-up: Changed 8 years ago by jdemeyer

  • Merged in sage-4.7.1.alpha3 deleted
  • Resolution fixed deleted
  • Status changed from closed to new

There are some issues on Solaris. On the "marks" and "t2" buildbots, I get the following error (identical on both systems):

sage -t -long  -force_lib devel/sage/sage/rings/polynomial/term_order.py
**********************************************************************
File "/scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.1.alpha3/devel/sage-main/sage/rings/polynomial/term_order.py", line 1878:
    sage: singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')
Exception raised:
    Traceback (most recent call last):
      File "/scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.1.alpha3/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.1.alpha3/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.1.alpha3/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_64[2]>", line 1, in <module>
        singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')###line 1878:
    sage: singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')
      File "/scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.1.alpha3/local/lib/python/site-packages/sage/interfaces/singular.py", line 567, in eval
        raise RuntimeError, 'Singular error:\n%s'%s
    RuntimeError: Singular error:
       ? cannot open `gftables/9`
       ? cannot make ring
       ? error occurred in or before STDIN line 33: `ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp);`
       ? expected ring-expression. type 'help ring;'
**********************************************************************
File "/scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.1.alpha3/devel/sage-main/sage/rings/polynomial/term_order.py", line 1881:
    sage: termorder_from_singular(singular)
Expected:
    Block term order with blocks:
    (Matrix term order with matrix
    [1 2]
    [3 0],
     Weighted degree reverse lexicographic term order with weights (2, 3),
     Lexicographic term order of length 2)
Got:
    Block term order with blocks:
    (Lexicographic term order of length 3,
     Degree lexicographic term order of length 5,
     Lexicographic term order of length 2)
**********************************************************************

comment:11 in reply to: ↑ 10 ; follow-up: Changed 8 years ago by SimonKing

Hi Jeroen,

Replying to jdemeyer:

sage: singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')

...

RuntimeError?: Singular error:

? cannot open gftables/9 ? cannot make ring ? error occurred in or before STDIN line 33: ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp); ? expected ring-expression. type 'help ring;'

That is very strange. I can not imagine that my patch introduced that bug: It changes the conversion from Singular to Sage, but it should not change Singular itself (and what you get in your example is an error from Singular, not from the interface).

Hence, I wonder whether my patch has introduced or uncovered that bug.

Can you test on marks and t2 with unpatched Sage whether the line

sage: singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')

fails?

comment:12 in reply to: ↑ 11 Changed 8 years ago by jdemeyer

  • Status changed from new to needs_info

Replying to SimonKing:

Can you test on marks and t2 with unpatched Sage whether the line

sage: singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')

fails?

Unfortunately, I cannot. I am not able to compile Sage on marks and I do not have an account on t2.

comment:13 Changed 8 years ago by SimonKing

I was able to build sage-4.7.1.rc1 on mark (which is supposed to be identical with mark2). It turns out that the bug was not introduced but revealed by my patch.

With the freshly built Sage-on-mark, I get

sage: singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)

/home/simonking/SAGE/sage-4.7.1.rc1mark/<ipython console> in <module>()

/home/simonking/SAGE/sage-4.7.1.rc1mark/local/lib/python2.6/site-packages/sage/interfaces/singular.pyc in eval(self, x, allow_semicolon, strip, **kwds)
    548
    549         if s.find("error") != -1 or s.find("Segment fault") != -1:
--> 550             raise RuntimeError, 'Singular error:\n%s'%s
    551
    552         if get_verbose() > 0:

RuntimeError: Singular error:
   ? cannot open `gftables/9`
   ? cannot make ring
   ? error occurred in or before STDIN line 25: `ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp);`
   ? expected ring-expression. type 'help ring;'

The question is: How shall we proceed? Removing that particular test would be a shame, since it shows that my patch can manage to parse pretty complicated stuff. Perhaps it is possible to simplify it, so that it is still complicated, but works on mark and t2.

In either case, I will report on sage-solaris, and probably will file a bug report to the Singular team as well.

comment:14 follow-up: Changed 8 years ago by SimonKing

  • Report Upstream changed from N/A to Reported upstream. Little or no feedback.

For the record: I reported the problem on the singular trac.

comment:15 in reply to: ↑ 14 Changed 8 years ago by SimonKing

Replying to SimonKing:

For the record: I reported the problem on the singular trac.

... and I opened #11645.

comment:16 Changed 8 years ago by SimonKing

  • Work issues set to Singular trouble on Solaris

As I have indicated on #11645, there is a potential work around for the missing gftables: If one has a polynomial ring over a finite non-prime field in Sage and converts it into Singular, then the missing gftable is created. Hence, one could ensure that the problematic doc test passes by repending that test with another one that converts a ring from Sage into Singular.

That would be a nasty work-around, but at least it would mean that my patch could be used.

Changed 8 years ago by SimonKing

Work around a bug on Solaris revealed by the first patch

comment:17 Changed 8 years ago by SimonKing

  • Description modified (diff)
  • Report Upstream changed from Reported upstream. Little or no feedback. to None of the above - read trac for reasoning.
  • Status changed from needs_info to needs_review
  • Work issues Singular trouble on Solaris deleted

I provided a third patch that uses - as a workaround - the fact that libsingular is able to create missing gftables files.

On Singular trac, Hannes said "No, gftables are parts of the sources and will be copied during installation, but usually not computed (the program for that, a part of factory, will not be build during a standard compilation and installation).

So, I wonder why the files aren't copied on Solaris and how libsingular is able to create them. But I think the third patch (that isn't tested, yet) should suffice to make the doc tests of the first two patches work on solaris.

Back to "needs review" - I hope that someone can doctest on Solaris.

comment:18 Changed 8 years ago by SimonKing

Note that #11645 also fixes the problem with the new doc test.

So, if someone would like to review my patches, I see two ways to make things work:

Either

  1. get #11645 and trac11431_singular_sage_conversion.patch and trac11431_singular_sage_documentation.patch

or

  1. get trac11431_singular_sage_conversion.patch, trac11431_singular_sage_documentation.patch and trac11431_solaris_workaround.patch.

comment:19 follow-up: Changed 8 years ago by jdemeyer

  • Dependencies changed from #11316 to #11316, #11645

What's the status of this ticket now? Since #11645 has been merged in sage-4.7.1, we can assume that as prerequisite. Which patches should be applied then?

If somebody can give these patches a "tentative positive review" (without testing on Solaris), I can test them on the buildbot machines, including Solaris.

comment:20 in reply to: ↑ 19 Changed 8 years ago by jdemeyer

Replying to jdemeyer:

If somebody can give these patches a "tentative positive review" (without testing on Solaris), I can test them on the buildbot machines, including Solaris.

To the reviewer: if you do give such a "tentative positive review", you can set the status to "positive review" (even if it has not been tested properly).

comment:21 follow-up: Changed 8 years ago by malb

Simon, can you rebase your patches to 4.7.1?

comment:22 in reply to: ↑ 21 Changed 8 years ago by SimonKing

Replying to malb:

Simon, can you rebase your patches to 4.7.1?

Since #11645 got fixed, I have only applied the first two patches, to sage-4.7.2.alpha1. They applied without fuzz. Why is it needed to rebase them?

comment:23 Changed 8 years ago by malb

Hi, I get this:

SAGE_ROOT=/mnt/usb1/scratch/malb/sage-4.7.2.alpha0
(sage subshell) sage:sage malb$ hg qimport http://trac.sagemath.org/sage_trac/raw-attachment/ticket/11431/trac11431_singular_sage_conversion.patch && hg qpush
adding trac11431_singular_sage_conversion.patch to series file
applying trac11431_singular_sage_conversion.patch
patching file sage/rings/polynomial/term_order.py
Hunk #1 FAILED at 1857
1 out of 1 hunks FAILED -- saving rejects to file sage/rings/polynomial/term_order.py.rej
patch failed, unable to continue (try -v)
patch failed, rejects left in working dir
errors during apply, please fix and refresh trac11431_singular_sage_conversion.patch
SAGE_ROOT=/mnt/usb1/scratch/malb/sage-4.7.2.alpha0

comment:24 Changed 8 years ago by SimonKing

Hi Martin,

you forgot the other dependency: #11316 was only merged in sage-4.7.2.alpha1, but you started with alpha0.

comment:25 Changed 8 years ago by malb

Right, now it applies!

comment:26 Changed 8 years ago by malb

  • Status changed from needs_review to positive_review

applies & doctests pass. I read the patch a while ago so positive review it is.

comment:27 follow-up: Changed 8 years ago by leif

  • Description modified (diff)

Simon, are you ok with dropping the (IMHO obsolete) Solaris patch?

comment:28 in reply to: ↑ 27 Changed 8 years ago by SimonKing

Replying to leif:

Simon, are you ok with dropping the (IMHO obsolete) Solaris patch?

Yes, it can be dropped, since that problem has been successfully dealt with on a different ticket.

comment:29 Changed 8 years ago by leif

  • Dependencies changed from #11316, #11645 to #11316 #11645

comment:30 Changed 8 years ago by leif

  • Merged in set to sage-4.7.2.alpha3
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.