Problems interrupting Singular

[jdemeyer]

Type the following:

sage: singular._expect_expr("1")

Now type CTRL-C. It takes a very long time before something happens. This was the cause of #9163 (which was fixed to avoid this issue).

---

Upstream bug: ​http://www.singular.uni-kl.de:8002/trac/ticket/730

[jdemeyer]

I suspect this is behind it, start Singular and press CTRL-C:

$ sage -singular
                     SINGULAR                             /  Development
 A Computer Algebra System for Polynomial Computations   /   version 3-1-1
                                                       0<
     by: G.-M. Greuel, G. Pfister, H. Schoenemann        \   Feb 2010
FB Mathematik der Universitaet, D-67653 Kaiserslautern    \
> ^C// ** Interrupt at cmd:`$INVALID$` in line:''
abort command(a), continue(c) or quit Singular(q) ?

So we get a stupid prompt instead.

[vbraun]

I find that it takes about 5 seconds, not "very long". On a closer look, it actually takes 6 seconds = 20*0.3:

Definition:     singular.interrupt(self, tries=20, timeout=0.29999999999999999, quit_on_fail=True)
Source:
    def interrupt(self, tries=20, timeout=0.3, quit_on_fail=True):
        E = self._expect
        if E is None:
            return True
        success = False
        try:
            for i in range(tries):
                E.sendline(self._quit_string())
                E.sendline(chr(3))
                try:
                    E.expect(self._prompt, timeout=timeout)
                    success= True
                    break
                except (pexpect.TIMEOUT, pexpect.EOF), msg:
                    #print msg
                    pass
        except Exception, msg:
            pass
        if success:
            pass
        elif quit_on_fail:
            self.quit()
        return success

• The inherited method interrupt() actually tries to quit Singular (self._quit_string()=="quit")
• If quitting did not work the first time, is it going to work the next 19 times?
• From personal experience, I find that actually interrupting (using ctrl-c and "continue") Singular often does not work. Restarting is very fast, though.

I propose that the Singular pexpect interface override interrupt() to be a synonym for quit().

[jdemeyer]

I'm having a hard time fixing this with pexpect, it's too unreliable. Next strategy is patching Singular not to show the stupid interrupt prompt...

[jdemeyer]

Diff for the singular spkg. For reference / review only.

[vbraun]

Running the doctest in a loop, I get occasional (maybe 1 in 20) failures

sage -t -force_lib "devel/sage/sage/interfaces/singular.py" 
**********************************************************************
File "/home/vbraun/opt/sage-5.1.beta5/devel/sage/sage/interfaces/singular.py", line 480:
    sage: 2*a
Exception raised:
    Traceback (most recent call last):
      File "/home/vbraun/opt/sage-5.1.beta5/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/home/vbraun/opt/sage-5.1.beta5/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/home/vbraun/opt/sage-5.1.beta5/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_9[6]>", line 1, in <module>
        Integer(2)*a###line 480:
    sage: 2*a
      File "element.pyx", line 1687, in sage.structure.element.RingElement.__mul__ (sage/structure/element.c:12755)
      File "coerce.pyx", line 749, in sage.structure.coerce.CoercionModel_cache_maps.bin_op (sage/structure/coerce.c:6778)
      File "coerce.pyx", line 745, in sage.structure.coerce.CoercionModel_cache_maps.bin_op (sage/structure/coerce.c:6718)
      File "element.pyx", line 1682, in sage.structure.element.RingElement.__mul__ (sage/structure/element.c:12681)
      File "element.pyx", line 1689, in sage.structure.element.RingElement._mul_ (sage/structure/element.c:12807)
      File "/home/vbraun/opt/sage-5.1.beta5/local/lib/python/site-packages/sage/interfaces/interface.py", line 1118, in _mul_
        return self._operation('*', right)
      File "/home/vbraun/opt/sage-5.1.beta5/local/lib/python/site-packages/sage/interfaces/interface.py", line 1075, in _operation
        raise TypeError, msg
    TypeError: Singular error:
       ? `sage449` is not defined
       ? error occurred in or before STDIN line 11: `def sage458=sage450 * sage449;`
**********************************************************************
1 items had failures:
   1 of   8 in __main__.example_9

[jdemeyer]

Well, improving the function from working less than 5% of the time to about 95% of the time isn't bad :-)

But you're right that this needs work. Since I will have to make some more serious changes, let's not do it anymore in sage-5.1.

[rws]

Replying to vbraun:

Running the doctest in a loop, I get occasional (maybe 1 in 20) failures

This is a good estimate of the rate when the issue leads to patchbot false test failures. But #13237 was fixed 21 months ago, so obviously they too didn't fix it completely.

[rws]

After 6.3beta2 the rate of failure under Ubuntu+patchbot (both SageMath/Google cloud) has gone up significantly to the order of 50%.

File "src/sage/interfaces/expect.py", line 790, in sage.interfaces.expect.Expect._eval_line
Failed example:
    singular.interrupt(timeout=3)  # sometimes very slow (up to 60s on sage.math, 2012)
Expected:
    False
Got:
    True

[vbraun]

#15178 might improve things a bit (will be in beta3)

[jdemeyer]

Replying to vbraun:

#15178 might improve things a bit (will be in beta3)

I don't see how that could be related.

[vbraun]

Since the cntrlc=a patch is now in upstream we should at least make use of the command line option...

[jakobkroeker]

Since the cntrlc=a patch is now in upstream we should at least make use of the command line option

That should be easy now, since the patch is in 3.1.7 and 3.1.7 is in sage! I just do not know where this change has to be done.

Following choices for --cntrlc= are possible: a,r,b,c,q :

abort after this command(a), abort immediately(r), print backtrace(b), continue(c) or quit Singular(q) ?

[jdemeyer]

I'm in pexpect mood anyway (#17686, #17704), so I might as well have a look at this.

[chapoton]

New commits:

​ac310c7

Fix interrupting Singular

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​1d3289c	Merge branch 'public/ticket/10476' into 6.7
​79bce4d	trac #10476 doc formatting

[chapoton]

There is one failing doctest in the singular interface.

[jdemeyer]

I would rather have #17686 in Sage first before continuing on this ticket.

[vbraun]

This seems to be much more frequent now with #17686

sage -t --long src/sage/interfaces/expect.py
**********************************************************************
File "src/sage/interfaces/expect.py", line 823, in sage.interfaces.expect.Expect._eval_line
Failed example:
    singular.interrupt(timeout=3)  # sometimes very slow (up to 60s on sage.math, 2012)
Expected:
    False
Got:
    True
**********************************************************************
1 item had failures:
   1 of  15 in sage.interfaces.expect.Expect._eval_line
    [89 tests, 1 failure, 4.74 s]

[jdemeyer]

Good, so we're making progress!

The correct result of that doctest is True.

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​42197f6

Fix interrupting Singular

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​aa4a156

Fix interrupting Singular

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​45969a6	Clean up interrupt()
​a4f2d83	Fix interrupting Singular

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​17d23e9

Fix interrupting Singular

[kdilks]

Minor doc error:

OSError: [interface] docstring of sage.interfaces.sagespawn.SageSpawn.expect_peek:12: WARNING: Block quote ends without a blank line; unexpected unindent.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​02decd8

Fix documentation

[kdilks]

The documentation builds now.

Not a thorough review of the code, but when repeatedly doctesting the file ./src/sage/interfaces/expect.py on 6.8beta5, I can get singular.interrupt(timeout=3) to give an error every hundred or so times. With this patch applied, I've all had all tests pass a few thousand times.

[wluebbe]

I tried ./sage -t src/sage/interfaces/expect.py on sage-6.8.beta7 and Ubuntu 15.04 a dozen times: it failed in about 50% of the runs (seemingly at random).

[kdilks]

Upstream bug ticket has been updated.

[vbraun]

Well they don't really deny its a bug, just difficult to fix in the current architecture...

[jdemeyer]

Replying to vbraun:

Well they don't really deny its a bug, just difficult to fix in the current architecture...

I sort of agree, but they did say it's considered a "proposed feature".