Opened 5 years ago

Last modified 6 weeks ago

#16688 new enhancement

replace pexpect interface calls with library where possible

Reported by: rws Owned by:
Priority: major Milestone: sage-wishlist
Component: interfaces Keywords: expect, pexpect, library
Cc: fbissey, frederichan, leif, slelievre Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by slelievre)

This meta-ticket tracks efforts to replace pexpect interface calls with library calls, in order to minimize the usage of expect for time-critical code. Packages used by Sage where this seems possible include

The following packages are used via pexpect in the Sage library (noninteractively) and these calls should be replaced somehow in the long term (this is a wishlist):

  • Axiom (through fricas)
  • giac
  • kash
  • Macaulay2
  • Octave
  • qepcad
  • Scilab (two unused functions in matrix/matrix1.pyx)

Change History (10)

comment:1 follow-up: Changed 5 years ago by fbissey

  • Cc fbissey added

The notebook also uses pexpect. I am not sure there is much of an alternative there but at least we should try to move it to a more modern pexpect. Trying to do so results in breakages that I have no idea how to solve currently.

comment:2 Changed 5 years ago by frederichan

  • Cc frederichan added

comment:3 Changed 5 years ago by was

Further motivation for this:

sage: scilab(str(range(20)))  # works
sage: scilab(str(range(30)))  # BOOM!

(Note -- scilab(range(20)) doesn't work but should!)

And also

sage: r(range(30))  # works
sage: r(range(10000)) # boom

Worksheet with the above: https://cloud.sagemath.com/projects/4a5f0542-5873-4eed-a85c-a18c706e8bcd/files/support/2015-02-04-062417-scilab-limits.sagews

These sorts of problems were reported by many people over the years, most recently by JC Boulet of INRA.

comment:4 Changed 5 years ago by jdemeyer

I would say it's a motivation to improve the pexpect interfaces (see for example #17704, #17718, #17686, #17719).

I claim that such interfaces can be made to work, it's just a mess in Sage.

comment:5 in reply to: ↑ 1 ; follow-up: Changed 5 years ago by nbruin

Replying to fbissey:

The notebook also uses pexpect. I am not sure there is much of an alternative there but at least we should try to move it to a more modern pexpect. Trying to do so results in breakages that I have no idea how to solve currently.

There is, especially when you're programming both sides. You can have two processes communicate through a pipe or stream without one of the parties pretending to be a terminal.

That said, the fact that the notebook *does* use pexpect makes it easier to use it to interface with R, octave, magma, etc, so we do get some benefit from it.

comment:6 in reply to: ↑ 5 Changed 5 years ago by was

Replying to nbruin:

Replying to fbissey:

The notebook also uses pexpect. I am not sure there is much of an alternative there but at least we should try to move it to a more modern pexpect. Trying to do so results in breakages that I have no idea how to solve currently.

There is, especially when you're programming both sides. You can have two processes communicate through a pipe or stream without one of the parties pretending to be a terminal.

I completely agree. IPython and also https://github.com/sagemath/cloud/blob/master/sage_server.py are both examples of better ways of communication between two processes without using pexpect. Basically, you setup a traditional client/server and a messaging protocol. Python and many other languages makes that sort of thing pretty easy. But... it would be harder to implement something like that in a special purpose system like Pari or Magma. It might be impossible with Magma without source code access (not sure what the status of networking is now).

That said, the fact that the notebook *does* use pexpect makes it easier to use it to interface with R, octave, magma, etc, so we do get some benefit from it.

As the author of exactly the part of the notebook that uses pexpect, etc., I don't agree with this statement so much -- instead, the two are somewhat orthogonal. The connection (1) "notebook_server <--> python processes" is done using pexpect in sagenb. The interfaces to R/octave/magma, etc. are connections (2) "python processes <--> R/octave/magma/etc.". You can certainly swap out (1) for a connection built using TCP (rather than pexpect) with no impact on (2); in fact, that's precisely what I did with SageMathCloud?.

That said, maybe you just meant that work on the abstract base class Expect for the interfaces benefits both sagenb and the R/octave/magma, etc., interfaces...

comment:7 Changed 5 years ago by jdemeyer

I do agree that when alternatives to pexpect are available, we should use the alternatives. But for the programs which don't offer alternatives, pexpect should be able to work.

comment:8 Changed 5 years ago by rws

  • Description modified (diff)

comment:9 Changed 3 years ago by leif

  • Cc leif added

comment:10 Changed 6 weeks ago by slelievre

  • Cc slelievre added
  • Description modified (diff)
Note: See TracTickets for help on using tickets.