#22833 closed defect (fixed)

fix a calculus doctest (giac, laplace, integration)

Reported by: chapoton Owned by:
Priority: major Milestone: sage-8.0
Component: calculus Keywords:
Cc: rws, frederichan Merged in:
Authors: Frédéric Chapoton, Marcelo Forets Reviewers: Travis Scrimshaw, Steven Trogdon
Report Upstream: N/A Work issues:
Branch: e2d74c6 (Commits) Commit: e2d74c645b48c3288802f9080d6651aaccf658fb
Dependencies: Stopgaps:

Description (last modified by chapoton)

that currently prevents many patchbots to run smoothly

Change History (47)

comment:1 Changed 22 months ago by chapoton

  • Branch set to u/chapoton/22833
  • Status changed from new to needs_review

comment:2 Changed 22 months ago by git

  • Commit set to 2186acae2da3826b0a29d196d6b22b8fbc1dad4b

Branch pushed to git repo; I updated commit sha1. New commits:

2186acafixing calculus doctest (giac)

comment:3 Changed 22 months ago by tscrim

  • Reviewers set to Travis Scrimshaw
  • Status changed from needs_review to positive_review

I thought someone had already fixed this, but I guess not.

comment:4 Changed 22 months ago by mforets

the error was because of the double :: in TETS or the Unable to parse Giac output? or both? and how did you know that integration is ok? (i still get the NotImplementedError in sage8.0beta2) because the "dummy integrate" keyword of maxima is integrate.

comment:5 Changed 22 months ago by chapoton

The change to TESTS:: was just something I could not keep like that once I had seen it.

I am not sure that the change to integration doctest is ok. If it fails on your machine, then you can set this ticket back to needs work.

comment:6 Changed 22 months ago by mforets

  • Status changed from positive_review to needs_work

failing in my machine with:

Doctesting 1 file.
sage -t src/sage/calculus/calculus.py
**********************************************************************
File "src/sage/calculus/calculus.py", line 1365, in sage.calculus.calculus.laplace
Failed example:
    (p1+p2).save(os.path.join(SAGE_TMP, "de_plot.png"))
Expected nothing
Got:
    doctest:warning
      File "/Users/forets/sage-src/sage/src/bin/sage-runtests", line 89, in <module>
        err = DC.run()
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/sage/doctest/control.py", line 1134, in run
        self.run_doctests()
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/sage/doctest/control.py", line 858, in run_doctests
        self.dispatcher.dispatch()
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 1705, in dispatch
        self.parallel_dispatch()
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 1595, in parallel_dispatch
        w.start()  # This might take some time
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 1871, in start
        super(DocTestWorker, self).start()
      File "/Users/forets/sage-src/sage/local/lib/python2.7/multiprocessing/process.py", line 130, in start
        self._popen = Popen(self)
      File "/Users/forets/sage-src/sage/local/lib/python2.7/multiprocessing/forking.py", line 126, in __init__
        code = process_obj._bootstrap()
      File "/Users/forets/sage-src/sage/local/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
        self.run()
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 1844, in run
        task(self.options, self.outtmpfile, msgpipe, self.result_queue)
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 2137, in __call__
        runner.run(test)
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 641, in run
        return self._run(test, compileflags, out)
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 503, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 866, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.calculus.calculus.laplace[23]>", line 1, in <module>
        (p1+p2).save(os.path.join(SAGE_TMP, "de_plot.png"))
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/sage/misc/decorators.py", line 471, in wrapper
        return func(*args, **kwds)
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/sage/plot/graphics.py", line 3170, in save
        figure = self.matplotlib(**options)
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/sage/plot/graphics.py", line 2566, in matplotlib
        from matplotlib.figure import Figure
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/matplotlib/figure.py", line 38, in <module>
        import matplotlib.colorbar as cbar
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/matplotlib/colorbar.py", line 34, in <module>
        import matplotlib.collections as collections
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/matplotlib/collections.py", line 27, in <module>
        import matplotlib.backend_bases as backend_bases
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/matplotlib/backend_bases.py", line 62, in <module>
        import matplotlib.textpath as textpath
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/matplotlib/textpath.py", line 15, in <module>
        import matplotlib.font_manager as font_manager
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/matplotlib/font_manager.py", line 1413, in <module>
        fontManager = pickle_load(_fmcache)
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/matplotlib/font_manager.py", line 965, in pickle_load
        data = pickle.load(fh)
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/matplotlib/font_manager.py", line 1044, in __init__
        self.ttffiles = findSystemFonts(paths) + findSystemFonts()
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/matplotlib/font_manager.py", line 324, in findSystemFonts
        for f in get_fontconfig_fonts(fontext):
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/matplotlib/font_manager.py", line 273, in get_fontconfig_fonts
        warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
    :
    UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
**********************************************************************
File "src/sage/calculus/calculus.py", line 1407, in sage.calculus.calculus.laplace
Failed example:
    laplace(t^n, t, s, algorithm='giac')
Exception raised:
    Traceback (most recent call last):
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 503, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 866, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.calculus.calculus.laplace[34]>", line 1, in <module>
        laplace(t**n, t, s, algorithm='giac')
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/sage/calculus/calculus.py", line 1456, in laplace
        return result.sage()
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/sage/interfaces/interface.py", line 1053, in sage
        return self._sage_(*args, **kwds)
      File "/Users/forets/sage-src/sage/local/lib/python2.7/site-packages/sage/interfaces/giac.py", line 1095, in _sage_
        raise NotImplementedError("Unable to parse Giac output: %s" % result)
    NotImplementedError: Unable to parse Giac output: integrate(t^n*exp(-s*t),t,0,+infinity)
**********************************************************************
1 item had failures:
   2 of  39 in sage.calculus.calculus.laplace
    [419 tests, 2 failures, 11.53 s]
----------------------------------------------------------------------
sage -t src/sage/calculus/calculus.py  # 2 doctests failed
----------------------------------------------------------------------
Total time for all tests: 11.6 seconds
    cpu time: 10.8 seconds
    cumulative wall time: 11.5 seconds

comment:7 Changed 22 months ago by tscrim

The first failure is not a true failure and is independent. We need to figure out what makes this test pass on some systems and not on others.

comment:8 Changed 22 months ago by chapoton

  • Description modified (diff)
  • Summary changed from fix a calculus doctest to fix a calculus doctest (giac, laplace, integration)

comment:9 Changed 22 months ago by chapoton

I see this failure with giac 1.2.3.

The branch works for me.

This problem is rather annoying, it prevents my patchbot from giving any green light..

comment:10 Changed 22 months ago by chapoton

@mforets, could you try the following, please:

sage: var('t,n,s')
sage: ex=t^n
sage: from sage.interfaces.giac import giac
sage: giac.laplace(ex, t, s)
integration(t^n*exp(-s*t),t,0,+infinity)

comment:11 Changed 22 months ago by git

  • Commit changed from 2186acae2da3826b0a29d196d6b22b8fbc1dad4b to 200ba71b935a6573ee9aac38a35f0eefd5507b5d

Branch pushed to git repo; I updated commit sha1. New commits:

d79c73fMerge branch 'u/chapoton/22833' in 8.0.b4
200ba71trac 22833 adding a conversion back from giac for definite integrals

comment:12 Changed 22 months ago by git

  • Commit changed from 200ba71b935a6573ee9aac38a35f0eefd5507b5d to 79442fe1dd9e24059f3c88db153c4f84dd090bfe

Branch pushed to git repo; I updated commit sha1. New commits:

79442fetrac 222833 detail

comment:13 follow-up: Changed 22 months ago by chapoton

And also tell me what you get when doing

sage: %giac

giac: Int(exp(x),x,0,1)
integration(exp(x),x,0,1)

comment:14 in reply to: ↑ 13 Changed 22 months ago by mforets

Replying to chapoton:

And also tell me what you get when doing

sage: %giac

giac: Int(exp(x),x,0,1)
integration(exp(x),x,0,1)

EDIT: no no, ignore my last message, I hadn't notice that you pushed changes < 1hr ago.

i've recompiled this branch and i'm getting:

sage: %giac


  --> Switching to Giac <--

giac: Int(exp(x),x,0,1)
integrate(exp(x),x,0,1)
Last edited 22 months ago by mforets (previous) (diff)

comment:15 follow-ups: Changed 22 months ago by mforets

in my machine, and for the commit 79442f3, i have

sage: var('t,n,s')
(t, n, s)
sage: ex=t^n
sage: from sage.interfaces.giac import giac
sage: giac.laplace(ex, t, s)
integrate(t^n*exp(-s*t),t,0,+infinity)
sage: %giac
 
  --> Switching to Giac <--
 
giac: Int(exp(x),x,0,1)

integrate(exp(x),x,0,1)

and i still get:

sage: laplace(t^n*exp(-s*t),t,s,algorithm='giac')
...
NotImplementedError: Unable to parse Giac output: integrate(t^n*exp(-2*s*t),t,0,+infinity)

this exception is raised when we transform back to sage. maybe it's better to mimick the unevaluated laplace as with the SymPy? case, that is:

...
if 'integrate' in format(result):
   return dummy_laplace(ex, t, s)
else:
   return result.sage()

i git pulled and re-compiled everything, so i ignore why you get "integration" instead.

that said, if you have pressing need then you may as well consider removing this "no-go" test altogether..

comment:16 in reply to: ↑ 15 Changed 22 months ago by mforets

Replying to mforets:

...
if 'integrate' in format(result):
   return dummy_laplace(ex, t, s)
else:
   return result.sage()

with that modif, the expected output for the test at hand becomes: laplace(t^n, t, s)

in my machine the updated code passes. let me know if you want me to push that or try something else.

Last edited 22 months ago by mforets (previous) (diff)

comment:17 in reply to: ↑ 15 Changed 22 months ago by rws

Replying to mforets:

and i still get:

sage: laplace(t^n*exp(-s*t),t,s,algorithm='giac')
...
NotImplementedError: Unable to parse Giac output: integrate(t^n*exp(-2*s*t),t,0,+infinity)

Same here.

comment:18 Changed 22 months ago by rws

Would there be need for symbolic laplace? Maxima might return such unevaluated results as anonymous functions too, and they are difficult to handle.

comment:19 Changed 22 months ago by mforets

in this case it is raising a NotImplementedError, which i find a bit misleading if this kind of exception belongs to the level of Sage the library. perhaps an intermediate point is to return just integrate(t^n*exp(-2*s*t),t,0,+infinity), what do you think?

on the other hand, Laplace placeholder could be needed here (in opposition to a naked integral):

  • to attach properties of the transform which are not 'obvious', eg. properties as a function of a complex variable s, convergence radius, final and initial value theorems
  • if you have many of them, it looks nicer with respect to printing the equations!

comment:20 Changed 22 months ago by chapoton

Got an answer from Parisse:

si vous etes en francais (export LANG=fr_FR.UTF-8 par exemple), 
les mots-clefs de giac qui ont une traduction s'affichent en francais.
 Le fichier qui contient les traductions sous linux est 
/usr/share/giac/doc/fr/keywords.
Si vous passez en export LANG=en_US.UTF-8 ca restera en anglais.

Dans l'interface de Xcas les fichiers de session sont sauvegardes 
avec les mots-clefs en anglais.

comment:21 Changed 22 months ago by tscrim

So what he is saying is that it has to with the localization settings?

comment:22 Changed 22 months ago by chapoton

I think so. The English "integrate" got translated to "integration" in the output for people whose locale is French.

https://fossies.org/linux/giac/doc/fr/keywords

Last edited 22 months ago by chapoton (previous) (diff)

comment:23 Changed 22 months ago by tscrim

I think there is a deeper issue. I suspect that is failing out at some other place because it can't find the symbol integration in the symbol table. At least this is coming from trying to unravel things and get a better traceback:

sage: t,n,s = var('t,n,s')
sage: from sage.libs.pynac.pynac import symbol_table
sage: from sage.calculus.calculus import symbolic_expression_from_string
sage: ex = t^n*exp(-s*t)
sage: result = ex._giac_().integrate(t._giac_(), 0._giac_(), oo._giac_())
sage: symbolic_expression_from_string(repr(result), symbol_table['giac'], True)
...
/home/travis/sage-build/local/lib/python2.7/site-packages/sage/interfaces/maxima_lib.pyc in sr_integral(self, *args)
    808                 raise ValueError("Integral is divergent.")
    809             elif "Is" in s: # Maxima asked for a condition
--> 810                 self._missing_assumption(s)
    811             else:
    812                 raise

/home/travis/sage-build/local/lib/python2.7/site-packages/sage/interfaces/maxima_lib.pyc in _missing_assumption(self, errstr)
   1018              + errstr[jj+1:k] +">0)', see `assume?` for more details)\n" + errstr
   1019         outstr = outstr.replace('_SAGE_VAR_','')
-> 1020         raise ValueError(outstr)
   1021 
   1022 def is_MaximaLibElement(x):

ValueError: Computation failed since Maxima requested additional constraints; using the 'assume' command before evaluation *may* help (example of legal syntax is 'assume(s>0)', see `assume?` for more details)
Is s positive, negative or zero?

Do you get a similar error?

comment:24 Changed 22 months ago by chapoton

no, I just get

integration(t^n*e^(-s*t), t, 0, +Infinity)

comment:25 Changed 22 months ago by tscrim

I think the problem is actually with Maxima. What happens when you try to explicitly evaluate the integral?

sage: integrate(t^n*exp(-s*t),t,0,oo)

I get the same error as above when I do this. If I impose s > 0 and n in Z>0, then I get

s^(-n - 1)*gamma(n + 1)

comment:26 Changed 22 months ago by chapoton

sage: integrate(t^n*exp(-s*t),t,0,oo)

gives me the same "assume" failure as you got ;

and then works as for you with the assumes done

comment:27 Changed 22 months ago by tscrim

Hmm...that is strange. So something is happening between the attempt to parse the giac output and actually evaluate the maxima integral.

comment:28 Changed 22 months ago by tscrim

Is 'integration' a key in your symbol_table['functions']?

comment:29 Changed 22 months ago by tscrim

Also, do you have

sage: symbol_table['giac']
{'(1+sqrt(5))/2': golden_ratio,
 'Dirac': dirac_delta,
 'Heaviside': heaviside,
 'pi': pi}

comment:30 Changed 22 months ago by chapoton

'integrate': integrate,
 'integration': integration,

in symbol_tablefunctions?

and

sage: symbol_table['giac']

{'(1+sqrt(5))/2': golden_ratio,
 'Dirac': dirac_delta,
 'Heaviside': heaviside,
 'pi': pi}

comment:31 Changed 22 months ago by tscrim

That's the difference: I don't have the 'integration' in the functions symbol table. What is the integration function, specifically, what does import_statements give?

comment:32 Changed 22 months ago by chapoton

sage: import_statements('integration')
# **Warning**: distinct objects with name 'integration' in:
#   - sage.calculus
#   - sage.symbolic
import sage.symbolic.integration

comment:33 Changed 22 months ago by tscrim

Sorry, I was vague, I meant:

sage: import_statements(symbol_table['functions]['integration'])

comment:34 Changed 22 months ago by chapoton

sage: symbol_table['functions']['integration']
integration
sage: import_statements(_)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)

ValueError: no import statement found for 'integration'.

comment:35 Changed 22 months ago by chapoton

sage: symbol_table['functions']['integration']
integration
sage: type(_)
<class 'sage.symbolic.function_factory.NewSymbolicFunction'>

comment:36 Changed 22 months ago by tscrim

Now I'm not so sure about what to look for or ask, but I'm investigating. Ralf, do you have any ideas?

comment:37 Changed 22 months ago by tscrim

So I think what is happening is that integration is being considered by Maxima as a formal function. Hence, it is not simplified as an integral, and I don't think you could manipulate it as a function.

comment:38 Changed 22 months ago by mforets

does this pass?

...
if 'integrate' in format(result) or 'integration' in format(result):
   return dummy_laplace(ex, t, s)
else:
   return result.sage()

comment:39 Changed 22 months ago by rws

Let me try to summarize. We cannot fix the doctest because of conflicting results due to giac's clever use of locale which breaks its interface to the world. The immediate way to adapt to this IMO is to look for both integrate and integration in the result when using laplace of giac. The more far-sighted way would be to fix the giac-Sage interface so that in the laplace code we can just say return result.sage() unconditionally.

comment:40 Changed 22 months ago by chapoton

could somebody please take action here, and propose a better branch ?

comment:41 Changed 22 months ago by strogdon

What is the desired output of

sage: laplace(t^n, t, s, algorithm='giac')

If it is integration(t^n*e^(-s*t), t, 0, +Infinity) then one could add

integrate integration

to local/share/giac/doc/en/keywords and any other provided $LANG/keywords file. Doing this I get

sage: var('t,n,s')
(t, n, s)
sage: ex=t^n
sage: from sage.interfaces.giac import giac
sage: giac.laplace(ex, t, s)
integration(t^n*exp(-s*t),t,0,+infinity)
sage: %giac

  --> Switching to Giac <--

giac: Int(exp(x),x,0,1)
integration(exp(x),x,0,1)

Without the change the doctest would pass but with the change I get

sage -t --long src/sage/calculus/calculus.py
**********************************************************************
File "src/sage/calculus/calculus.py", line 1406, in sage.calculus.calculus.laplace
Failed example:
    laplace(t^n, t, s, algorithm='giac')
Expected:
    Traceback (most recent call last):
    ...
    NotImplementedError: Unable to parse Giac output: integrate(t^n*exp(-s*t),t,0,+infinity)
Got:
    integration(t^n*e^(-s*t), t, 0, +Infinity)

comment:42 Changed 22 months ago by mforets

again, for consistency the desired output of laplace(t^n, t, s, algorithm='giac') should be the same as the other two interfaces when they don't know the answer, don't you think?

from your answers, i can reproduce the issue playing with $ export LANG=fr_FR.UTF-8 and $ export LANG=en_US.UTF-8 in the shell :)

hence, let me upload a branch doing rws's comment:39 and my comment:38, if i understood correctly.

comment:43 Changed 22 months ago by mforets

  • Authors changed from Frédéric Chapoton to Frédéric Chapoton, Marcelo Forets
  • Branch changed from u/chapoton/22833 to u/mforets/22833
  • Cc frederichan added
  • Commit changed from 79442fe1dd9e24059f3c88db153c4f84dd090bfe to e2d74c645b48c3288802f9080d6651aaccf658fb
  • Status changed from needs_work to needs_review

New commits:

e2d74c6parse unevaluated expression in EN and FR

comment:44 Changed 21 months ago by egourgoulhon

Just to confirm that merging this branch in Sage 8.0.beta6 fixes the doctest error on my system.

comment:45 Changed 21 months ago by strogdon

  • Reviewers changed from Travis Scrimshaw to Travis Scrimshaw, Steven Trogdon
  • Status changed from needs_review to positive_review

I think this is good.

comment:46 Changed 21 months ago by mforets

Thanks for the review.

Here are two somehow related issues: #22997 and #22995.

comment:47 Changed 21 months ago by vbraun

  • Branch changed from u/mforets/22833 to e2d74c645b48c3288802f9080d6651aaccf658fb
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.