Opened 9 years ago

Closed 3 years ago

#9424 closed defect (fixed)

Provide symbolic sum function with evalf

Reported by: burcin Owned by: burcin
Priority: major Milestone: sage-6.8
Component: symbolics Keywords:
Cc: whuss, kcrisman, eviatarbach Merged in:
Authors: Ralf Stephan Reviewers: Daniel Krenn
Report Upstream: N/A Work issues:
Branch: 4f7b161 (Commits) Commit: 4f7b1610a3fd78eeea2e188605bccf3983a76490
Dependencies: #17759 Stopgaps:

Description (last modified by rws)

Symbolics sums returned from maxima cannot be numerically evaluated, since they don't define an _evalf_() method.

This was reported by dirkd on sage-support:

Why is evaluating this expression problematical?

y1(x)=x^2;y2(x)=5-x;
a0=1;an=3;Delta=(an-a0)/n;p(k)=a0+(k-1/2)*Delta;
I(n)=sum(abs(y2(p(k))-y1(p(k)))*Delta,k,1,n);
N(I(10))

SAGE respons:
<snipped traceback>
  File "expression.pyx", line 3797, in
sage.symbolic.expression.Expression.n (sage/symbolic/expression.cpp:
17022)
TypeError: cannot evaluate symbolic expression numerically

Here is the thread:

http://groups.google.com/group/sage-support/t/615b15ca638c9652

See also #15346

Change History (35)

comment:1 Changed 9 years ago by whuss

  • Cc whuss added

comment:2 Changed 9 years ago by kcrisman

  • Cc kcrisman added

comment:3 Changed 6 years ago by eviatarbach

  • Cc eviatarbach added

comment:4 Changed 6 years ago by jdemeyer

  • Milestone changed from sage-5.11 to sage-5.12

comment:5 Changed 5 years ago by vbraun_spam

  • Milestone changed from sage-6.1 to sage-6.2

comment:6 Changed 5 years ago by vbraun_spam

  • Milestone changed from sage-6.2 to sage-6.3

comment:7 Changed 5 years ago by rws

  • Milestone changed from sage-6.3 to sage-duplicate/invalid/wontfix
  • Status changed from new to needs_review

Your problem is threefold: 1. you use n both as sum endpoint and variable, 2. Sage sum is only intended with symbolic endpoints, and 3. the need to use Python summation may require defining a Python function instead of a Sage symbolic function:

sage: sum?
...
Warning: This function only works with symbolic expressions. To sum any
     other objects like list elements or function return values,
     please use python summation...

sage: I(n)=sum(abs(y2(p(k))-y1(p(k)))*Delta,k,1,n);
sage: I
n |--> 2*sum(abs(-4*k^2 - 3*(2*k - 1)*n + 3*n^2 + 4*k - 1), k, 1, n)/n^3

sage: def I(n):
....:     return (2*sum(abs(-4*k^2 - 3*(2*k - 1)*n + 3*n^2 + 4*k - 1) for k in range(1,n+1))/n^3)
....: 
sage: I(10)
1301/250
sage: [I(i) for i in range(1,11)]
[2, 5, 46/9, 5, 26/5, 277/54, 254/49, 665/128, 418/81, 1301/250]

comment:8 Changed 5 years ago by kcrisman

  • Milestone changed from sage-duplicate/invalid/wontfix to sage-6.3
  • Type changed from defect to enhancement

I don't think any of these invalidate the ticket; the point is to extend the behavior. Why is 1. a problem? This seems like it should be a nice function to me. See Burcin's reply in the thread:

> If I leave out the N( )-operator on the last line the block evaluates
> to
> 
> 
> 1/500*sum(abs(-4*k^2 - 56*k + 329), k, 1, 10)
> 
> which can be evaluated in a new inputbox. Why not in one step?
The result returned from maxima uses a symbolic function object created
on the fly. This is quite different from the sum() function
available on the command line, and unfortunately, it doesn't define a
numerical evaluation function, _evalf_().

Burcin knows this code very well, so I would be surprised if he misdiagnosed this. But I figure maybe changing to enhancement will appease everyone :)

comment:9 Changed 5 years ago by nbruin

Burcin is spot-on:

sage: S=(I(10).operands()[0].operator()); S
sum
sage: type(S)
<class 'sage.symbolic.function_factory.NewSymbolicFunction'>

(Note the "New", not "BuiltIn?" or similar. It's a completely generic placeholder) we just need a symbolic function hooked up that can do some mildly intelligent evaluation when asked for it.

Incidentally, we can just map back to maxima and do the right thing there:

sage: maxima_calculus(I(10))
('sum(abs(4*_SAGE_VAR_k^2+56*_SAGE_VAR_k-329),_SAGE_VAR_k,1,10))/500
sage: SR(maxima_calculus(I(10)).simplify_sum())
1301/250

(I haven't checked if it's correct). You can see why the "simplify_sum" is required: the newly created "sum" function in SR is linked to the inert "'sum".

comment:10 Changed 5 years ago by rws

  • Status changed from needs_review to needs_work

comment:11 Changed 5 years ago by vbraun_spam

  • Milestone changed from sage-6.3 to sage-6.4

comment:12 follow-up: Changed 5 years ago by kcrisman

A relevant ask.sagemath question.

Another one.

Maybe it's time we fixed this.

comment:13 in reply to: ↑ 12 Changed 5 years ago by rws

  • Description modified (diff)
  • Summary changed from numerical evaluation of symbolic sums to FP evaluation of symbolic sums fails

Replying to kcrisman:

Maybe it's time we fixed this.

It is not clear if forcing people to use N() and getting a float, even if there is an integer simplification of the sum, is the right thing to do. Granted, the error thrown on N(I(10)) is a bug, and this ticket is about it. Here is a minimal example:

sage: (k,n) = var('k,n')
sage: f(n)=sum(abs(-k*k+n),k,1,n)
sage: f(n=8)
sum(abs(-k^2 + 8), k, 1, 8)
sage: N(f(8))

However, I would expect f(n=8).simplify() or .expand() to give me the result 162, and this is #17422

comment:14 Changed 5 years ago by rws

  • Description modified (diff)
  • Type changed from enhancement to defect

comment:15 Changed 4 years ago by rws

  • Status changed from needs_work to needs_info

As #15346 would implement simplify_sum, what remains for this ticket to do?

comment:16 Changed 4 years ago by kcrisman

  • Summary changed from FP evaluation of symbolic sums fails to Provide symbolic sum function with evalf

As #15346 would implement simplify_sum, what remains for this ticket to do?

I suppose there is the issue that one might not want to have to simplify in order to get a floating-point number. Really, we shouldn't require that. With #15346 one needs

sage: I(10).expand_sum().n()

but really I(10).n() should suffice, and for that we need a symbolic sum function. I'll clarify the title, though, as you are right that this is at least now much more possible. I'll probably add the example as a reviewer patch on #15346, in fact.

comment:17 Changed 4 years ago by kcrisman

  • Status changed from needs_info to needs_work

comment:18 Changed 4 years ago by rws

This is a proof of concept-patch that works, but only with bare sums. I guess the walk on the expression tree happens somewhere in Pynac.

diff --git a/src/sage/symbolic/expression.pyx b/src/sage/symbolic/expression.pyx
--- a/src/sage/symbolic/expression.pyx
+++ b/src/sage/symbolic/expression.pyx
@@ -4847,6 +4847,19 @@ cdef class Expression(CommutativeRingElement):
             sage: all(len(str(e.n(digits=k)))-1 >= k for k in ks)
             True
 
+        We also allow evaluation of unexpanded symbolic sums
+        with numeric limits (:trac:`9424`)::
+
+            sage: (k,n) = var('k,n')
+            sage: f(n) = sum(abs(-k*k+n), k, 1, n)
+            sage: f(n=8)
+            sum(abs(-k^2 + 8), k, 1, 8)
+            sage: f(n=8).n()
+            162.0
+            sage: f(n=x).n()
+            Traceback (most recent call last):
+            ...
+            TypeError: cannot evaluate symbolic expression numerically
         """
         if prec is None:
             if digits is None:
@@ -4865,11 +4878,16 @@ cdef class Expression(CommutativeRingElement):
             x = self._convert(kwds)
 
         # we have to consider constants as well, since infinity is a constant
-        # in pynac
+        # in pynac. Also, catch symbolic sums with numeric limits.
         if is_a_numeric(x._gobj):
             res = py_object_from_numeric(x._gobj)
-        elif  is_a_constant(x._gobj):
+        elif is_a_constant(x._gobj):
             res = x.pyobject()
+        elif x.operator().name() == 'sum' and (
+                    is_a_numeric((<Expression>x.operands()[2])._gobj)
+                and is_a_numeric((<Expression>x.operands()[3])._gobj)):
+            from sage.calculus.calculus import symbolic_sum
+            return symbolic_sum(*(x.operands()))
         else:
             raise TypeError("cannot evaluate symbolic expression numerically")

but

sage: (k,n) = var('k,n')
sage: f(n) = sum(abs(-k*k+n),k,1,n)
sage: 1+f(n=8)
sum(abs(-k^2 + 8), k, 1, 8) + 1
sage: _.n()
BOOM

comment:19 Changed 4 years ago by rws

  • Branch set to u/rws/provide_symbolic_sum_function_with_evalf

comment:20 Changed 4 years ago by rws

  • Authors set to Ralf Stephan
  • Commit set to 32bca6ab2b1cd43e92fb3bdc262c93a9e8ca6540
  • Dependencies set to #17759
  • Status changed from needs_work to needs_review

I think I found a satisfying solution.


New commits:

3781eec17759: convenience class symbolic ExpressionTreeWalker(Converter)
32bca6a9424: Provide symbolic sum function with evalf

comment:21 follow-up: Changed 4 years ago by kcrisman

You'd think that given the massive amounts of snow here lately I'd have time to carefully look this over - but I don't :-(

However I think this looks like a good start at a solution, and definitely the first commit is an improvement in any case - perhaps one that deserves its own ticket, for quicker review? (There might be more people interested in that, honestly.) Unless someone complained with the name change - but it was never globally imported and frankly we do need a general expression tree walker more easily accessible, see #9329 for instance. So connecting this (not here, maybe) to the Converter class it inherits from and explaining when one would use each one would be useful too.

The main concern I have in terms of the main purpose of the ticket is to adequately test and debug DefiniteSumExpander, especially since it is in the code in such a way that it isn't doctested (other than the case in question).

comment:22 in reply to: ↑ 21 ; follow-up: Changed 4 years ago by rws

Replying to kcrisman:

However I think this looks like a good start at a solution, and definitely the first commit is an improvement in any case - perhaps one that deserves its own ticket, for quicker review?

I don't understand. You did see the number #17759?

The main concern I have in terms of the main purpose of the ticket is to adequately test and debug DefiniteSumExpander, especially since it is in the code in such a way that it isn't doctested (other than the case in question).

Most of it is doctested in #17759, i.e., its base class.

comment:23 in reply to: ↑ 22 ; follow-up: Changed 4 years ago by kcrisman

Replying to kcrisman:

However I think this looks like a good start at a solution, and definitely the first commit is an improvement in any case - perhaps one that deserves its own ticket, for quicker review?

I don't understand. You did see the number #17759?

Nope! I really didn't have a lot of time - never even saw the dependency field. Maybe I shouldn't look at tickets after shoveling ;)

Most of it is doctested in #17759, i.e., its base class.

I did see that doctesting, I just would want to see what was different in the sum case.

I'll keep these on the front burner.

comment:24 in reply to: ↑ 23 Changed 4 years ago by rws

Replying to kcrisman:

I did see that doctesting, I just would want to see what was different in the sum case.

There the composition method is overwritten (and tested).

comment:25 Changed 4 years ago by rws

  • Status changed from needs_review to needs_work
sage -t --long src/sage/functions/hypergeometric.py  # 11 doctests failed
sage -t --long src/sage/functions/trig.py  # 1 doctest failed

comment:26 Changed 4 years ago by rws

Fixing these doctests will need the solutions found for fixing #17849 (or vice versa).

comment:27 Changed 4 years ago by git

  • Commit changed from 32bca6ab2b1cd43e92fb3bdc262c93a9e8ca6540 to 27a16927956651abf9361d8c0a12dffb00cba963

Branch pushed to git repo; I updated commit sha1. New commits:

5a1ac7eMerge branch 'develop' into t/9424/provide_symbolic_sum_function_with_evalf
9375510Merge branch 'develop' into t/17759/public/17759
16aa81d17759: handle hold=True and hypergeometric
1f4edf2Merge branch 'public/17759' of trac.sagemath.org:sage into t/9424/provide_symbolic_sum_function_with_evalf
27a16929424: delegate to superclass; fix doctest

comment:28 Changed 4 years ago by rws

  • Milestone changed from sage-6.4 to sage-6.6
  • Status changed from needs_work to needs_review

comment:29 Changed 4 years ago by rws

  • Branch changed from u/rws/provide_symbolic_sum_function_with_evalf to u/rws/9424

comment:30 Changed 4 years ago by rws

  • Commit changed from 27a16927956651abf9361d8c0a12dffb00cba963 to b213d69bf13e1c7e21d2da2868bfd7908f5d31eb
  • Milestone changed from sage-6.6 to sage-6.8

New commits:

fa7328517759: convenience class symbolic ExpressionTreeWalker(Converter)
b213d69Merge branch 'u/rws/provide_symbolic_sum_function_with_evalf' of trac.sagemath.org:sage into tmp2

comment:31 Changed 4 years ago by mmezzarobba

  • Status changed from needs_review to needs_work

TestsFailed (says the patchbot)

comment:32 Changed 4 years ago by git

  • Commit changed from b213d69bf13e1c7e21d2da2868bfd7908f5d31eb to 4f7b1610a3fd78eeea2e188605bccf3983a76490

Branch pushed to git repo; I updated commit sha1. New commits:

985fdb4Merge branch 'develop' into t/9424/9424
4f7b1619424: make ceil/floor accept kwds; fix doctests

comment:33 Changed 4 years ago by rws

  • Status changed from needs_work to needs_review

comment:34 Changed 3 years ago by dkrenn

  • Reviewers set to Daniel Krenn
  • Status changed from needs_review to positive_review

LGTM

comment:35 Changed 3 years ago by vbraun

  • Branch changed from u/rws/9424 to 4f7b1610a3fd78eeea2e188605bccf3983a76490
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.