Sage: Ticket #11576: make it possible to generate sequences of variables easily
https://trac.sagemath.org/ticket/11576
<p>
People are <em>always</em> asking how to get sequences of variables, like <code>a1,a2,a3,a4</code> or the like. See <a class="ext-link" href="http://ask.sagemath.org/question/611/implicitly-defining-a-sequence-of-variables"><span class="icon"></span>this ask.sagemath.org</a> question, for example.
</p>
<p>
Jason Grout has an interesting possible solution that should find a home somewhere in Sage
(see <a class="ext-link" href="https://groups.google.com/d/topic/sage-support/GFJdjFvKCvo/discussion"><span class="icon"></span>sage-support discussion</a>).
</p>
<blockquote class="citation">
<pre class="wiki">class VariableGenerator(object):
def __init__(self, prefix):
self.__prefix = prefix
@cached_method
def __getitem__(self, key):
return SR.var("%s%s"%(self.__prefix,key))
</pre><p>
Now just specify a prefix, and then you can index to your heart's
content to generate variables.
</p>
<pre class="wiki">a = VariableGenerator('a') # some people may like 'a_' as the prefix
a[0], a[1], a[2] # all variables
</pre><p>
Of course, this can easily be extended to using function call syntax:
a(0), or to using multiple indices: a[1,3]. Indeed, you can let your
imagination run wild and even do things like return full symbolic
matrices or vectors with slices: a[0:5, 0:5].
</p>
</blockquote>
<p>
Apply <a class="attachment" href="https://trac.sagemath.org/attachment/ticket/11576/trac_11576-indexed_expression.patch" title="Attachment 'trac_11576-indexed_expression.patch' in Ticket #11576">trac_11576-indexed_expression.patch</a><a class="trac-rawlink" href="https://trac.sagemath.org/raw-attachment/ticket/11576/trac_11576-indexed_expression.patch" title="Download"></a>
</p>
en-usSagehttps://trac.sagemath.org/chrome/site/logo_sagemath_trac.png
https://trac.sagemath.org/ticket/11576
Trac 1.1.6burcinTue, 05 Jul 2011 14:34:27 GMT
https://trac.sagemath.org/ticket/11576#comment:1
https://trac.sagemath.org/ticket/11576#comment:1
<p>
GiNaC already has indexed expressions. We should try to use what is there instead of coming up with a pure python solution ourselves. This was discussed on sage-devel a while ago, I even put up an experimental patch for it:
</p>
<p>
<a class="ext-link" href="http://groups.google.com/group/sage-devel/browse_thread/thread/69ab50fe11672111"><span class="icon"></span>http://groups.google.com/group/sage-devel/browse_thread/thread/69ab50fe11672111</a>
</p>
TicketkcrismanTue, 05 Jul 2011 14:43:17 GMT
https://trac.sagemath.org/ticket/11576#comment:2
https://trac.sagemath.org/ticket/11576#comment:2
<p>
Thanks, Burcin! I must have missed this somehow.
</p>
<p>
<a class="ext-link" href="http://sage.math.washington.edu/home/burcin/indexed_expression.patch"><span class="icon"></span>Here</a> is a link to the patch.
</p>
<p>
From that thread:
</p>
<pre class="wiki">Note that GiNaC cannot take derivatives of indexed expressions.
</pre><p>
So that means one couldn't say that <code>diff(a[1],a[1])==1</code>? That would seem to be unfortunate.
</p>
<p>
By the way, note that Jason was just trying to find a way to make lots of !GiNaC variables. I don't think that these approaches are necessarily that different - for instance, a class like Jason's for non-indexed, but still many, variables of similar names, would, be, useful. , , ,
</p>
TicketburcinTue, 05 Jul 2011 16:20:55 GMT
https://trac.sagemath.org/ticket/11576#comment:3
https://trac.sagemath.org/ticket/11576#comment:3
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/11576#comment:2" title="Comment 2">kcrisman</a>:
</p>
<blockquote class="citation">
<p>
From that thread:
</p>
</blockquote>
<pre class="wiki">Note that GiNaC cannot take derivatives of indexed expressions.
</pre><blockquote class="citation">
<p>
So that means one couldn't say that <code>diff(a[1],a[1])==1</code>? That would seem to be unfortunate.
</p>
</blockquote>
<p>
I don't have time to test this right now, but if it really doesn't work, we could fix it or ask the GiNaC developers why they chose to omit this.
</p>
<blockquote class="citation">
<p>
By the way, note that Jason was just trying to find a way to make lots of !GiNaC variables. I don't think that these approaches are necessarily that different - for instance, a class like Jason's for non-indexed, but still many, variables of similar names, would, be, useful. , , ,
</p>
</blockquote>
<p>
It would be useful to extend the <code>var()</code> syntax to support creating variables like 'a1, a2, ..., a10'. We should keep in mind the signature of other constructors like <code>PolynomialRing()</code> if we attempt this.
</p>
<p>
If we go further and try to support vectors and matrices with symbolic entries, it would be better to use the underlying GiNaC functionality. There is an example at the end of this page:
</p>
<p>
<a class="ext-link" href="http://www.ginac.de/tutorial/Indexed-objects.html"><span class="icon"></span>http://www.ginac.de/tutorial/Indexed-objects.html</a>
</p>
<p>
If you can propose a basic user interface, I'd be happy to help with at least the first steps of interfacing C++.
</p>
TicketnbruinTue, 05 Jul 2011 19:20:36 GMT
https://trac.sagemath.org/ticket/11576#comment:4
https://trac.sagemath.org/ticket/11576#comment:4
<p>
Perhaps another motivation for using GiNaC functionality for this is that the alternative approach leaks. Watch the memory usage of
</p>
<pre class="wiki">sage: for i in range(10^8):
....: l=1+SR.symbol("a%s"%str(i))
</pre><p>
Perhaps <a class="missing wiki">GiNaCs?</a> indexed variables are less prone to leaking?
</p>
TicketiandrusTue, 05 Jul 2011 20:33:40 GMTcc changed
https://trac.sagemath.org/ticket/11576#comment:5
https://trac.sagemath.org/ticket/11576#comment:5
<ul>
<li><strong>cc</strong>
<em>iandrus</em> added
</li>
</ul>
TickethivertThu, 24 Nov 2011 21:01:11 GMTattachment set
https://trac.sagemath.org/ticket/11576
https://trac.sagemath.org/ticket/11576
<ul>
<li><strong>attachment</strong>
set to <em>indexed_expression-experimental-fh.patch</em>
</li>
</ul>
TickethivertThu, 24 Nov 2011 21:20:50 GMT
https://trac.sagemath.org/ticket/11576#comment:6
https://trac.sagemath.org/ticket/11576#comment:6
<p>
Hi there,
</p>
<p>
As as said on sage-devel, I started to work on Burcin patch. My goal was to be
able to index a variable by <strong>any</strong> Sage object (eg: integers, group
element, matrices...). So I extended a little Burcin patch. Now I'm faced with
the problem that GiNaC indexed variable seems to have a not trivial semantic
that I don't understand at all. Moreover I don't know how to debug Cython /
C++ code so I'm quite stuck. Here is the problem I have.
</p>
<p>
To get it you should apply <a class="attachment" href="https://trac.sagemath.org/attachment/ticket/11576/indexed_expression-experimental-fh.patch" title="Attachment 'indexed_expression-experimental-fh.patch' in Ticket #11576">attachment:indexed_expression-experimental-fh.patch</a><a class="trac-rawlink" href="https://trac.sagemath.org/raw-attachment/ticket/11576/indexed_expression-experimental-fh.patch" title="Download"></a>
on a fresh Sage-4.7.2 install. Then I can
</p>
<pre class="wiki">sage: m1 = matrix([1,2]); m1.set_immutable()
sage: m2 = matrix([2,1]); m2.set_immutable()
sage: a = x.ind[m1]; b = 2*x.ind[m2]
sage: a + a
2*x.[1 2]
sage: a + b
x.[1 2] + 2*x.[2 1]
</pre><p>
which is, in my view very cool !
</p>
<p>
However strange thing has occurred under the hood, in particular for a strange
reason I don't understand Sage added the two matrices ! This is apparent in
</p>
<pre class="wiki">sage: a = x.ind[Permutation([3,2,1])]; b = 2*x.ind[1]
sage: a+b
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/home/data/Sage-Install/sage-4.7.2/devel/sage-review/sage/symbolic/<ipython console> in <module>()
/home/florent/src/Sage/sage/local/lib/python2.6/site-packages/sage/structure/element.so in sage.structure.element.RingElement.__add__ (sage/structure/element.c:11488)()
/home/florent/src/Sage/sage/local/lib/python2.6/site-packages/sage/symbolic/expression.so in sage.symbolic.expression.Expression._add_ (sage/symbolic/expression.cpp:11082)()
2186 relational_operator(_right._gobj))
2187 else:
-> 2188 x = gadd(left._gobj, _right._gobj)
2189 return new_Expression_from_GEx(left._parent, x)
2190
/home/florent/src/Sage/sage/local/lib/python2.6/site-packages/sage/structure/element.so in sage.structure.element.RingElement.__sub__ (sage/structure/element.c:11816)()
/home/florent/src/Sage/sage/local/lib/python2.6/site-packages/sage/structure/coerce.so in sage.structure.coerce.CoercionModel_cache_maps.bin_op (sage/structure/coerce.c:7489)()
TypeError: unsupported operand parent(s) for '-': '<class 'sage.combinat.permutation.Permutation_class'>' and 'Integer Ring'
</pre><p>
To make thing crystal clear (I hope):
</p>
<pre class="wiki">sage: class bla(SageObject):
... def __add__(self, other):
... print "Addition called"
sage: a, b = x.ind[m1],2*x.ind[m2]
sage: m1 = bla()
sage: m2 = bla()
sage: m1 + m2
Addition called
sage: m1*m2
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/home/data/Sage-Install/sage-4.7.2/devel/sage-review/sage/symbolic/<ipython console> in <module>()
TypeError: unsupported operand type(s) for *: 'bla' and 'bla'
</pre><p>
WTF ! Why is it adding or multiplying my indices for nothing ! It is a problem
of Ginac ? of the wrapper ? or behind the chair and the screen ?
</p>
<p>
Any help greatly appreciated !
</p>
<p>
Cheers,
</p>
<p>
Florent
</p>
<p>
PS: crosspost on sage-devel.
</p>
TicketburcinMon, 06 Feb 2012 12:42:31 GMTkeywords set
https://trac.sagemath.org/ticket/11576#comment:7
https://trac.sagemath.org/ticket/11576#comment:7
<ul>
<li><strong>keywords</strong>
<em>Cernay2012</em> added
</li>
</ul>
TicketmjoMon, 12 Nov 2012 04:32:34 GMTcc changed
https://trac.sagemath.org/ticket/11576#comment:8
https://trac.sagemath.org/ticket/11576#comment:8
<ul>
<li><strong>cc</strong>
<em>mjo</em> added
</li>
</ul>
TicketmjoSat, 01 Dec 2012 04:19:04 GMT
https://trac.sagemath.org/ticket/11576#comment:9
https://trac.sagemath.org/ticket/11576#comment:9
<p>
Allow me to throw my entry into the ring.
</p>
<p>
It's not nearly as ambitious as Burcin & Florent's patches, but it does scratch the itch that most people have, and it's working and tested already.
</p>
TicketkcrismanSat, 01 Dec 2012 14:47:06 GMT
https://trac.sagemath.org/ticket/11576#comment:10
https://trac.sagemath.org/ticket/11576#comment:10
<p>
I like a lot of it. Obviously needs tests in the new class. A few questions, which may betray ignorance:
</p>
<ul><li>Is it okay that we don't overwrite the global variables? I like that in principle, but there is similar confusion at times with PolynomialRings that have an x which isn't in SR while <code>sage: x</code> still returns the symbolic <code>x</code>. Just asking.
</li><li>One thing I don't like, or need explanation for, is the confusion in multi-index situations between the string (not LaTeX) reps of <code>x_{1}_{1}</code> and <code>x_{11}</code>, which are apparently both <code>x11</code>. It's almost <em>too</em> flexible, because you don't have to say how many indices there will be ahead of time - which is useful, of course, but there might have to be something with the reps.
</li><li>Should <code>SymbolSequence</code> inherit from something?
</li></ul>
TicketmjoSat, 01 Dec 2012 17:02:31 GMTattachment set
https://trac.sagemath.org/ticket/11576
https://trac.sagemath.org/ticket/11576
<ul>
<li><strong>attachment</strong>
set to <em>sage-trac_11576.patch</em>
</li>
</ul>
<p>
Add tests for the individual class methods
</p>
TicketmjoSat, 01 Dec 2012 17:21:17 GMT
https://trac.sagemath.org/ticket/11576#comment:11
https://trac.sagemath.org/ticket/11576#comment:11
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/11576#comment:10" title="Comment 10">kcrisman</a>:
</p>
<blockquote class="citation">
<p>
I like a lot of it. Obviously needs tests in the new class.
</p>
</blockquote>
<p>
<br />
</p>
<p>
I just updated the patch, each of the individual class methods now have their own examples.
</p>
<p>
<br />
</p>
<blockquote class="citation">
<p>
A few questions, which may betray ignorance:
</p>
<ul><li>Is it okay that we don't overwrite the global variables? I like that in principle, but there is similar confusion at times with PolynomialRings that have an x which isn't in SR while <code>sage: x</code> still returns the symbolic <code>x</code>. Just asking.
</li></ul></blockquote>
<p>
<br />
</p>
<p>
Having two different <code>x</code>-things floating around -- one a python variable and one a symbol name -- is a little bad, but clobbering them is in my opinion worse. Essentially what I'm trying to do is make this act like <code>SR.symbol()</code> instead of <code>SR.var()</code>.
</p>
<p>
The <code>symbol()</code> function leaves the variables alone:
</p>
<pre class="wiki">sage: y = 7
sage: SR.symbol('y')
y
sage: y
7
</pre><p>
Now there's a symbol floating around somewhere called <code>y</code>, but it isn't the python variable. The alternative is, in my opinion, much worse:
</p>
<pre class="wiki">sage: y = 7
sage: var('y')
y
sage: y
y
</pre><p>
You could argue that people expect that from <code>var()</code> these days, but they certainly wouldn't expect it from indexing some object. So this is what I <strong>don't</strong> want:
</p>
<pre class="wiki">sage: y1 = 7
sage: ys = SR.symbols('y')
sage: foo = ys[1]
sage: y1
y1
</pre><p>
The user never even wrote 'y1' anywhere, so I think it's dangerous to overwrite it.
</p>
<p>
<br />
</p>
<blockquote class="citation">
<ul><li>One thing I don't like, or need explanation for, is the confusion in multi-index situations between the string (not LaTeX) reps of <code>x_{1}_{1}</code> and <code>x_{11}</code>, which are apparently both <code>x11</code>. It's almost <em>too</em> flexible, because you don't have to say how many indices there will be ahead of time - which is useful, of course, but there might have to be something with the reps.
</li></ul></blockquote>
<p>
<br />
</p>
<p>
You're right... my motivation was simply, "make it easy to create a bunch of symbols," but I don't like this:
</p>
<pre class="wiki">sage: a = SR.symbols('a')
sage: bool(a[11] == a[1,1])
True
</pre><p>
The next-best thing I can think of is to use underscores, like the latex representation. This makes everything a little uglier, but avoids the ambiguity. Thoughts?
</p>
<p>
<br />
</p>
<blockquote class="citation">
<ul><li>Should <code>SymbolSequence</code> inherit from something?
</li></ul></blockquote>
<p>
<br />
</p>
<p>
I dunno. I think this is the first standalone class I've written for the sage library. Anyone know?
</p>
TicketmjoSat, 01 Dec 2012 17:34:11 GMTattachment set
https://trac.sagemath.org/ticket/11576
https://trac.sagemath.org/ticket/11576
<ul>
<li><strong>attachment</strong>
set to <em>underscores.patch</em>
</li>
</ul>
<p>
Use underscores to separate indices (applies on top of previous patch)
</p>
TicketmjoSat, 01 Dec 2012 17:35:40 GMT
https://trac.sagemath.org/ticket/11576#comment:12
https://trac.sagemath.org/ticket/11576#comment:12
<p>
If you want to see what it would look like with underscores, the patch I just posted adds them and has a test for subscript collision.
</p>
TicketkcrismanSat, 01 Dec 2012 17:41:54 GMT
https://trac.sagemath.org/ticket/11576#comment:13
https://trac.sagemath.org/ticket/11576#comment:13
<p>
Thanks for your comments - I think I agree with most of that.
</p>
<blockquote class="citation">
<p>
You're right... my motivation was simply, "make it easy to create a bunch of symbols," but I don't like this:
</p>
<pre class="wiki">sage: a = SR.symbols('a')
sage: bool(a[11] == a[1,1])
True
</pre><p>
The next-best thing I can think of is to use underscores, like the latex representation. This makes everything a little uglier, but avoids the ambiguity. Thoughts?
</p>
</blockquote>
<p>
Yeah, I'm not sure either. I'd appreciate input from people who actually <em>use</em> this stuff! I don't make such variable sequences in my own work. I see your new patch, but it still looks like
</p>
<pre class="wiki">a_1_2_3_4_5_6
</pre><p>
could represent a whole Catalan stew of different indices with the nested thing going on. I don't feel like making this decision, but it would be a shame for something to languish. Maybe Burcin or Florent have ideas - it <em>is</em> true that it's silly to reinvent the wheel if Ginac has this already, but we do now have a patch...
</p>
<p>
Also, what do you know about the memory leak issue in <a class="ticket" href="https://trac.sagemath.org/ticket/11576#comment:4" title="Comment 4">comment:4</a>?
</p>
<blockquote class="citation">
<blockquote class="citation">
<ul><li>Should <code>SymbolSequence</code> inherit from something?
</li></ul></blockquote>
<p>
I dunno. I think this is the first standalone class I've written for the sage library. Anyone know?
</p>
</blockquote>
<p>
Usually at least from <code>SageObject</code>, I guess. Would <code>Symbol</code> be appropriate, or maybe not? What generic methods would you want available to this class via inheritance that <code>Symbol</code> already has? I guess each individual symbol has everything that <code>Symbol</code> has.
</p>
TicketmjoSun, 06 Jan 2013 21:12:05 GMT
https://trac.sagemath.org/ticket/11576#comment:14
https://trac.sagemath.org/ticket/11576#comment:14
<p>
I'm not thinking very good right now -- can you give an example of e.g. <code>a_1_2_3</code> being ambiguous?
</p>
TicketkcrismanMon, 07 Jan 2013 14:06:09 GMT
https://trac.sagemath.org/ticket/11576#comment:15
https://trac.sagemath.org/ticket/11576#comment:15
<p>
Is it <code>a_1_{2_3}</code> or <code>a_{123}</code>? I guess I was thinking of subscripts of subscripts. Maybe that's not at issue here; I can't quite reconstruct my thinking then either. Wasn't there some nesting somewhere in your patch?
</p>
TicketmjoMon, 07 Jan 2013 14:39:02 GMT
https://trac.sagemath.org/ticket/11576#comment:16
https://trac.sagemath.org/ticket/11576#comment:16
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/11576#comment:15" title="Comment 15">kcrisman</a>:
</p>
<blockquote class="citation">
<p>
Is it <code>a_1_{2_3}</code> or <code>a_{123}</code>? I guess I was thinking of subscripts of subscripts. Maybe that's not at issue here; I can't quite reconstruct my thinking then either. Wasn't there some nesting somewhere in your patch?
</p>
</blockquote>
<p>
It's <code>a_{1)_{2}_{3}</code>. To create <code>a_{123}</code>, you'd call <code>a[123]</code>. There is nesting going on, but each "level" should be separated by an underscore now. In fact now that we're only accepting the square brackets, I think commas should map directly to underscores. For example,
</p>
<pre class="wiki">sage: xs[3, 8:10, 2:4]
[xs_3_8_2, xs_3_8_3, xs_3_9_2, xs_3_9_3]
</pre><p>
You're allowed to think of <code>a[2,3]</code> as <code>a_{2_3}</code>, but I don't think there's any way to create it distinct from <code>a_{2}_{3}</code>. The implementation creates an <code>a_{2}</code> first, and then subscripts that with 3.
</p>
<p>
I still think the underscores are a little ugly, but I've gotten used to them and it's preferable to having <code>a[1,1] - a[11] == 0</code>.
</p>
TicketkcrismanMon, 07 Jan 2013 14:42:21 GMT
https://trac.sagemath.org/ticket/11576#comment:17
https://trac.sagemath.org/ticket/11576#comment:17
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/11576#comment:16" title="Comment 16">mjo</a>:
</p>
<blockquote class="citation">
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/11576#comment:15" title="Comment 15">kcrisman</a>:
</p>
<blockquote class="citation">
<p>
Is it <code>a_1_{2_3}</code> or <code>a_{123}</code>? I guess I was thinking of subscripts of subscripts. Maybe that's not at issue here; I can't quite reconstruct my thinking then either. Wasn't there some nesting somewhere in your patch?
</p>
</blockquote>
<p>
It's <code>a_{1)_{2}_{3}</code>. To create <code>a_{123}</code>, you'd call <code>a[123]</code>. There is nesting going on, but each "level" should be separated by an underscore now. In fact now that we're only accepting the square brackets, I think commas should map directly to underscores. For example,
</p>
</blockquote>
<p>
So how would you distinguish <code>a_(1,2_3)</code> from <code>a_(1_2,3)</code> - or is that not at issue? That's what I was getting at, I think.
</p>
TicketmjoMon, 07 Jan 2013 14:50:19 GMT
https://trac.sagemath.org/ticket/11576#comment:18
https://trac.sagemath.org/ticket/11576#comment:18
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/11576#comment:17" title="Comment 17">kcrisman</a>:
</p>
<blockquote class="citation">
<p>
So how would you distinguish <code>a_(1,2_3)</code> from <code>a_(1_2,3)</code> - or is that not at issue? That's what I was getting at, I think.
</p>
</blockquote>
<p>
<br />
</p>
<p>
Neither of those are directly constructible. It might be useful to make the bijection in your head, but <code>a_{1}_{2}_{3}</code> is all that my patch allows you to create. The user assigns alternate semantics at his own risk =)
</p>
TicketkcrismanMon, 07 Jan 2013 15:15:49 GMT
https://trac.sagemath.org/ticket/11576#comment:19
https://trac.sagemath.org/ticket/11576#comment:19
<blockquote class="citation">
<p>
Neither of those are directly constructible. It might be useful to make the bijection in your head, but <code>a_{1}_{2}_{3}</code> is all that my patch allows you to create. The user assigns alternate semantics at his own risk =)
</p>
</blockquote>
<p>
Okay, so there is only <code>a_(1,2,3)</code> and <code>a_(12,3)</code> and <code>a_(123)</code> and friends, everything at one level.
</p>
<p>
I'd probably be okay with this then, but I'd really like some input from Burcin and/or Florent as to how compatible this might be with <em>eventually</em> using Ginac's capabilities. I would view doing this natively as a temporary measure. For instance, have you checked how this performs with respect to Nils' comment about memory usage?
</p>
TicketmjoMon, 07 Jan 2013 15:31:49 GMT
https://trac.sagemath.org/ticket/11576#comment:20
https://trac.sagemath.org/ticket/11576#comment:20
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/11576#comment:19" title="Comment 19">kcrisman</a>:
</p>
<blockquote class="citation">
<p>
I'd probably be okay with this then, but I'd really like some input from Burcin and/or Florent as to how compatible this might be with <em>eventually</em> using Ginac's capabilities. I would view doing this natively as a temporary measure. For instance, have you checked how this performs with respect to Nils' comment about memory usage?
</p>
</blockquote>
<p>
Hopefully we could just deprecate <code>SR.Symbols()</code> and tell people to use the indexed symbols instead. Which memory usage comment do you mean?
</p>
TicketkcrismanMon, 07 Jan 2013 15:39:48 GMT
https://trac.sagemath.org/ticket/11576#comment:21
https://trac.sagemath.org/ticket/11576#comment:21
<blockquote class="citation">
<p>
<em>eventually</em> using Ginac's capabilities. I would view doing this natively as a temporary measure. For instance, have you checked how this performs with respect to Nils' comment about memory usage?
</p>
<p>
Hopefully we could just deprecate <code>SR.Symbols()</code>
</p>
</blockquote>
<p>
? isn't this the new thing you just added? (well, <code>symbols</code>, but I don't see <code>Symbols</code> or <code>Symbol</code> - or did you mean <code>symbol</code>?)
</p>
<blockquote class="citation">
<blockquote>
<p>
and tell people to use the indexed symbols instead. Which memory usage comment do you mean?
</p>
</blockquote>
</blockquote>
<p>
<a class="ticket" href="https://trac.sagemath.org/ticket/11576#comment:4" title="Comment 4">comment:4</a>
</p>
TicketburcinMon, 07 Jan 2013 15:50:52 GMTattachment set
https://trac.sagemath.org/ticket/11576
https://trac.sagemath.org/ticket/11576
<ul>
<li><strong>attachment</strong>
set to <em>trac_11576-indexed_expression.20130107.patch</em>
</li>
</ul>
TicketburcinMon, 07 Jan 2013 16:01:09 GMT
https://trac.sagemath.org/ticket/11576#comment:22
https://trac.sagemath.org/ticket/11576#comment:22
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/11576#comment:4" title="Comment 4">nbruin</a>:
</p>
<blockquote class="citation">
<p>
Perhaps another motivation for using GiNaC functionality for this is that the alternative approach leaks. Watch the memory usage of
</p>
<pre class="wiki">sage: for i in range(10^8):
....: l=1+SR.symbol("a%s"%str(i))
</pre><p>
Perhaps <a class="missing wiki">GiNaCs?</a> indexed variables are less prone to leaking?
</p>
</blockquote>
<p>
I'm not sure if anything is _leaking_ here. Pynac stores a symbol lookup table, in order to give you the same variable if you ask for SR.symbol('x') twice. As you pass new strings to this function, the lookup table grows. Note that if you repeat the same loop, memory usage does not grow.
</p>
<p>
I agree that this is a good argument for using indexed variables from Pynac/GiNaC instead of trying to invent our own. Here are some numbers:
</p>
<pre class="wiki">sage: get_memory_usage()
866.94921875
sage: x.ind[5]
x.5
sage: for i in range(10000):
....: t = x.ind[i]
....:
sage: get_memory_usage()
867.2265625
sage: for i in range(10000):
....: t = SR.symbol('a_'+str(i))
....:
sage: get_memory_usage()
870.4140625
</pre><p>
This is with <a class="attachment" href="https://trac.sagemath.org/attachment/ticket/11576/trac_11576-indexed_expression.20130107.patch" title="Attachment 'trac_11576-indexed_expression.20130107.patch' in Ticket #11576">trac_11576-indexed_expression.20130107.patch</a><a class="trac-rawlink" href="https://trac.sagemath.org/raw-attachment/ticket/11576/trac_11576-indexed_expression.20130107.patch" title="Download"></a>. I just rebased the patch I was working on at the Sage days in Cernay.
</p>
<p>
Apart from lack of documentation and tests, this patch provides the functionality discussed in this ticket. The main problem holding it up was the fact that it allows you to use arbitrary Python objects as indices, and these are not handled properly when the expression is being converted to Maxima.
</p>
TicketkcrismanMon, 07 Jan 2013 16:17:28 GMT
https://trac.sagemath.org/ticket/11576#comment:23
https://trac.sagemath.org/ticket/11576#comment:23
<blockquote class="citation">
<p>
I just rebased the patch I was working on at the Sage days in Cernay.
</p>
</blockquote>
<p>
With Sage 5.6.beta2:
</p>
<pre class="wiki">patching file sage/libs/ginac.pxd
Hunk #2 FAILED at 101
</pre><p>
The two lines with <code>dbgprint</code> are gone in beta2.
</p>
<blockquote class="citation">
<p>
Apart from lack of documentation and tests, this patch provides the functionality discussed in this ticket. The main problem holding it up was the fact that it allows you to use arbitrary Python objects as indices, and these are not handled properly when the expression is being converted to Maxima.
</p>
</blockquote>
<p>
We could just disallow anything that didn't fit a number or regular expression with commas or something.
</p>
<p>
This patch is somewhat different also in that
</p>
<pre class="wiki">raise NotImplementedError("don't know what to do with multiple indices yet!")
</pre><p>
and I have to say I do like being able to use slice notation directly on the variable.
</p>
<p>
Also,
</p>
<pre class="wiki">sage: x.ind[2:6]
x.2
</pre><p>
seems counterintuitive for a couple reasons - the dot suggests an attribute to me in Python, and where are the other indices? I'm also not sure I like allowing indexing of non-variables, but that might be ignorance and fear speaking. I just don't know what
</p>
<pre class="wiki">sage: ex.ind[p]
(x + 1).[2, 1, 3]
</pre><p>
really means.
</p>
TicketmjoMon, 07 Jan 2013 19:54:47 GMT
https://trac.sagemath.org/ticket/11576#comment:24
https://trac.sagemath.org/ticket/11576#comment:24
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/11576#comment:21" title="Comment 21">kcrisman</a>:
</p>
<blockquote class="citation">
<blockquote class="citation">
<p>
<em>eventually</em> using Ginac's capabilities. I would view doing this natively as a temporary measure. For instance, have you checked how this performs with respect to Nils' comment about memory usage?
</p>
<p>
Hopefully we could just deprecate <code>SR.Symbols()</code>
</p>
</blockquote>
<p>
? isn't this the new thing you just added? (well, <code>symbols</code>, but I don't see <code>Symbols</code> or <code>Symbol</code> - or did you mean <code>symbol</code>?)
</p>
</blockquote>
<p>
<br />
</p>
<p>
Typo, I meant lowercase <code>SR.symbols()</code>. Yeah, it's the thing I just added. But if there's ever a simpler solution with the same functionality, we could deprecate it. If I can just call <code>x[1, 2:3]</code> directly, then there's no need to do <code>x = SR.symbols(); x[1, 2:3]</code>.
</p>
<p>
<br />
</p>
<blockquote class="citation">
<blockquote class="citation">
<blockquote>
<p>
and tell people to use the indexed symbols instead. Which memory usage comment do you mean?
</p>
</blockquote>
</blockquote>
<p>
<a class="ticket" href="https://trac.sagemath.org/ticket/11576#comment:4" title="Comment 4">comment:4</a>
</p>
</blockquote>
<p>
<br />
</p>
<p>
Oh, right. As Burcin pointed out, there's no leak, it's just creating symbols and they use up some memory. If you stick to the same symbols, memory usage won't grow.
</p>
<p>
I can see the value in being able to use arbitrary subscripts, but I also like being able to do the common case quickly and easily. <code>x.ind[1]</code> is a little weird, especially if we can't use multiple indices. I also don't think it makes much sense to subscript <code>SR(1).ind[5]</code> and have it display <code>1.5</code>.
</p>
TicketnbruinMon, 07 Jan 2013 20:58:16 GMT
https://trac.sagemath.org/ticket/11576#comment:25
https://trac.sagemath.org/ticket/11576#comment:25
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/11576#comment:24" title="Comment 24">mjo</a>:
</p>
<blockquote class="citation">
<p>
Oh, right. As Burcin pointed out, there's no leak, it's just creating symbols and they use up some memory. If you stick to the same symbols, memory usage won't grow.
</p>
</blockquote>
<p>
Indeed. The issue is that once you can create sequences of symbols <em>easily</em>, people might start doing that more and hence memory footprint might become more of an issue.
</p>
<p>
When creating arbitrary symbols, one cannot really avoid adding an entry in a table somewhere. In principle, weak caching strategies should allow reclaiming that at some point, but coordination across various interfaces and libraries (specifically maxima) will make it <em>very</em> hard to estimate the lifetime of a symbol properly. Hence, a permanent memory cost for the creation of a new symbol is so hard to avoid that we should probably consider it unavoidable.
</p>
<p>
It would be nice if the permanent cost of a symbol sequence is only for the sequence, not for every member generated in it. Note that
</p>
<pre class="wiki">for i in [1..10^8]:
print sin(i)
</pre><p>
does <em>not</em> explode in memory, because the entry <code>sin</code> in the symbol table gets combined with an <em>argument</em> <code>i</code> that varies. No further permanent entries are made. I bet that <code>Ginac</code> does something similar for its sequences of symbols (i.e., it probably stores a base symbol together with an index. Essentially a function call, but labelled in such a way that it gets handles as an atomic, free, entity. Implementation is really easy in theory: any simplification/rewrite routine should simply <em>not</em> descend further into the tree)
</p>
<p>
Ideally, one would have to find a way of encoding such symbols for <code>maxima</code> and <code>maxima_lib</code> as well, to avoid the (permanent) translation tables there to blow up as well. I don't know how easy it is to create structures capable of storing a serial number AND allowing differentiating etc. against it. Perhaps just encoding as an (uninterned) symbol might work. If you encode the string heavily enough you may be able to recognize them on the way out. For <code>maxima_lib</code> you could also store a flag as an attribute on the symbol and test for that in conversion back to sage (maxima uses something like this for "dummy" variables somewhere).
</p>
<p>
However, if you can't figure out how to do this for maxima, at least you may be able to avoid memory blow-up as long as these symbols don't touch various interfaces.
</p>
<p>
I think that if you think this is worth doing, it's worth doing it well. I think sage has now matured to the point where putting in features via cheap hacks does more harm than good. (In fact the cheap hacks that were necessary to get the project up and running originally are now hurting further development.)
</p>
TicketeviatarbachMon, 03 Jun 2013 22:22:52 GMTcc changed
https://trac.sagemath.org/ticket/11576#comment:26
https://trac.sagemath.org/ticket/11576#comment:26
<ul>
<li><strong>cc</strong>
<em>eviatarbach</em> added
</li>
</ul>
TicketvbraunTue, 18 Jun 2013 17:56:38 GMTattachment set
https://trac.sagemath.org/ticket/11576
https://trac.sagemath.org/ticket/11576
<ul>
<li><strong>attachment</strong>
set to <em>trac_11576-indexed_expression.patch</em>
</li>
</ul>
<p>
Rebased patch
</p>
TicketvbraunTue, 18 Jun 2013 17:57:17 GMTstatus, description changed
https://trac.sagemath.org/ticket/11576#comment:27
https://trac.sagemath.org/ticket/11576#comment:27
<ul>
<li><strong>status</strong>
changed from <em>new</em> to <em>needs_review</em>
</li>
<li><strong>description</strong>
modified (<a href="/ticket/11576?action=diff&version=27">diff</a>)
</li>
</ul>
<p>
Rebased on top of Sage-5.10.rc2
</p>
TicketvbraunTue, 18 Jun 2013 18:10:22 GMTstatus changed
https://trac.sagemath.org/ticket/11576#comment:28
https://trac.sagemath.org/ticket/11576#comment:28
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
</ul>
<p>
Doctest failures:
</p>
<pre class="wiki">sage -t sage/symbolic/getitem.pyx # 4 doctests failed
sage -t sage/symbolic/expression.pyx # 1 doctest failed
</pre>
TicketjdemeyerTue, 13 Aug 2013 15:35:53 GMTmilestone changed
https://trac.sagemath.org/ticket/11576#comment:29
https://trac.sagemath.org/ticket/11576#comment:29
<ul>
<li><strong>milestone</strong>
changed from <em>sage-5.11</em> to <em>sage-5.12</em>
</li>
</ul>
Ticketvbraun_spamThu, 30 Jan 2014 21:20:52 GMTmilestone changed
https://trac.sagemath.org/ticket/11576#comment:30
https://trac.sagemath.org/ticket/11576#comment:30
<ul>
<li><strong>milestone</strong>
changed from <em>sage-6.1</em> to <em>sage-6.2</em>
</li>
</ul>
Ticketvbraun_spamTue, 06 May 2014 15:20:58 GMTmilestone changed
https://trac.sagemath.org/ticket/11576#comment:31
https://trac.sagemath.org/ticket/11576#comment:31
<ul>
<li><strong>milestone</strong>
changed from <em>sage-6.2</em> to <em>sage-6.3</em>
</li>
</ul>
Ticketvbraun_spamSun, 10 Aug 2014 16:51:03 GMTmilestone changed
https://trac.sagemath.org/ticket/11576#comment:32
https://trac.sagemath.org/ticket/11576#comment:32
<ul>
<li><strong>milestone</strong>
changed from <em>sage-6.3</em> to <em>sage-6.4</em>
</li>
</ul>
TicketslelievreFri, 11 Nov 2016 06:09:17 GMTcc, description changed
https://trac.sagemath.org/ticket/11576#comment:33
https://trac.sagemath.org/ticket/11576#comment:33
<ul>
<li><strong>cc</strong>
<em>slelievre</em> added
</li>
<li><strong>description</strong>
modified (<a href="/ticket/11576?action=diff&version=33">diff</a>)
</li>
</ul>
TicketkcrismanFri, 14 Apr 2017 14:36:51 GMT
https://trac.sagemath.org/ticket/11576#comment:34
https://trac.sagemath.org/ticket/11576#comment:34
<p>
Conceivably related: <a class="closed ticket" href="https://trac.sagemath.org/ticket/22809" title="enhancement: Pass number of variables to polygens (closed: fixed)">#22809</a>
</p>
TicketmforetsFri, 21 Apr 2017 11:48:50 GMT
https://trac.sagemath.org/ticket/11576#comment:35
https://trac.sagemath.org/ticket/11576#comment:35
<p>
please see also the recent: <a class="needs_info ticket" href="https://trac.sagemath.org/ticket/22813" title="enhancement: Pass number of variables to var (needs_info)">#22813</a>. the motivation is the 1st sentence of this ticket's description,
</p>
<blockquote class="citation">
<p>
People are always asking how to get sequences of variables, like a1,a2,a3,a4 or the like.
</p>
</blockquote>
<p>
It has been suggested to not overwrite global variables as <code>var('a0 a1 a2')</code> would normally do, but instead return the tuple <code>a</code> without injection.
</p>
<p>
certainly less comprehensive than this ticket, but related. if i can partner with any of you, i'm willing to (try to) implement the updates needed for this one / to fix doctests.
</p>
Ticket