Sage: Ticket #12659: build the sage library in place
https://trac.sagemath.org/ticket/12659
<p>
Use distutils to build the Sage library inplace to save on space / make it clearer which files are actually being used in runtime.
</p>
<p>
See the discussion at <a class="ext-link" href="http://groups.google.com/group/sage-devel/browse_thread/thread/d5aef15acc4d178b"><span class="icon"></span>http://groups.google.com/group/sage-devel/browse_thread/thread/d5aef15acc4d178b</a>
</p>
en-usSagehttps://trac.sagemath.org/chrome/site/logo_sagemath_trac.png
https://trac.sagemath.org/ticket/12659
Trac 1.1.6mhansenTue, 13 Mar 2012 05:12:42 GMTstatus changed; cc set
https://trac.sagemath.org/ticket/12659#comment:1
https://trac.sagemath.org/ticket/12659#comment:1
<ul>
<li><strong>cc</strong>
<em>wstein</em> added
</li>
<li><strong>status</strong>
changed from <em>new</em> to <em>needs_review</em>
</li>
</ul>
TicketmhansenTue, 13 Mar 2012 05:29:12 GMTdescription changed
https://trac.sagemath.org/ticket/12659#comment:2
https://trac.sagemath.org/ticket/12659#comment:2
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/12659?action=diff&version=2">diff</a>)
</li>
</ul>
TicketwasTue, 13 Mar 2012 06:21:19 GMT
https://trac.sagemath.org/ticket/12659#comment:3
https://trac.sagemath.org/ticket/12659#comment:3
<p>
Hey Mike, could you explain briefly why you delete (not move) all this code? Just curious.
</p>
<pre class="wiki">747 # if cython worked, copy the file to the build directory
748 pyx_inst_file = '%s/%s'%(SITE_PACKAGES, f)
749 retval = os.system('cp %s %s 2>/dev/null'%(f, pyx_inst_file))
750 # we could do this more elegantly -- load the files, use
751 # os.path.exists to check that they exist, etc. ... but the
752 # *vast* majority of the time, the copy just works. so this is
753 # just specializing for the most common use case.
754 if retval:
755 dirname, filename = os.path.split(pyx_inst_file)
756 try:
757 os.makedirs(dirname)
758 except OSError, e:
759 assert e.errno==errno.EEXIST, 'Cannot create %s.' % dirname
760 retval = os.system('cp %s %s 2>/dev/null'%(f, pyx_inst_file))
761 if retval:
762 raise OSError, "cannot copy %s to %s"%(f,pyx_inst_file)
763 print "%s --> %s"%(f, pyx_inst_file)
764
743 return r
</pre>
TicketmhansenTue, 13 Mar 2012 06:31:18 GMT
https://trac.sagemath.org/ticket/12659#comment:4
https://trac.sagemath.org/ticket/12659#comment:4
<p>
I removed it since we don't need to copy those files anymore -- they're already in the right place.
</p>
TicketkiniTue, 13 Mar 2012 06:34:16 GMTcc changed
https://trac.sagemath.org/ticket/12659#comment:5
https://trac.sagemath.org/ticket/12659#comment:5
<ul>
<li><strong>cc</strong>
<em>kini</em> added
</li>
</ul>
TicketwasTue, 13 Mar 2012 07:17:07 GMT
https://trac.sagemath.org/ticket/12659#comment:6
https://trac.sagemath.org/ticket/12659#comment:6
<p>
Mike -- that makes sense. However, why do we have this code still:
</p>
<pre class="wiki"> 422 if not self.inplace:
423 relative_ext_dir = os.path.split(relative_ext_filename)[0]
424 prefixes = ['', self.build_lib, self.build_temp]
425 for prefix in prefixes:
426 path = os.path.join(prefix, relative_ext_dir)
427 try:
428 os.makedirs(path)
429 except OSError, e:
430 assert e.errno==errno.EEXIST, 'Cannot create %s.' % path
</pre><p>
Wouldn't the above only be run if we are not building in place? It seems to me that it would be best to either keep the code discussed in the comment above, or should replace all the code above by
</p>
<pre class="wiki"> 422 if not self.inplace:
423 ERROR MESSAGE
</pre><p>
I'm in favor of the latter, unless I'm misunderstanding things.
</p>
TicketmhansenTue, 13 Mar 2012 07:22:08 GMTattachment set
https://trac.sagemath.org/ticket/12659
https://trac.sagemath.org/ticket/12659
<ul>
<li><strong>attachment</strong>
set to <em>trac_12659.patch</em>
</li>
</ul>
TicketmhansenTue, 13 Mar 2012 07:22:28 GMT
https://trac.sagemath.org/ticket/12659#comment:7
https://trac.sagemath.org/ticket/12659#comment:7
<p>
Agreed -- I've put an updated patch.
</p>
TicketwasTue, 13 Mar 2012 07:38:22 GMT
https://trac.sagemath.org/ticket/12659#comment:8
https://trac.sagemath.org/ticket/12659#comment:8
<p>
Regarding everything else about this patch, it seems to work perfectly (though I'm running a few tests), and in fact is REALLY, REALLY AWESOME. This must go into Sage ASAP! It's wonderful.
</p>
TicketmhansenTue, 13 Mar 2012 07:40:35 GMT
https://trac.sagemath.org/ticket/12659#comment:9
https://trac.sagemath.org/ticket/12659#comment:9
<p>
Thanks!
</p>
TicketmderickxTue, 13 Mar 2012 13:33:17 GMT
https://trac.sagemath.org/ticket/12659#comment:10
https://trac.sagemath.org/ticket/12659#comment:10
<p>
It seems to me that if this get's merged then we should undo the changes at <a class="closed ticket" href="https://trac.sagemath.org/ticket/11235" title="defect: Make the ipython edit magic command edit the right file and show both ... (closed: fixed)">#11235</a>
</p>
TicketwasWed, 14 Mar 2012 02:58:59 GMTstatus changed
https://trac.sagemath.org/ticket/12659#comment:11
https://trac.sagemath.org/ticket/12659#comment:11
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>positive_review</em>
</li>
</ul>
<p>
I'm happy with this code.
</p>
<p>
And indeed the work at <a class="closed ticket" href="https://trac.sagemath.org/ticket/11235" title="defect: Make the ipython edit magic command edit the right file and show both ... (closed: fixed)">#11235</a> seems not necessary anymore.
</p>
TicketjdemeyerWed, 14 Mar 2012 09:12:46 GMTreviewer set
https://trac.sagemath.org/ticket/12659#comment:12
https://trac.sagemath.org/ticket/12659#comment:12
<ul>
<li><strong>reviewer</strong>
set to <em>William Stein</em>
</li>
</ul>
TicketjdemeyerThu, 15 Mar 2012 14:29:48 GMT
https://trac.sagemath.org/ticket/12659#comment:13
https://trac.sagemath.org/ticket/12659#comment:13
<p>
Could this have caused
</p>
<pre class="wiki">sage -t "devel/sage-main/sage/graphs/graph_decompositions/rankwidth.pyx"
**********************************************************************
File "/tmp/jdemeyer/merger/sage-5.0.beta9/devel/sage-main/sage/graphs/graph_decompositions/rankwidth.pyx", line 47:
sage: rw, tree = g.rank_decomposition()
Exception raised:
Traceback (most recent call last):
File "/tmp/jdemeyer/merger/sage-5.0.beta9/local/bin/ncadoctest.py", line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/tmp/jdemeyer/merger/sage-5.0.beta9/local/bin/sagedoctest.py", line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
File "/tmp/jdemeyer/merger/sage-5.0.beta9/local/bin/ncadoctest.py", line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_0[3]>", line 1, in <module>
rw, tree = g.rank_decomposition()###line 47:
sage: rw, tree = g.rank_decomposition()
File "/tmp/jdemeyer/merger/sage-5.0.beta9/local/lib/python/site-packages/sage/graphs/graph.py", line 2959, in rank_decomposition
from sage.graphs.graph_decompositions.rankwidth import rank_decomposition
ImportError: cannot import name rank_decomposition
**********************************************************************
</pre>
TicketjdemeyerThu, 15 Mar 2012 19:04:08 GMTstatus changed
https://trac.sagemath.org/ticket/12659#comment:14
https://trac.sagemath.org/ticket/12659#comment:14
<ul>
<li><strong>status</strong>
changed from <em>positive_review</em> to <em>needs_work</em>
</li>
</ul>
<p>
Indeed, it is. I have no explanation for the fact that only the import of this particular module fails. It was recently added in <a class="closed ticket" href="https://trac.sagemath.org/ticket/11754" title="enhancement: Computation of rank-decompositions in Sage (closed: fixed)">#11754</a>, but how should that be relevant?
</p>
TicketmhansenFri, 16 Mar 2012 00:50:43 GMT
https://trac.sagemath.org/ticket/12659#comment:15
https://trac.sagemath.org/ticket/12659#comment:15
<p>
How did you get that particular error? How did you merge in the patch here?
</p>
TicketkcrismanFri, 16 Mar 2012 01:39:40 GMT
https://trac.sagemath.org/ticket/12659#comment:16
https://trac.sagemath.org/ticket/12659#comment:16
<p>
Dumbest question ever, I'm sure, but I'll ask it - will this affect upgrades at all?
</p>
TicketjdemeyerFri, 16 Mar 2012 08:35:01 GMT
https://trac.sagemath.org/ticket/12659#comment:17
https://trac.sagemath.org/ticket/12659#comment:17
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12659#comment:15" title="Comment 15">mhansen</a>:
</p>
<blockquote class="citation">
<p>
How did you get that particular error? How did you merge in the patch here?
</p>
</blockquote>
<p>
Apply the patch, sdist, build Sage from scratch.
</p>
TicketjdemeyerFri, 16 Mar 2012 08:35:51 GMT
https://trac.sagemath.org/ticket/12659#comment:18
https://trac.sagemath.org/ticket/12659#comment:18
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12659#comment:16" title="Comment 16">kcrisman</a>:
</p>
<blockquote class="citation">
<p>
Dumbest question ever, I'm sure, but I'll ask it - will this affect upgrades at all?
</p>
</blockquote>
<p>
I certainly don't think this is a dumb question, in fact it is a very good question. I don't know the answer though.
</p>
TicketmhansenFri, 16 Mar 2012 15:56:29 GMT
https://trac.sagemath.org/ticket/12659#comment:19
https://trac.sagemath.org/ticket/12659#comment:19
<p>
Upgrades should be fine -- when <code>sage -b</code> gets run, the site-packages link will update to the new location, and the new extension modules should be built in place.
</p>
TicketkcrismanFri, 16 Mar 2012 16:07:00 GMT
https://trac.sagemath.org/ticket/12659#comment:20
https://trac.sagemath.org/ticket/12659#comment:20
<blockquote class="citation">
<p>
Upgrades should be fine -- when <code>sage -b</code> gets run, the site-packages link will update to the new location, and the new extension modules should be built in place.
</p>
</blockquote>
<p>
So the first time an upgrade happens, the entire Sage library will have to rebuild its extensions and then put them in the new location, correct?
</p>
TicketmhansenFri, 16 Mar 2012 16:15:04 GMT
https://trac.sagemath.org/ticket/12659#comment:21
https://trac.sagemath.org/ticket/12659#comment:21
<p>
Yes -- it does not try to copy existing extension modules over.
</p>
TicketjhpalmieriFri, 16 Mar 2012 16:59:58 GMT
https://trac.sagemath.org/ticket/12659#comment:22
https://trac.sagemath.org/ticket/12659#comment:22
<p>
By the way, I had the same experience as Jeroen: I applied the patches, did <code>./sage -b</code> and <code>./sage --sdist ...</code>. Then I built Sage from the resulting tar file, and got doctest failures:
</p>
<pre class="wiki"> sage -t --long -force_lib devel/sage/sage/graphs/graph.py # 4 doctests failed
sage -t --long -force_lib devel/sage/sage/graphs/graph_decompositions/rankwidth.pyx # 11 doctests failed
</pre><p>
Is the issue the C code in the directory <code>sage/graphs/graph_decompositions/rankwidth/</code>? Somehow that code is now not being found by the Python and Cython code that uses it?
</p>
TicketmhansenFri, 16 Mar 2012 17:38:40 GMT
https://trac.sagemath.org/ticket/12659#comment:23
https://trac.sagemath.org/ticket/12659#comment:23
<p>
Looking at <a class="closed ticket" href="https://trac.sagemath.org/ticket/11754" title="enhancement: Computation of rank-decompositions in Sage (closed: fixed)">#11754</a>, I think the issue is that there is a extension module sage.graphs.graph_decompositions.rankwidth as well as a package (with <code>__init__.py</code>) sage/graphs/decompositions/rankwidth/ . I think that should really be fixed as it's ambiguous which should be used.
</p>
TicketmhansenSat, 17 Mar 2012 17:01:07 GMTdependencies set
https://trac.sagemath.org/ticket/12659#comment:24
https://trac.sagemath.org/ticket/12659#comment:24
<ul>
<li><strong>dependencies</strong>
set to <em>#12684</em>
</li>
</ul>
<p>
The code here should work fine when <a class="closed ticket" href="https://trac.sagemath.org/ticket/12684" title="defect: Rename sage/graphs/graph_decompositions/rankwidth/ (closed: fixed)">#12684</a> is applied.
</p>
TicketmhansenSat, 17 Mar 2012 17:01:12 GMTstatus changed
https://trac.sagemath.org/ticket/12659#comment:25
https://trac.sagemath.org/ticket/12659#comment:25
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
</ul>
TicketjhpalmieriWed, 21 Mar 2012 14:24:40 GMTstatus changed
https://trac.sagemath.org/ticket/12659#comment:26
https://trac.sagemath.org/ticket/12659#comment:26
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
</ul>
<p>
The code at <a class="closed ticket" href="https://trac.sagemath.org/ticket/12684" title="defect: Rename sage/graphs/graph_decompositions/rankwidth/ (closed: fixed)">#12684</a> fixes the problems with the graph files. For me, though, the banner is not getting updated when I run <code>sage --sdist ...</code>. The file <code>devel/sage/sage/version.py</code> is correct, but the file <code>devel/sage/build/sage/version.py</code> is not. (This is after applying the patches here, running <code>./sage -b</code>, then <code>./sage --sdist ...</code>.)
</p>
TicketjhpalmieriThu, 29 Mar 2012 23:15:59 GMT
https://trac.sagemath.org/ticket/12659#comment:27
https://trac.sagemath.org/ticket/12659#comment:27
<p>
I took a fresh Sage tarball, applied the patch here to the sage spkg, and then built Sage. Then I ran <code>./sage --sdist ...</code>. I got this error:
</p>
<pre class="wiki">running build_ext
Traceback (most recent call last):
File "setup.py", line 983, in <module>
include_dirs = include_dirs)
File "/scratch/palmieri/12659/sage-5.0.beta12-gcc/local/lib/python/distutils/core.py", line 152, in setup
dist.run_commands()
File "/scratch/palmieri/12659/sage-5.0.beta12-gcc/local/lib/python/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/scratch/palmieri/12659/sage-5.0.beta12-gcc/local/lib/python/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/scratch/palmieri/12659/sage-5.0.beta12-gcc/local/lib/python/distutils/command/install.py", line 563, in run
self.run_command('build')
File "/scratch/palmieri/12659/sage-5.0.beta12-gcc/local/lib/python/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/scratch/palmieri/12659/sage-5.0.beta12-gcc/local/lib/python/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/scratch/palmieri/12659/sage-5.0.beta12-gcc/local/lib/python/distutils/command/build.py", line 127, in run
self.run_command(cmd_name)
File "/scratch/palmieri/12659/sage-5.0.beta12-gcc/local/lib/python/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/scratch/palmieri/12659/sage-5.0.beta12-gcc/local/lib/python/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/scratch/palmieri/12659/sage-5.0.beta12-gcc/local/lib/python/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "setup.py", line 318, in build_extensions
raise RuntimeError, "only in-place builds are currently supported"
RuntimeError: only in-place builds are currently supported
</pre><p>
This doesn't stop the build, but do we need to worry about it? If not, we should suppress it.
</p>
<p>
Also, as mentioned above, I took an existing Sage installation, applied the patch to devel/sage, and ran <code>sage --sdist ...</code>. I got the same error message. Also, in the existing Sage installation, while <code>devel/sage/sage/version.py</code> was correct, <code>devel/sage/build/sage/version.py</code> was not. <code>local/lib/python/site-packages/sage</code> linked to <code>build/sage/</code>, and this is what led to the incorrect sage-banner that I mentioned earlier. Should the installation instructions include forcibly relinking <code>local/lib/.../sage</code> to the correct thing, if applying to an existing Sage installation?
</p>
TicketjhpalmieriFri, 30 Mar 2012 20:59:46 GMT
https://trac.sagemath.org/ticket/12659#comment:28
https://trac.sagemath.org/ticket/12659#comment:28
<p>
I think the scripts patch should be this instead (that is, removing the old link in a different part of the script):
</p>
<div class="wiki-code"><div xmlns="http://www.w3.org/1999/xhtml" class="diff">
<ul class="entries">
<li class="entry">
<h2>
<a>sage-build</a>
</h2>
<pre>diff --git a/sage-build b/sage-build</pre>
<table class="trac-diff inline" summary="Differences" cellspacing="0">
<colgroup><col class="lineno" /><col class="lineno" /><col class="content" /></colgroup>
<thead>
<tr>
<th title="File a/sage-build">
a
</th>
<th title="File b/sage-build">
b
</th>
<td><em> build() {</em> </td>
</tr>
</thead>
<tbody class="unmod">
<tr>
<th>15</th><th>15</th><td class="l"><span> echo "sage: Building and installing modified Sage library files."</span></td>
</tr><tr>
<th>16</th><th>16</th><td class="l"><span> echo ""</span></td>
</tr><tr>
<th>17</th><th>17</th><td class="l"><span></span></td>
</tr>
</tbody><tbody class="add">
<tr class="first">
<th> </th><th>18</th><td class="r"><ins> # Remove this link. It will be recreated automatically; removing</ins></td>
</tr><tr>
<th> </th><th>19</th><td class="r"><ins> # it now allows for the transition to build Sage in place (trac</ins></td>
</tr><tr>
<th> </th><th>20</th><td class="r"><ins> # #12659), because it will replace a link to</ins></td>
</tr><tr>
<th> </th><th>21</th><td class="r"><ins> # devel/sage/build/sage with a link to devel/sage/sage.</ins></td>
</tr><tr class="last">
<th> </th><th>22</th><td class="r"><ins> rm -f "$SAGE_LOCAL/lib/python/site-packages/sage"</ins></td>
</tr>
</tbody><tbody class="unmod">
<tr>
<th>18</th><th>23</th><td class="l"><span> # install c_lib</span></td>
</tr><tr>
<th>19</th><th>24</th><td class="l"><span> # we're doing this here instead of setup.py for historical</span></td>
</tr><tr>
<th>20</th><th>25</th><td class="l"><span> # reasons:</span></td>
</tr>
</tbody>
</table>
</li>
</ul>
</div></div><p>
This doesn't help with the error message in my previous comment, but it might take care of the other issue.
</p>
TicketmhansenMon, 13 Aug 2012 10:22:04 GMTstatus changed; dependencies deleted
https://trac.sagemath.org/ticket/12659#comment:29
https://trac.sagemath.org/ticket/12659#comment:29
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>needs_review</em>
</li>
<li><strong>dependencies</strong>
<em>#12684</em> deleted
</li>
</ul>
<p>
I've updated trac_12659-script.patch which should fix all of the outstanding problems mentioned on this ticket.
</p>
TicketmhansenMon, 13 Aug 2012 21:24:19 GMTdependencies set
https://trac.sagemath.org/ticket/12659#comment:30
https://trac.sagemath.org/ticket/12659#comment:30
<ul>
<li><strong>dependencies</strong>
set to <em>#13363</em>
</li>
</ul>
<p>
I was wrong -- one doctest fails (due to a warning being shown) without the patch at <a class="closed ticket" href="https://trac.sagemath.org/ticket/13363" title="defect: Move sage/graphs/planarity/ to sage/graphs/planarity_c/ (closed: fixed)">#13363</a> added.
</p>
TicketjhpalmieriMon, 20 Aug 2012 22:36:57 GMT
https://trac.sagemath.org/ticket/12659#comment:31
https://trac.sagemath.org/ticket/12659#comment:31
<p>
Just for esthetics: could you fix the spacing in <a class="attachment" href="https://trac.sagemath.org/attachment/ticket/12659/trac_12659-script.patch" title="Attachment 'trac_12659-script.patch' in Ticket #12659">trac_12659-script.patch</a><a class="trac-rawlink" href="https://trac.sagemath.org/raw-attachment/ticket/12659/trac_12659-script.patch" title="Download"></a>?
</p>
<p>
The patches look okay to me, and it seems to work, but it would be nice if someone who knew the build process (in particular, setup.py for the Sage library) better took a look at this.
</p>
TicketmhansenMon, 20 Aug 2012 23:03:29 GMT
https://trac.sagemath.org/ticket/12659#comment:32
https://trac.sagemath.org/ticket/12659#comment:32
<p>
Fixed up the spacing.
</p>
TicketmhansenMon, 20 Aug 2012 23:06:41 GMTattachment set
https://trac.sagemath.org/ticket/12659
https://trac.sagemath.org/ticket/12659
<ul>
<li><strong>attachment</strong>
set to <em>trac_12659-script.patch</em>
</li>
</ul>
TicketjdemeyerWed, 05 Sep 2012 12:20:15 GMT
https://trac.sagemath.org/ticket/12659#comment:33
https://trac.sagemath.org/ticket/12659#comment:33
<p>
I am testing this ticket (and a bunch of others) on the buildbot. Upgrading fails: everything builds but Sage doesn't start up:
</p>
<pre class="wiki">Testing that Sage starts...
[2012-09-05 05:12:26]
Traceback (most recent call last):
File "/release/buildbot/sage/sage-1/sage_upgrade_4.5/build/sage-5.4.beta1/local/bin/sage-eval", line 4, in <module>
from sage.all import *
File "/release/buildbot/sage/sage-1/sage_upgrade_4.5/build/sage-5.4.beta1/local/lib/python2.7/site-packages/sage/all.py", line 73, in <module>
from sage.matrix.all import *
File "/release/buildbot/sage/sage-1/sage_upgrade_4.5/build/sage-5.4.beta1/local/lib/python2.7/site-packages/sage/matrix/all.py", line 1, in <module>
from matrix_space import MatrixSpace, is_MatrixSpace
File "/release/buildbot/sage/sage-1/sage_upgrade_4.5/build/sage-5.4.beta1/local/lib/python2.7/site-packages/sage/matrix/matrix_space.py", line 33, in <module>
import matrix
File "matrix.pyx", line 1, in init sage.matrix.matrix (sage/matrix/matrix.c:2076)
File "matrix2.pyx", line 1, in init sage.matrix.matrix2 (sage/matrix/matrix2.c:70164)
File "matrix1.pyx", line 1, in init sage.matrix.matrix1 (sage/matrix/matrix1.c:13377)
File "mutability.pxd", line 15, in init sage.matrix.matrix0 (sage/matrix/matrix0.c:29424)
ValueError: PyCapsule_GetPointer called with invalid PyCapsule object
Sage failed to start up.
</pre><p>
Running <code>./sage -ba-force</code> solves the problem.
</p>
TicketjdemeyerWed, 05 Sep 2012 20:42:35 GMTstatus changed; work_issues set
https://trac.sagemath.org/ticket/12659#comment:34
https://trac.sagemath.org/ticket/12659#comment:34
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>needs_work</em>
</li>
<li><strong>work_issues</strong>
set to <em>upgrading</em>
</li>
</ul>
TicketjdemeyerFri, 07 Jun 2013 13:09:07 GMTstatus, reviewer, milestone changed; author, work_issues deleted
https://trac.sagemath.org/ticket/12659#comment:35
https://trac.sagemath.org/ticket/12659#comment:35
<ul>
<li><strong>status</strong>
changed from <em>needs_work</em> to <em>positive_review</em>
</li>
<li><strong>reviewer</strong>
changed from <em>William Stein</em> to <em>Jeroen Demeyer</em>
</li>
<li><strong>author</strong>
<em>Mike Hansen</em> deleted
</li>
<li><strong>work_issues</strong>
<em>upgrading</em> deleted
</li>
<li><strong>milestone</strong>
changed from <em>sage-5.10</em> to <em>sage-duplicate/invalid/wontfix</em>
</li>
</ul>
<p>
Wontfix because of <a class="closed ticket" href="https://trac.sagemath.org/ticket/14570" title="enhancement: Cythonize Sage library out of tree (closed: fixed)">#14570</a> I assume?
</p>
TicketmhansenFri, 07 Jun 2013 16:35:55 GMT
https://trac.sagemath.org/ticket/12659#comment:36
https://trac.sagemath.org/ticket/12659#comment:36
<p>
I think that <a class="closed ticket" href="https://trac.sagemath.org/ticket/14570" title="enhancement: Cythonize Sage library out of tree (closed: fixed)">#14570</a> is orthogonal to this.
</p>
TicketjdemeyerSat, 08 Jun 2013 08:35:58 GMTstatus, reviewer, milestone changed; author set
https://trac.sagemath.org/ticket/12659#comment:37
https://trac.sagemath.org/ticket/12659#comment:37
<ul>
<li><strong>status</strong>
changed from <em>positive_review</em> to <em>needs_work</em>
</li>
<li><strong>reviewer</strong>
changed from <em>Jeroen Demeyer</em> to <em>William Stein</em>
</li>
<li><strong>author</strong>
set to <em>Mike Hansen</em>
</li>
<li><strong>milestone</strong>
changed from <em>sage-duplicate/invalid/wontfix</em> to <em>sage-5.11</em>
</li>
</ul>
TicketjdemeyerTue, 13 Aug 2013 15:35:53 GMTmilestone changed
https://trac.sagemath.org/ticket/12659#comment:38
https://trac.sagemath.org/ticket/12659#comment:38
<ul>
<li><strong>milestone</strong>
changed from <em>sage-5.11</em> to <em>sage-5.12</em>
</li>
</ul>
Ticketvbraun_spamThu, 30 Jan 2014 21:20:52 GMTmilestone changed
https://trac.sagemath.org/ticket/12659#comment:39
https://trac.sagemath.org/ticket/12659#comment:39
<ul>
<li><strong>milestone</strong>
changed from <em>sage-6.1</em> to <em>sage-6.2</em>
</li>
</ul>
Ticketvbraun_spamTue, 06 May 2014 15:20:58 GMTmilestone changed
https://trac.sagemath.org/ticket/12659#comment:40
https://trac.sagemath.org/ticket/12659#comment:40
<ul>
<li><strong>milestone</strong>
changed from <em>sage-6.2</em> to <em>sage-6.3</em>
</li>
</ul>
TicketnthieryThu, 03 Jul 2014 21:55:10 GMT
https://trac.sagemath.org/ticket/12659#comment:41
https://trac.sagemath.org/ticket/12659#comment:41
<p>
As I mentionned on sage-devel, I recently noticed some preliminary
work in src/setup.py toward compiling the Sage library in place,
instead of putting the produced .so and copies of the .py files in
src/build.
</p>
<p>
Compiling in place means saving 15s of <code>sage -b</code> each time I just
modify Python code, and I do that all the time; so we are speaking of
dozen of minutes per day. So I have been dreaming of it for
years. Besides, it would also remove one of the regular source of
confusions for our new contributors.
</p>
<p>
So I explored this further, and with very minimal further changes ,
this apparently seems to work smoothly: I haven't yet run all tests,
but Sage starts, the documentation compiles, etc.
</p>
<p>
See branch u/nthiery/compile_library_inplace:
</p>
<p>
<a class="ext-link" href="http://git.sagemath.org/sage.git/commit/?h=u/nthiery/compile_library_inplace"><span class="icon"></span>http://git.sagemath.org/sage.git/commit/?h=u/nthiery/compile_library_inplace</a>
</p>
<p>
I can't wait until I can use this for real Sage development. But for
this, I basically need to get this feature into Sage. It would be
adventurous to enable this feature by default for all users at this
point. On the other hand, we should allow the adventurous of us to use
this extensively to chase out the bugs.
</p>
<p>
What would be the right approach to make this feature configurable
when one compiles Sage for the first time? Using just an environment
variable set by the user is not good, since a given user might have
Sage installations configured differently. Some piece of information
must be stored somewhere at configuration time.
</p>
<p>
I am happy to use a different ticket for this if this is deemed
preferable.
</p>
<p>
Cheers,
</p>
<blockquote>
<p>
Nicolas
</p>
</blockquote>
Ticketvbraun_spamSun, 10 Aug 2014 16:51:03 GMTmilestone changed
https://trac.sagemath.org/ticket/12659#comment:42
https://trac.sagemath.org/ticket/12659#comment:42
<ul>
<li><strong>milestone</strong>
changed from <em>sage-6.3</em> to <em>sage-6.4</em>
</li>
</ul>
TicketwasThu, 28 May 2015 17:43:49 GMT
https://trac.sagemath.org/ticket/12659#comment:43
https://trac.sagemath.org/ticket/12659#comment:43
<p>
This isn't *just* an issue of efficiency. Based on my experience, probably hundreds of potential sage developers have been confused by precisely the thing that just confused you. I remember Robert Miller running around at one Sage days going crazy because it seemed like everybody was massively confused by the fact there are multiple copies of arith.py (etc.). If I had any employees who could work on Sage, this would definitely be a good thing to put in a top 5 list of ticket priorities.
</p>
TicketkcrismanThu, 28 May 2015 17:47:39 GMT
https://trac.sagemath.org/ticket/12659#comment:44
https://trac.sagemath.org/ticket/12659#comment:44
<p>
+1
</p>
TicketjdemeyerThu, 28 May 2015 18:24:34 GMT
https://trac.sagemath.org/ticket/12659#comment:45
https://trac.sagemath.org/ticket/12659#comment:45
<p>
I wouldn't like the clutter of <code>.pyc</code> and <code>.so</code> files in the source directory. If the source directory would be kept as clean as it currently is, +1. Otherwise, -1.
</p>
TicketdimpaseMon, 01 Jun 2015 09:29:58 GMT
https://trac.sagemath.org/ticket/12659#comment:46
https://trac.sagemath.org/ticket/12659#comment:46
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12659#comment:45" title="Comment 45">jdemeyer</a>:
</p>
<blockquote class="citation">
<p>
I wouldn't like the clutter of <code>.pyc</code> and <code>.so</code> files in the source directory. If the source directory would be kept as clean as it currently is, +1. Otherwise, -1
</p>
</blockquote>
<p>
How about making links to .py files in SAGELOCAL rather than copying them into SAGELOCAL ?
</p>
<p>
Why was copying chosen in the 1st place, I wonder...
</p>
TicketjdemeyerMon, 01 Jun 2015 09:42:12 GMT
https://trac.sagemath.org/ticket/12659#comment:47
https://trac.sagemath.org/ticket/12659#comment:47
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12659#comment:46" title="Comment 46">dimpase</a>:
</p>
<blockquote class="citation">
<p>
Why was copying chosen in the 1st place, I wonder...
</p>
</blockquote>
<p>
Portability?
</p>
TicketwasMon, 01 Jun 2015 09:48:35 GMT
https://trac.sagemath.org/ticket/12659#comment:48
https://trac.sagemath.org/ticket/12659#comment:48
<p>
Copying wasn't chosen by us but by Python - it's just how setup.py install works.
</p>
<p>
The so and pyc files would definitely be in tree for this ticket and I believe this change would avoid tons of confusion.
Being in tree is the entire point of being able to work on and run the source directly rather than copying
It to a target location before build. Maybe jereon not wanting to run the source files directly is why this
Ticket never got in? I don't know.
</p>
<ul><li>sent from a phone
</li></ul>
TicketjdemeyerMon, 01 Jun 2015 09:54:53 GMT
https://trac.sagemath.org/ticket/12659#comment:49
https://trac.sagemath.org/ticket/12659#comment:49
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12659#comment:48" title="Comment 48">was</a>:
</p>
<blockquote class="citation">
<p>
The so and pyc files would definitely be in tree for this ticket and I believe this change would avoid tons of confusion.
</p>
</blockquote>
<p>
What kind of confusion are you talking about? Perhaps the confusion can be fixed in a different way?
</p>
<blockquote class="citation">
<p>
Maybe jereon not wanting to run the source files directly is why this
Ticket never got in? I don't know.
</p>
</blockquote>
<p>
No, it never got in Sage because it never actually worked.
</p>
TicketdimpaseMon, 01 Jun 2015 10:21:35 GMT
https://trac.sagemath.org/ticket/12659#comment:50
https://trac.sagemath.org/ticket/12659#comment:50
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12659#comment:48" title="Comment 48">was</a>:
</p>
<blockquote class="citation">
<p>
Copying wasn't chosen by us but by Python - it's just how setup.py install works.
</p>
</blockquote>
<p>
well, what does prevent us to add an extra step of replacing copies with links?
(This might not work well on native Windows, but we don't need to support this...)
</p>
<p>
This would combine the advantage of not having multiple copies of py files lying around (and not needing to <code>sage -b</code> every time you modify a .py file in Sage library), and
retaining .pyc and .so-free source directory.
</p>
<blockquote class="citation">
<p>
The so and pyc files would definitely be in tree for this ticket and I believe this change would avoid tons of confusion.
</p>
</blockquote>
TicketnthieryMon, 01 Jun 2015 16:04:05 GMT
https://trac.sagemath.org/ticket/12659#comment:51
https://trac.sagemath.org/ticket/12659#comment:51
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12659#comment:43" title="Comment 43">was</a>:
</p>
<blockquote class="citation">
<p>
If I had any employees who could work on Sage, this would definitely
be a good thing to put in a top 5 list of ticket priorities.
</p>
</blockquote>
<p>
We will definitely consider this in ODK. Actually the main reason we
did not put an explicit task about this is that I was (and still am)
hoping it would be finished before that. Writing the grant distracted
me away of working on it ...
</p>
<p>
Cheers,
</p>
<blockquote>
<p>
Nicolas
</p>
</blockquote>
TicketnthieryMon, 01 Jun 2015 16:05:45 GMT
https://trac.sagemath.org/ticket/12659#comment:52
https://trac.sagemath.org/ticket/12659#comment:52
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/12659#comment:49" title="Comment 49">jdemeyer</a>:
</p>
<blockquote class="citation">
<blockquote class="citation">
<p>
Maybe jereon not wanting to run the source files directly is why this
Ticket never got in? I don't know.
</p>
</blockquote>
<p>
No, it never got in Sage because it never actually worked.
</p>
</blockquote>
<p>
Well, see <a class="ticket" href="https://trac.sagemath.org/ticket/12659#comment:41" title="Comment 41">41</a>; it seemed pretty close from working. But of course the devil might be in the last bits to be polished.
</p>
TicketjdemeyerWed, 16 Sep 2015 08:20:24 GMT
https://trac.sagemath.org/ticket/12659#comment:53
https://trac.sagemath.org/ticket/12659#comment:53
<p>
I still don't understand what the problem is that this ticket is trying to fix.
</p>
Ticket