Opened 9 years ago

Closed 9 years ago

Last modified 9 years ago

#10187 closed defect (fixed)

Update ECL to 10.4.1 and Maxima to 5.22.1 - currently the latest releases.

Reported by: vbraun Owned by: drkirkby
Priority: major Milestone: sage-4.6.1
Component: packages: standard Keywords:
Cc: was, jhpalmieri, mpatel, leif, jsp, jason, kcrisman, fbissey, jpflori Merged in: sage-4.6.1.alpha3
Authors: Volker Braun, David Kirkby Reviewers: Karl-Dieter Crisman, David Kirkby, Volker Braun, Leif Leonhardy
Report Upstream: Workaround found; Bug reported upstream. Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by mvngu)

Note: See #10434 for a follow-up ticket.

Please update ECL and Maxima to the newest upstream release. Sage packages are here:

It's unsafe to build either ECL or Maxima in parallel, so we must not do this. Updated versions of ECL and Maxima can be found here:

http://boxen.math.washington.edu/home/kirkby/patches/ecl-10.4.1.spkg

http://www.stp.dias.ie/~vbraun/Sage/spkg/maxima-5.22.1.spkg

Note that you cannot upgrade one without the other; Both ECL and Maxima need to be upgraded simultaneously or build will fail.

Relevant tickets for ECL:

  • #9493: Remove extra baggage from ECL 10.2.1.p1 (again). The package has been cleaned up, but I feel removing gmp sources is too dangerous.
  • #10185: the old ecl-10.2.1 does not build on Fedora 14: Hopefully fixed with the updated ECL from this ticket.

Relevant tickets for maxima:

  • #8645: ECL library "maxima.fasb" is built at new location. The patches from this bug have been incorporated into the maxima spkg, and #8645 can be closed after this ticket.
  • #8731: update/upgrade Maxima to latest upstream (5.21.1): Latest upstream is nowadays 5.22.1.
  • #8582: sum(1/(1+k^2), k, -oo, oo) returns 0: The new Maxima seems to be able to evaluate this correctly, at leasts it returns some expression with digamma functions.

The updated Maxima code seems to be more careful about signs which leads to doctest errors. Moreover, error reporting was changed. Therefore you need to apply the following patches to the Sage library (in this order):

  • trac_10187_fix_easy_doctests.patch
  • trac_10187_general_display_prefix_workaround.patch
  • trac_10187_mark_some_doctests_random_until_9880_gets_merged.patch

Finally, you need to patch $SAGE_ROOT/local/bin/sage-maxima.lisp with

  • trac_10187_sage-maxima_lisp.patch

Attachments (10)

trac_10187_general_display_prefix_workaround.patch (4.2 KB) - added by vbraun 9 years ago.
Updated patch
trac_10187_sage-maxima_lisp.patch (642 bytes) - added by vbraun 9 years ago.
patch to SAGE_LOCAL/bin/sage_maxima.lisp with commit message
10187-remove-encodings-and-solaris-header.patch (1.9 KB) - added by drkirkby 9 years ago.
Remove the comment about not removing the 'endcodings' directory, as it should have been remove.d Removed attempt to patch on Solaris, as the patch did not work. This patch is ror review purposes only, as the changes are in ecl-10.4.1.spkg
10187-cleanup.patch (3.7 KB) - added by drkirkby 9 years ago.
clean up SPKG.txt and spkg-install on the ECL package
trac_10187_fix_easy_doctests.patch (16.5 KB) - added by vbraun 9 years ago.
Updated patch
trac_10187_mark_some_doctests_random_until_9880_gets_merged.patch (2.6 KB) - added by vbraun 9 years ago.
Updated patch
10187-further-cleanup.patch (5.5 KB) - added by drkirkby 9 years ago.
Further clean up SPKG.txt and spkg-install on the ECL package. Reverse some previous changes
10187-Document-quoting-problem.patch (1.7 KB) - added by drkirkby 9 years ago.
Document that quoting SAGE_LOCAL can cause problems. Correct a spelling error too.
maxima-5.22.1.patch (2.3 KB) - added by vbraun 9 years ago.
Re-uploaded diff for the maxima spkg (for review purposes only)
10187-remove-inaccure-comment-in-SPKG.txt.patch (877 bytes) - added by drkirkby 9 years ago.
Removes an inaccurate comment from SPKG.txt

Download all attachments as: .zip

Change History (170)

comment:1 Changed 9 years ago by kcrisman

  • Cc kcrisman added

comment:2 Changed 9 years ago by vbraun

The doctests show one maxima regression:

sage: t = log(sqrt(2) - 1) + log(sqrt(2) + 1); t
log(sqrt(2) - 1) + log(sqrt(2) + 1)
sage: u = t.maxima_methods(); u
MaximaWrapper(log(sqrt(2) - 1) + log(sqrt(2) + 1))
sage: u.logcontract()
log((sqrt(2) - 1)*(sqrt(2) + 1))

This simplified to 0 in a previous version. But at least its not wrong ;-)

The attached patch fixes all doctests except for a timeout in maxima. That one is a different issue with maxima, which forgets to print the general-display-prefix variable for errors:

(sage subshell) volker-desktop:sage vbraun$ ./local/bin/maxima -p ./local/bin/sage-maxima.lisp 
;;; Loading #P"/home/vbraun/Sage/sage/local/lib/ecl/sb-bsd-sockets.fas"
;;; Loading #P"/home/vbraun/Sage/sage/local/lib/ecl/sockets.fas"
;;; Loading #P"/home/vbraun/Sage/sage/local/lib/ecl/defsystem.fas"
;;; Loading #P"/home/vbraun/Sage/sage/local/lib/ecl/cmp.fas"
;;; Loading #P"/home/vbraun/Sage/sage/local/lib/ecl/sysfun.lsp"
Maxima 5.22.1 http://maxima.sourceforge.net
using Lisp ECL 10.4.1
Distributed under the GNU Public License. See the file COPYING.
Dedicated to the memory of William Schelter.
The function bug_report() provides bug reporting information.
(%i1) 1;
<sage-display>(%o1)                                  1
(%i2) 1==1;
incorrect syntax: = is not a prefix operator
1==
 ^
(%i2) 1=1;
<sage-display>(%o2)                                1 = 1

comment:3 follow-up: Changed 9 years ago by drkirkby

FYI, here's the failures on OpenSolaris 06/2009 with those updated versions of Maxima and ECL.

sage -t  -long -force_lib devel/sage/sage/interfaces/maxima.py
*** *** Error: TIMED OUT! PROCESS KILLED! *** ***

	 [1800.2 s]
 
----------------------------------------------------------------------

The following tests failed:

	sage -t  -long -force_lib devel/sage/sage/symbolic/integration/integral.py # 6 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/maxima_wrapper.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/expression.pyx # 11 doctests failed
	sage -t  -long -force_lib devel/sage/sage/misc/functional.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/wester.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/tests.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/calculus.py # 5 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/functional.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/plot/plot3d/transform.pyx # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/interfaces/maxima.py # Time out
----------------------------------------------------------------------
Total time for all tests: 2628.2 seconds
make: *** [ptestlong] Error 192
drkirkby@hawk:~/sage-4.6.1.alpha0$ 

I've not applied any patches at this point. I'll apply patches and rerun later.

comment:4 in reply to: ↑ 3 Changed 9 years ago by kcrisman

Replying to drkirkby:

FYI, here's the failures on OpenSolaris 06/2009 with those updated versions of Maxima and ECL.

sage -t  -long -force_lib devel/sage/sage/interfaces/maxima.py
*** *** Error: TIMED OUT! PROCESS KILLED! *** ***

	 [1800.2 s]
 

I'd be interested what happened if you manually removed the tests in that file which test the tab-completion for Maxima. I get a problem on Mac OS X Tiger with that file for exactly that reason. We haven't filed a ticket because nobody knows how to fix it, and tab-completion does work on that platform, just not the testing.

comment:5 follow-up: Changed 9 years ago by vbraun

The TIMED OUT! error is the missing general-display-prefix, which causes the expect interface to wait forever for maxima to print its next prompt.

I've filed a bug upstream here: https://sourceforge.net/tracker/?func=detail&aid=3098375&group_id=4933&atid=104933

comment:6 in reply to: ↑ 5 Changed 9 years ago by kcrisman

Replying to vbraun:

The TIMED OUT! error is the missing general-display-prefix, which causes the expect interface to wait forever for maxima to print its next prompt.

Okay, thanks - so that's unrelated to the Tiger timeout from earlier Maximas.

comment:7 follow-ups: Changed 9 years ago by vbraun

  • Report Upstream changed from N/A to Workaround found; Bug reported upstream.

I've modified sage/interfaces/maxima.py to work with maxima's prompt_prefix instead. Patch is attached and fixes the remaining doctest error.

This requires that `$SAGE_LOCAL/bin/sage-maxima.lisp sets

; (setf *general-display-prefix* "<sage-display>")
(setf *prompt-prefix* "<sage-display>")

I've made a updated sage_scripts spkg here: http://www.stp.dias.ie/~vbraun/Sage/spkg/sage_scripts-4.6.rc0.p0.spkg

To test this ticket, you need all three spkgs and both patches.

comment:8 Changed 9 years ago by vbraun

  • Status changed from new to needs_review

comment:9 Changed 9 years ago by fbissey

  • Cc fbissey added

comment:10 Changed 9 years ago by drkirkby

One thing we don't don, which I think would be sensible, is to include the test suite for ECL. That adds 1.7 MB. From the README.ECL file:

	cd tests
	make

You will find two types of output files:

	*.out	The whole output of a test
	*.erg	The errors arising from a test

When there is no *.erg, it means that ECL passed the test.

Would it not be sensible to add these in? Then we could execute the tests with a spkg-check file, like we do other code which has self-tests.

Dave

comment:11 in reply to: ↑ 7 Changed 9 years ago by kcrisman

Replying to vbraun:

I've modified sage/interfaces/maxima.py to work with maxima's prompt_prefix instead. Patch is attached and fixes the remaining doctest error.

Just a very minor note:

# we are now getting three lines of commented verbosity 

but of course you changed it to get rid of the extra two lines from our constantly added packages :) so it's now five lines. Might as well change that too so future people are confused.

comment:12 Changed 9 years ago by kcrisman

Also, be sure to include a test that shows #8729 is fixed, as that was closed as a duplicate on the condition that #8731's doctests would show that.

comment:13 Changed 9 years ago by aginiewicz

I looked trough doctest patch and compared it with mine from #8731, I have few comments.

I think that doctest fix for sage/calculus/functional.py should be modified - it was documentation of the use of assume, but now - when maxima can calculate this integral without assumption, isn't that same as applying assumption to result? And the integral is checked in other doctest now. Maybe there is better example that still needs the assume to get result?

Also the doctest in sage/misc/functional.py was here to demonstrate numerical approximation of integral that cannot be evaluated iirc, and now when it can be evaluated, I think it should be changed to something that cannot be - like Jason did in patch in #8731.

Finally, there is typo in doctest to sage/symbolic/expression.pyx - it should say that this doctest is here to check that #7334 not #7344 is fixed (#7344 point to libjpeg issue so it cannot be it) - and it should stay in here unmodified. Also - the log thing - it's not exactly regression, it's feature. This is because of Maxima ticket 947808 - http://sourceforge.net/tracker/?func=detail&aid=947808&group_id=4933&atid=104933 - they now try to keep the expression as factored as they can without using factor. This results in observed behaviour. This doctest I think cannot be changed as it demonstrated that ticket is solved, but we should apply "x = x.simplify_rational()" after "x = x.simplify_log('one')" in definition of simplify_full - that way full simplify would result in same results as before I think, and #7334 would be still fixed. Now, this doctest shows nothing.

comment:14 Changed 9 years ago by drkirkby

  • Status changed from needs_review to needs_work

There is a problem with the ECL package here. The history of all recent changes has been lost. The recent changes to the package have been:

=== ecl-10.2.1.p3 (David Kirkby, David Kirkby, 17th September 2010) ===
=== ecl-10.2.1.p2 (David Kirkby, 30th July 2010) ===
=== ecl-10.2.1.p1 (Mitesh Patel, 11th July 2010) ===
=== ecl-10.2.1.p0 (David Kirkby, 11th July 2010) ===
=== ecl-10.2.1 (William Stein, 14 February 2010) ===

but instead SPKG.txt shows

=== ecl-10.4.1.p0 (Leif Leonhardy, Volker Braun, 29th September 2010) ===
=== ecl-10.4.1 (N. Bruin, W. Stein, D. Kirkby and M. Patel 19th June 2010) ===
=== ecl-10.2.1 (William Stein, 14 February 2010) ===

This error has occurred since the package is based on one that I created several months ago, which never got merged, because their were conflicts with that and the Maxima package at the time.

The current version of ECL in Sage should have been used as a starting point - not an old one that was never merged.

I'll sort the above problem out, and add the ECL test code today.

Dave

comment:15 Changed 9 years ago by fbissey

The patch fix_easy_doctest doesn't apply cleanly on either 4.6.rc0 or 4.6.1.alpha0. Was it prepared against 4.5.3?

comment:16 in reply to: ↑ 7 ; follow-up: Changed 9 years ago by leif

Replying to vbraun:

I've modified sage/interfaces/maxima.py to work with maxima's prompt_prefix instead. Patch is attached and fixes the remaining doctest error.

This requires that `$SAGE_LOCAL/bin/sage-maxima.lisp sets

; (setf *general-display-prefix* "<sage-display>")
(setf *prompt-prefix* "<sage-display>")

I've made a updated sage_scripts spkg here: http://www.stp.dias.ie/~vbraun/Sage/spkg/sage_scripts-4.6.rc0.p0.spkg

To test this ticket, you need all three spkgs and both patches.

Could you please attach a patch to the scripts repo, too (rather than a link to a complete new scripts spkg)?

Also, attaching diffs (or Mercurial patches) of the spkgs makes reviewing easier.

comment:17 in reply to: ↑ 16 Changed 9 years ago by leif

Replying to leif:

Could you please attach a patch to the scripts repo, too (rather than a link to a complete new scripts spkg)?

P.S.: Of course I could do that, too, but then you wouldn't be able to update it in case you later modify the patch.

Changed 9 years ago by vbraun

Updated patch

comment:18 Changed 9 years ago by vbraun

I've incorporated your suggestions and updated the patches. Both are (and were) against 4.6.rc0 and apply cleanly. Maybe you had the wrong order? It should be

  • trac_10187_fix_easy_doctests.patch
  • trac_10187_general_display_prefix_workaround.patch

I'll attach patches for the sage_scripts and maxima spgk for easier review. Once Dave is finished with the ecl package then this ticket is ready for review again.

comment:19 Changed 9 years ago by fbissey

Wrong order indeed! I thought the two patches were orthogonal.

comment:20 follow-up: Changed 9 years ago by drkirkby

I'm going to have to ask on the ECL list how to run the test suite - I can't work out where ones supposed to copy the source.

These changes are not going to make it into 4.6. The milestone is 4.6.1, and it will easily be resolved by then.

Dave

comment:21 in reply to: ↑ 20 Changed 9 years ago by drkirkby

Replying to drkirkby:

I'm going to have to ask on the ECL list how to run the test suite - I can't work out where ones supposed to copy the source.

These changes are not going to make it into 4.6. The milestone is 4.6.1, and it will easily be resolved by then.

Dave

I've now got the information on the ANSI test suite from the ECL developer, though I gather the copy on the ECL site is rather out of date. Also, the ECL developer has fixed the bug that causes #9840, so I'll patch that too, so #9840 can hopefully be closed at the same time.

I'll try to get this sorted out within the next few days.

Dave

comment:22 Changed 9 years ago by drkirkby

  • Authors changed from Volker Braun to Volker Braun, David Kirkby
  • Description modified (diff)

I have put an updated ECL file at http://boxen.math.washington.edu/home/kirkby/patches/ecl-10.4.1.spkg This has only been checked on OpenSolaris - I don't have access to a Fedora 14 machine, so can't verify if it actually fixes the issues reported at #10185

I should make a few comments about this:

  • I have not added an spkg-check file or the Lisp tests, as I gather from the ECL developer the Lisp tests on the ECL site are outdated. Fixing this appears to be a non-trivial issue.
  • The repository information from the ecl-10.2.1.p3 version actually in Sage is kept, as it should be. (Volker's package was based on a 10.4.1 package I created months ago, which never got merged into Sage. So the repository information was not correct).
  • Despite being told the Solaris text relocation issue was resolved, it appears it is not as simple as applying a single patch as I had hoped. So #9840 remains unresolved, though it should be fixed when the next stable ECL release is made.
  • I've cleaned the package up somewhat.
  • I did not remove the gmp sources, as doing so requires a new configure file to be created. Whilst I can see this is advantageous if done properly, I fear that this will be done incorrectly at some point in the future, which can result in chaos.

After

I get one reject out of six when adding

trac_10187_general_display_prefix_workaround.patch

to sage 4.6.1.alpha0, so I think that patch needs updating. The contents of the reject are:

--- maxima.py
+++ maxima.py
@@ -728,10 +729,12 @@
         
             sage: maxima._eval_line('1+1;')
             '2'
-            sage: maxima.eval('sage0: x == x;')
+            sage: maxima._eval_line('sage0: x == x;')
             Traceback (most recent call last):
             ...
-            TypeError: error evaluating "sage0: x == x;":...
+            TypeError: Error executing code in Maxima...
+
+
         """
         if len(line) == 0:
             return ''

But as I say, all doctests passed for me, despite one patch was not fully applied!!

----------------------------------------------------------------------
All tests passed!
Total time for all tests: 1871.3 seconds

I'm leaving as needs work, as clearly the fact one patch does not apply cleanly is a problem. It also needs testing on more than one system, but I don't have access to the Fedora 14 system where this was a particular problem.

Dave

comment:23 Changed 9 years ago by vbraun

Dave's ecl-10.4.1.spkg compiles just fine on Fedora 14! I don't have sage 4.6.1.alpha0 installed right now, so it'll take me a few hours to update the patch.

comment:24 follow-ups: Changed 9 years ago by vbraun

  • Status changed from needs_work to needs_review

Patches apply cleanly on sage 4.6.1.alpha0. As I wrote in http://trac.sagemath.org/sage_trac/ticket/10187#comment:18, they need to be applied in the order

  • trac_10187_fix_easy_doctests.patch
  • trac_10187_general_display_prefix_workaround.patch

comment:25 in reply to: ↑ 24 Changed 9 years ago by drkirkby

Replying to vbraun:

Patches apply cleanly on sage 4.6.1.alpha0. As I wrote in http://trac.sagemath.org/sage_trac/ticket/10187#comment:18, they need to be applied in the order

  • trac_10187_fix_easy_doctests.patch
  • trac_10187_general_display_prefix_workaround.patch

Perhaps I did not have a clean sage 4.6.1.alpha0. I will start a fresh build and try again.

Dave

comment:26 in reply to: ↑ 24 Changed 9 years ago by drkirkby

Replying to vbraun:

Patches apply cleanly on sage 4.6.1.alpha0. As I wrote in http://trac.sagemath.org/sage_trac/ticket/10187#comment:18, they need to be applied in the order

  • trac_10187_fix_easy_doctests.patch
  • trac_10187_general_display_prefix_workaround.patch

Ah, I missed that fact. I applied the patches in the order they were applied to the trac ticket, which is the reverse order to what you are saying should have been done!

I've just started a build. It takes a bit over an hour to build Sage and run all the long doctests on this machine, but it's 11 pm local time. I'll try to get it done before going to bed, so I can report on it, but it might have to wait until tomorrow. I'm feeling rather tired here.

Dave

comment:27 follow-up: Changed 9 years ago by drkirkby

OK, this is fine on OpenSolaris 06/2009 and Fedora 14. What about any other platforms?

comment:28 in reply to: ↑ 27 ; follow-up: Changed 9 years ago by fbissey

Replying to drkirkby:

OK, this is fine on OpenSolaris 06/2009 and Fedora 14. What about any other platforms?

I have switched sage-on-gentoo to maxima-5.22.1 on sage-4.6's release I haven't run a long test of alpha1 with it but I haven't heard complaint from anyone running tests on x86 and amd64 with 4.6.

comment:29 in reply to: ↑ 28 Changed 9 years ago by drkirkby

Replying to fbissey:

I have switched sage-on-gentoo to maxima-5.22.1 on sage-4.6's release I haven't run a long test of alpha1 with it but I haven't heard complaint from anyone running tests on x86 and amd64 with 4.6.

Thank you. I must do some other things today, so don't have time to test this much. But hopefully we can get some testers on other platforms. To my knowledge we know nothing about

  • Any OS X release
  • Solaris 10 on SPARC
  • Solaris 10 on x86

all of which are supported platforms.

I've asked on sage-devel, if someone else can do some testing. It would be nice to get this merged. Experience shows if we leave updating Maxima too long, it becomes a nightmare to update. It really is something we should regularly update - not only for new functionality, but also it saves a nightmare at a later date.

Dave

comment:30 Changed 9 years ago by jpflori

  • Cc jpflori added

comment:31 Changed 9 years ago by kcrisman

I'm currently testing on OS X 10.4 (Tiger) PPC. ECL installed fine, but I won't be able to see the results of the rest and some testing until tomorrow.

comment:32 Changed 9 years ago by jpflori

I tested both spkg on Debian, everything compiled fine, the patches applied cleanly on 4.6.1.alpha2 and sage_scripts spkg from that version (when applied in order of comment 18, otherwise 1 hunk does not get applied) and all tests passed.

sage -t -long -force_lib "devel/sage/sage/interfaces/maxima.py"
         [22.4 s]
 
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 22.5 seconds

It should be noted that the Maxima update fixes #7952 (and #10273 I opened by mistake and closed as duplicate).

comment:33 Changed 9 years ago by kcrisman

  • Reviewers set to Karl-Dieter Crisman
  • Status changed from needs_review to needs_work
  • Work issues set to new interface patch needs help

I get a LOT of timeouts on PPC OS X 10.4. I think that basically any time Maxima is invoked inside Sage, it times out or does something else weird. E.g.

sage: integrate(x^2,x)
Exception RuntimeError: 'maximum recursion depth exceeded in __subclasscheck__' in <type 'exceptions.RuntimeError'> ignored
Exception RuntimeError: RuntimeError('maximum recursion depth exceeded in cmp',) in Exception RuntimeError: 'maximum recursion depth exceeded in __subclasscheck__' in <type 'exceptions.RuntimeError'> ignored
 ignored
Exception RuntimeError: 'maximum recursion depth exceeded in __subclasscheck__' in <type 'exceptions.RuntimeError'> ignored
Exception RuntimeError: RuntimeError('maximum recursion depth exceeded in cmp',) in Exception RuntimeError: 'maximum recursion depth exceeded in __subclasscheck__' in <type 'exceptions.RuntimeError'> ignored
 ignored
Exception RuntimeError: 'maximum recursion depth exceeded in __subclasscheck__' in <type 'exceptions.RuntimeError'> ignored
Exception RuntimeError: RuntimeError('maximum recursion depth exceeded in cmp',) in Exception RuntimeError: 'maximum recursion depth exceeded in __subclasscheck__' in <type 'exceptions.RuntimeError'> ignored
 ignored
Exception RuntimeError: 'maximum recursion depth exceeded in __subclasscheck__' in <type 'exceptions.RuntimeError'> ignored
Exception RuntimeError: RuntimeError('maximum recursion depth exceeded in cmp',) in Exception RuntimeError: 'maximum recursion depth exceeded in __subclasscheck__' in <type 'exceptions.RuntimeError'> ignored
 ignored

I also get the following:

sage -t -long "devel/sage/sage/calculus/functions.py"       
**********************************************************************
File "/Users/student/Desktop/sage-4.6/devel/sage/sage/calculus/functions.py", line 46:
    sage: wronskian(*[x^k for k in range(1, 5)])
        assert len(self._before())==0, 'Maxima expect interface is confused!'
    AssertionError: Maxima expect interface is confused!

What the heck? I assume the change to the Maxima interface is to blame; reverting that patch only (not the doctest one) seems to fix the problem.

Though in that case I do get

File "/Users/student/Desktop/sage-4.6/devel/sage/sage/calculus/calculus.py", line 343:
    sage: taylor(gamma(1/3+x),x,0,3)
Expected:
    -1/432*((36*(pi*sqrt(3) + 9*log(3))*euler_gamma^2 + 27*pi^2*log(3) + 72*euler_gamma^3 + 243*log(3)^3 + 18*(6*pi*sqrt(3)*log(3) + pi^2 + 27*log(3)^2 + 12*psi(1, 1/3))*euler_gamma + 324*psi(1, 1/3)*log(3) + (pi^3 + 9*(9*log(3)^2 + 4*psi(1, 1/3))*pi)*sqrt(3))*gamma(1/3) - 72*gamma(1/3)*psi(2, 1/3))*x^3 + 1/24*(6*pi*sqrt(3)*log(3) + 4*(pi*sqrt(3) + 9*log(3))*euler_gamma + pi^2 + 12*euler_gamma^2 + 27*log(3)^2 + 12*psi(1, 1/3))*x^2*gamma(1/3) - 1/6*(6*euler_gamma + pi*sqrt(3) + 9*log(3))*x*gamma(1/3) + gamma(1/3)
Got:
    -1/432*((36*(pi*sqrt(3) + 9*log(3))*euler_gamma^2 + 27*pi^2*log(3) + 72*euler_gamma^3 + 243*log(3)^3 + 18*(6*pi*sqrt(3)*log(3) + pi^2 + 27*log(3)^2 + 12*psi(1, 1/3))*euler_gamma + 324*log(3)*psi(1, 1/3) + (pi^3 + 9*(9*log(3)^2 + 4*psi(1, 1/3))*pi)*sqrt(3))*gamma(1/3) - 72*gamma(1/3)*psi(2, 1/3))*x^3 + 1/24*(6*pi*sqrt(3)*log(3) + 4*(pi*sqrt(3) + 9*log(3))*euler_gamma + pi^2 + 12*euler_gamma^2 + 27*log(3)^2 + 12*psi(1, 1/3))*x^2*gamma(1/3) - 1/6*(6*euler_gamma + pi*sqrt(3) + 9*log(3))*x*gamma(1/3) + gamma(1/3)
**********************************************************************

which is interesting; if you find where they differ (hint - transposed multiplication), you win the Where's Waldo award.

I'll report back if there are any other errors without it.

comment:34 Changed 9 years ago by vbraun

kcrisman: looks like you did not install the updated sage_scripts-4.6.rc0.p0.spkg, see http://trac.sagemath.org/sage_trac/ticket/10187#comment:7

comment:35 Changed 9 years ago by jdemeyer

If sage_scripts needs to be patched, please provide a hg patch for that.

comment:36 Changed 9 years ago by jpflori

It is sage-maxima.lisp.patch provided in the ticked.

It applies cleanly on sage_script spkg of sage 4.6.1.alpha2 for me and tests pass.

comment:37 Changed 9 years ago by kcrisman

Replying to jdemeyer:

If sage_scripts needs to be patched, please provide a hg patch for that.

Looks like it needs a commit message to apply straight-up with hg_scripts, so it's not ready as it stands, perhaps.

Without *either* of those patches, I get most relevant tests passing EXCEPT the taylor(gamma(1/3+x),x,0,3) one, on both OS X 10.6 Intel and 10.4 PPC. The interfaces/maxima.py times out without both the workaround and lisp patch on 10.6, after which it's fine on that and sage/calculus, sage/symbolic, and sage/functions.

I haven't tested that last combination of patches yet on my 10.4 machine, as it's very slow - later today I might know more.

Changed 9 years ago by vbraun

patch to SAGE_LOCAL/bin/sage_maxima.lisp with commit message

comment:38 Changed 9 years ago by vbraun

  • Status changed from needs_work to needs_review
  • Work issues new interface patch needs help deleted

I've added a patch for sage-maxima.lisp as trac_10187_sage-maxima_lisp.patch.

To summarize, you need to

  1. Apply trac_10187_sage-maxima_lisp.patch to SAGE_LOCAL/bin
  2. http://boxen.math.washington.edu/home/kirkby/patches/ecl-10.4.1.spkg
  3. http://www.stp.dias.ie/~vbraun/Sage/spkg/maxima-5.22.1.spkg
  4. Apply trac_10187_fix_easy_doctests.patch to the sage library
  5. Apply trac_10187_general_display_prefix_workaround.patch to the sage library

Its true, but completely pointless, that some doctests pass without some of these patches. Maxima's error reporting changed and we cannot use the old parser any more. Read this ticket for details.

comment:39 follow-up: Changed 9 years ago by kcrisman

Okay, after running all (not long) doctests, I get the following error in addition to the taylor error above, on Mac OS X 10.6, applied to a brand-new Sage 4.6.1.alpha1 build:

sage -t  devel/sage/sage/plot/plot3d/transform.pyx
**********************************************************************
File "/Users/.../sage-4.6.1.alpha1/devel/sage-main/sage/plot/plot3d/transform.pyx", line 217:
    sage: m
Expected:
    [                                       -(cos(theta) - 1)*x^2 + cos(theta)              -(cos(theta) - 1)*sqrt(-x^2 - z^2 + 1)*x + sin(theta)*abs(z)      -((cos(theta) - 1)*x*z^2 + sqrt(-x^2 - z^2 + 1)*sin(theta)*abs(z))/z]
    [             -(cos(theta) - 1)*sqrt(-x^2 - z^2 + 1)*x - sin(theta)*abs(z)                           (cos(theta) - 1)*x^2 + (cos(theta) - 1)*z^2 + 1 -((cos(theta) - 1)*sqrt(-x^2 - z^2 + 1)*z*abs(z) - x*z*sin(theta))/abs(z)]
    [     -((cos(theta) - 1)*x*z^2 - sqrt(-x^2 - z^2 + 1)*sin(theta)*abs(z))/z -((cos(theta) - 1)*sqrt(-x^2 - z^2 + 1)*z*abs(z) + x*z*sin(theta))/abs(z)                                        -(cos(theta) - 1)*z^2 + cos(theta)]
Got:
    [                                       -(cos(theta) - 1)*x^2 + cos(theta)              -(cos(theta) - 1)*sqrt(-x^2 - z^2 + 1)*x + abs(z)*sin(theta)      -((cos(theta) - 1)*x*z^2 + sqrt(-x^2 - z^2 + 1)*abs(z)*sin(theta))/z]
    [             -(cos(theta) - 1)*sqrt(-x^2 - z^2 + 1)*x - abs(z)*sin(theta)                           (cos(theta) - 1)*x^2 + (cos(theta) - 1)*z^2 + 1 -((cos(theta) - 1)*sqrt(-x^2 - z^2 + 1)*z*abs(z) - x*z*sin(theta))/abs(z)]
    [     -((cos(theta) - 1)*x*z^2 - sqrt(-x^2 - z^2 + 1)*abs(z)*sin(theta))/z -((cos(theta) - 1)*sqrt(-x^2 - z^2 + 1)*z*abs(z) + x*z*sin(theta))/abs(z)                                        -(cos(theta) - 1)*z^2 + cos(theta)]
**********************************************************************

There are several transpositions of abs(z)*sin(theta) in this one, just like the transposition of psi(1, 1/3)*log(3) in the other error.

I get

sage: sage.calculus.calculus.maxima.eval('taylor(gamma(1/3+x),x,0,3)')
'gamma(1/3)-(6*%gamma+%pi*sqrt(3)+9*log(3))*gamma(1/3)*x/6+(12*%gamma^2+(4*%pi*sqrt(3)+36*log(3))*%gamma+6*log(3)*%pi*sqrt(3)+%pi^2+27*log(3)^2+12*psi[1](1/3))*gamma(1/3)*x^2/24+(72*gamma(1/3)*psi[2](1/3)+(-72*%gamma^3+(-36*%pi*sqrt(3)-324*log(3))*%gamma^2+(-108*log(3)*%pi*sqrt(3)-18*%pi^2-486*log(3)^2-216*psi[1](1/3))*%gamma+(-%pi^3+(-81*log(3)^2-36*psi[1](1/3))*%pi)*sqrt(3)-27*log(3)*%pi^2-243*log(3)^3-324*psi[1](1/3)*log(3))*gamma(1/3))*x^3/432'

which looks like the 'expected' line. I also get

sage: psi(1,1/3)*log(3)
log(3)*psi(1, 1/3)

which perhaps is the issue. What do people who do not have problems with this doctest get for this? (If the same for Maxima, but different for Sage, maybe it's a platform-dependent Pynac issue ... why on earth would that happen?)

comment:40 follow-up: Changed 9 years ago by vbraun

  • Status changed from needs_review to needs_info

Can you double-check that the maxima binary works? you should get:

[vbraun@volker-two ~]$ sage -sh

Starting subshell with Sage environment variables set.
Be sure to exit when you are done and do not do anything
with other copies of Sage!

Bypassing shell configuration files ...

SAGE_ROOT=/home/vbraun/Sage/sage
(sage subshell) volker-two:~ vbraun$ maxima
;;; Loading #P"/home/vbraun/Sage/sage/local/lib/ecl/sb-bsd-sockets.fas"
;;; Loading #P"/home/vbraun/Sage/sage/local/lib/ecl/sockets.fas"
;;; Loading #P"/home/vbraun/Sage/sage/local/lib/ecl/defsystem.fas"
;;; Loading #P"/home/vbraun/Sage/sage/local/lib/ecl/cmp.fas"
;;; Loading #P"/home/vbraun/Sage/sage/local/lib/ecl/sysfun.lsp"
Maxima 5.22.1 http://maxima.sourceforge.net
using Lisp ECL 10.4.1
Distributed under the GNU Public License. See the file COPYING.
Dedicated to the memory of William Schelter.
The function bug_report() provides bug reporting information.
(%i1) display2d : false;

(%o1) false
(%i2) psi(1,1/3)*log(3);

(%o2) psi(1,1/3)*log(3)

comment:41 in reply to: ↑ 40 ; follow-up: Changed 9 years ago by kcrisman

  • Status changed from needs_info to needs_review

RReplying to vbraun:

Can you double-check that the maxima binary works? you should get:

I'm not sure how it could have passed tests without working :) but yes, it works on OS X 10.6. From command line, from sage -sh, from maxima_console, etc.

On 10.4 it also works once I add in the script change, and I'm currently running doctests. It is even slower than before, but so far it does seem to be working. Just FYI that on that platform testing sage/interfaces/maxima.py has no point, because of the (unrelated, I think) #9361.

Putting to needs work because of this weird multiplication issue.

comment:42 in reply to: ↑ 41 ; follow-up: Changed 9 years ago by vbraun

yes, it works on OS X 10.6.

Does maxima return psi(1,1/3)*log(3) in this order or in the reversed order?

comment:43 in reply to: ↑ 42 Changed 9 years ago by kcrisman

  • Status changed from needs_review to needs_work

Replying to vbraun:

yes, it works on OS X 10.6.

Does maxima return psi(1,1/3)*log(3) in this order or in the reversed order?

I already reverted ... but as far as I recall, it was Sage itself that reversed this, as in comment 39, and in straight Maxima I got what you did.

Sorry, put to wrong thing before - needs work because of this, or needs info if you like.

comment:44 follow-up: Changed 9 years ago by vbraun

  • Status changed from needs_work to needs_info

I'm still confused what, if any, issue remains. This patch only touches the capture of the whole maxima output string, and not the parsing of expression. If maxima's output ordering has not changed yet some platform returns a different output sort order then this is an unrelated bug in the maxima interface.

comment:45 in reply to: ↑ 39 Changed 9 years ago by ddrake

On Ubuntu 10.04 64-bit with 4.6.1.alpha1, everything is working -- all "ptestlong" tests pass.

Replying to kcrisman:

I get

sage: sage.calculus.calculus.maxima.eval('taylor(gamma(1/3+x),x,0,3)')
'gamma(1/3)-(6*%gamma+%pi*sqrt(3)+9*log(3))*gamma(1/3)*x/6+(12*%gamma^2+(4*%pi*sqrt(3)+36*log(3))*%gamma+6*log(3)*%pi*sqrt(3)+%pi^2+27*log(3)^2+12*psi[1](1/3))*gamma(1/3)*x^2/24+(72*gamma(1/3)*psi[2](1/3)+(-72*%gamma^3+(-36*%pi*sqrt(3)-324*log(3))*%gamma^2+(-108*log(3)*%pi*sqrt(3)-18*%pi^2-486*log(3)^2-216*psi[1](1/3))*%gamma+(-%pi^3+(-81*log(3)^2-36*psi[1](1/3))*%pi)*sqrt(3)-27*log(3)*%pi^2-243*log(3)^3-324*psi[1](1/3)*log(3))*gamma(1/3))*x^3/432'

which looks like the 'expected' line. I also get

sage: psi(1,1/3)*log(3)
log(3)*psi(1, 1/3)

which perhaps is the issue. What do people who do not have problems with this doctest get for this? (If the same for Maxima, but different for Sage, maybe it's a platform-dependent Pynac issue ... why on earth would that happen?)

Here's what I get for those commands with all the patches and spkgs here, in 4.6.1.alpha1:

sage: sage.calculus.calculus.maxima.eval('taylor(gamma(1/3+x),x,0,3)')
'gamma(1/3)-(6*%gamma+%pi*sqrt(3)+9*log(3))*gamma(1/3)*x/6+(12*%gamma^2+(4*%pi*sqrt(3)+36*log(3))*%gamma+6*log(3)*%pi*sqrt(3)+%pi^2+27*log(3)^2+12*psi[1](1/3))*gamma(1/3)*x^2/24+(72*gamma(1/3)*psi[2](1/3)+(-72*%gamma^3+(-36*%pi*sqrt(3)-324*log(3))*%gamma^2+(-108*log(3)*%pi*sqrt(3)-18*%pi^2-486*log(3)^2-216*psi[1](1/3))*%gamma+(-%pi^3+(-81*log(3)^2-36*psi[1](1/3))*%pi)*sqrt(3)-27*log(3)*%pi^2-243*log(3)^3-324*psi[1](1/3)*log(3))*gamma(1/3))*x^3/432'
sage: psi(1,1/3)*log(3)
psi(1, 1/3)*log(3)

comment:46 in reply to: ↑ 44 ; follow-up: Changed 9 years ago by kcrisman

Dan, thanks for that confirmation of the issue.

Replying to vbraun:

I'm still confused what, if any, issue remains. This patch only touches the capture of the whole maxima output string, and not the parsing of expression. If maxima's output ordering has not changed yet some platform returns a different output sort order then this is an unrelated bug in the maxima interface.

Actually, I'm pretty sure it's an unrelated bug in something else. At first I thought Pynac/Ginac?, but maybe not (?) - in Sage 4.6 on PPC, I get

sage: psi(1,1/3)*log(3)
log(3)*psi(1, 1/3)
sage: from sage.misc.citation import get_systems
sage: get_systems('psi(1,1/3)*log(3)')
['GMP']

This machine doesn't have the new Maxima, so it's definitely unrelated.

Nonetheless, we can't ship a Sage with a failing doctest that we know about, except at great need - this is why I put needs work. The very minor change in how Maxima grouped the coefficient of x^3 created this.

So I recommend that we replace this doctest with something very similar which doesn't have this problem, and then open a new ticket for this issue. Maybe something like

sage: taylor(gamma(1/4+x),x,0,3)

except I can't tell you what the Mac output is on this until tomorrow - maybe someone else can.

The new ticket is #10282.

comment:47 follow-up: Changed 9 years ago by jpflori

I installed Sage on a Mac:

Machine Name: Power Mac G5

Machine Model: !PowerMac11,2

CPU Type: PowerPC G5 (1.1)

with Mac OS X:

System Version: Mac OS X 10.4.11 (8S165)

Kernel Version: Darwin 8.11.0

from binary packages:

http://www.sagemath.org/download-mac.html

and using cp -R - A ... to a custom directory because I don't have admin rights.

So I got Sage 4.6.

I ran Sage once.

I then patched sage_scripts (downloaded spkg, uncompressed, hg qimport patch, hg qpush) and installed it.

I installed ecl-10.4 and maxima5.22.1 spkgs.

I ran ./sage -b but got problem of paths, it was looking in /Users/Shared?/sage/sage-4.6/ ...

So I moved my installation folder there and ./sage -b worked.

I then ran ./sage -t  -long -force_lib devel/sage/sage/interfaces/maxima.py

and everything was fine:

init.sage does not exist ... creating
sage -t -long -force_lib "devel/sage/sage/interfaces/maxima.py"
         [53.8 s]
 
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 53.8 seconds

I'll try to test on a SPARC machine with Solaris 10 soon.

comment:48 Changed 9 years ago by jpflori

I'm currently running all long doctest on the Mac machine I have access to.

The Mac output for taylor(gamma(1/4+x),x,0,3) is:

sage: taylor(gamma(1/4+x),x,0,3)
-1/48*((12*(pi + 6*log(2))*euler_gamma^2 + pi^3 + 18*pi^2*log(2) + 8*euler_gamma^3 + 216*log(2)^3 + 12*(9*log(2)^2 + psi(1, 1/4))*pi + 6*(pi^2 + 12*pi*log(2) + 36*log(2)^2 + 4*psi(1, 1/4))*euler_gamma + 72*log(2)*psi(1, 1/4))*gamma(1/4) - 8*gamma(1/4)*psi(2, 1/4))*x^3 + 1/8*(4*(pi + 6*log(2))*euler_gamma + pi^2 + 12*pi*log(2) + 4*euler_gamma^2 + 36*log(2)^2 + 4*psi(1, 1/4))*x^2*gamma(1/4) - 1/2*(pi + 2*euler_gamma + 6*log(2))*x*gamma(1/4) + gamma(1/4)

On Linux with latest alpha I get :

sage: taylor(gamma(1/4+x),x,0,3)
-1/48*((12*(pi + 6*log(2))*euler_gamma^2 + pi^3 + 18*pi^2*log(2) + 8*euler_gamma^3 + 216*log(2)^3 + 12*(9*log(2)^2 + psi(1, 1/4))*pi + 6*(pi^2 + 12*pi*log(2) + 36*log(2)^2 + 4*psi(1, 1/4))*euler_gamma + 72*psi(1, 1/4)*log(2))*gamma(1/4) - 8*gamma(1/4)*psi(2, 1/4))*x^3 + 1/8*(4*(pi + 6*log(2))*euler_gamma + pi^2 + 12*pi*log(2) + 4*euler_gamma^2 + 36*log(2)^2 + 4*psi(1, 1/4))*x^2*gamma(1/4) - 1/2*(pi + 2*euler_gamma + 6*log(2))*x*gamma(1/4) + gamma(1/4)

which is not the same.Look at 72*...

comment:49 in reply to: ↑ 47 ; follow-up: Changed 9 years ago by kcrisman

Machine Name: Power Mac G5

System Version: Mac OS X 10.4.11 (8S165)

I then ran ./sage -t  -long -force_lib devel/sage/sage/interfaces/maxima.py

Wow, you got no timeout with that file? Georg W. also said that he got this to pass... maybe it's the force lib option, I still get (unrelated to this ticket) timeouts...

Unfortunately, I did not have any luck with relieving the timeouts on my machine in any file which actually uses Maxima. With SAGE_TIMEOUT_LONG=5000 I still get them on all those files. The only difference is I have a G4, not a G5. Maxima *does* work - I tried it in various ways - but it's amazingly slow to start up.

I'll try to look at this more later today.

comment:50 in reply to: ↑ 49 Changed 9 years ago by jpflori

Replying to kcrisman:

Wow, you got no timeout with that file? Georg W. also said that he got this to pass... maybe it's the force lib option, I still get (unrelated to this ticket) timeouts...

It also passes without the froce_lib option (in 52.1 seconds).

Here is the output of ./sage -t -long -force_lib devel/sage on my Debian with 4.6.1.alpha2, the timeouts are mentionned at http://groups.google.com/group/sage-devel/browse_thread/thread/09e4cef8c8558150# :

----------------------------------------------------------------------
The following tests failed:


        sage -t -long -force_lib "devel/sage/build/lib.linux-x86_64-2.6/sage/misc/sagedoc.py"
        sage -t -long -force_lib "devel/sage/build/lib.linux-x86_64-2.6/sage/homology/examples.py" # Time out
        sage -t -long -force_lib "devel/sage/build/sage/misc/sagedoc.py"
        sage -t -long -force_lib "devel/sage/build/sage/homology/examples.py" # Time out
        sage -t -long -force_lib "devel/sage/sage/misc/sagedoc.py"
        sage -t -long -force_lib "devel/sage/sage/homology/examples.py" # Time out
        sage -t -long -force_lib "devel/sage/doc/en/numerical_sage/cvxopt.rst"
Total time for all tests: 28820.2 seconds

And on the Mac with 4.6:

----------------------------------------------------------------------
The following tests failed:


        sage -t -long -force_lib "devel/sage/build/lib.macosx-10.4-ppc-2.6/sage/calculus/calculus.py"
        sage -t -long -force_lib "devel/sage/build/sage/calculus/calculus.py"
        sage -t -long -force_lib "devel/sage/build/sage/plot/plot3d/transform.pyx"
        sage -t -long -force_lib "devel/sage/sage/calculus/calculus.py"
        sage -t -long -force_lib "devel/sage/sage/plot/plot3d/transform.pyx"
Total time for all tests: 60789.1 seconds

comment:51 Changed 9 years ago by kcrisman

Okay, so the failures are the same as on mine *before* I add the two patches to the Maxima interface.

Summary:

  1. All Macs have some sort of multiplication transposition, so we will either have to simultaneously fix #10282 or find different doctests for those two examples (the taylor of gamma and the matrix in plot3d/transform.pyx).
  1. Debian and OpenSolaris? seem ok.
  1. Other Linux/Solaris? still needed?
  1. Finally, someone needs to actually review the content of the two Maxima interface change patches (one to Sage library, one to scripts library) and whether the spkgs do, in fact, have all properly changed (not that we don't believe the authors! just someone different needs to check them).

I still am getting this weird behavior of timeouts on mine. The problem seems to be restarting Maxima. Once Maxima is started, timings are slow but normal for this computer:

sage: time integrate(x^2,x)
CPU times: user 0.15 s, sys: 0.16 s, total: 0.31 s
Wall time: 134.04 s
1/3*x^3
sage: time integrate(x^2,x)
CPU times: user 0.08 s, sys: 0.01 s, total: 0.09 s
Wall time: 0.83 s
1/3*x^3
sage: time integrate(x^2,x)
CPU times: user 0.08 s, sys: 0.02 s, total: 0.10 s
Wall time: 0.92 s
1/3*x^3

But I think the way doctests work is that things are restarted from function to function - is that right?

I'm going to try to see what is going on with my computer, but if jpflori is ok with a very similar one, it shouldn't hold things up - unless there is a reason a G4 would be this much slower than a G5? I'm going to try downloading 4.6.1.alpha1, replacing the spkgs with the new ones, and then building from scratch to see what happens.

comment:52 in reply to: ↑ 46 Changed 9 years ago by jpflori

Nonetheless, we can't ship a Sage with a failing doctest that we know about, except at great need - this is why I put needs work. The very minor change in how Maxima grouped the coefficient of x^3 created this. So I recommend that we replace this doctest with something very similar which doesn't have this problem, and then open a new ticket for this issue. Maybe something like sage: taylor(gamma(1/4+x),x,0,3) except I can't tell you what the Mac output is on this until tomorrow - maybe someone else can. The new ticket is #10282.

As noted by Burcin in #10282 , it is related to pynac ordering and applying the patch I provided in #9880 gives me a consistent behavior on Mac and Linux:

sage: psi(1,1/3)*log(3)
log(3)*psi(1, 1/3)

(I order the functions according to their name in lexicographic order)

However #9880 still needs a lot of work:

  • my patch must be reviewed
  • pynac must be updated
  • a lot of doctests will have to be fixed afterwards

comment:53 follow-up: Changed 9 years ago by kcrisman

Okay, reverting the patches made no difference on the timing of starting Maxima, so maybe there is something wrong with Maxima itself on my computer... but it *does* allow some of the doctest files to actually pass. Hmm...

Yup, I just saw that update on #10282. So we have three options for this ticket:

Ignore the Mac doctest failure as a known issue with an open ticket OR make the failing doctests optional based on platform until #9880 is reviewed and merged etc. OR find similar doctests where the order doesn't matter.

comment:54 in reply to: ↑ 53 Changed 9 years ago by kcrisman

Replying to kcrisman:

Okay, reverting the patches made no difference on the timing of starting Maxima, so maybe there is something wrong with Maxima itself on my computer... but it *does* allow some of the doctest files to actually pass. Hmm...

Even weirder - testing a file with no long doctests, but which uses Maxima, passes without -long and times out with it. WHAT?

comment:55 Changed 9 years ago by kcrisman

Okay, after reverting to the previous ECL/Maxima on this machine, I get

sage: time integrate(x^3,x)
CPU times: user 0.14 s, sys: 0.14 s, total: 0.29 s
Wall time: 17.79 s
1/4*x^4
sage: time integrate(x^3,x)
CPU times: user 0.07 s, sys: 0.01 s, total: 0.08 s
Wall time: 0.20 s
1/4*x^4
sage: time integrate(x^3,x)
CPU times: user 0.08 s, sys: 0.02 s, total: 0.10 s
Wall time: 0.21 s
1/4*x^4

So Maxima is taking four times as long to do things, and the startup time is woefully long. Is anyone else noticing a slowdown on Maxima via Sage this way with the new spkg? I'd especially appreciate hearing from PPC OR Tiger users. As it stands, this package makes it so slow it makes Maxima (and integration etc.) almost useless on this machine.

comment:56 follow-up: Changed 9 years ago by vbraun

  • Status changed from needs_info to needs_review

The new maxima certainly comes with a bigger library; Its inevitable that it is somewhat slower in exchange for being better at integration. I don't know why startup is so slow on kcrisman's antique computer, but stracing the maxima startup shows that it thrashes around quite a lot in the $SAGE_LOCAL/share/maxima/5.22.1/ directory tree.

For the record, Fedora 14 on a Thinkpad T410s Core i5:

sage: time integrate(x^3,x)
CPU times: user 0.03 s, sys: 0.01 s, total: 0.04 s
Wall time: 1.15 s
1/4*x^4
sage: time integrate(x^3,x)
CPU times: user 0.01 s, sys: 0.00 s, total: 0.01 s
Wall time: 0.05 s
1/4*x^4
sage: time integrate(x^3,x)
CPU times: user 0.01 s, sys: 0.00 s, total: 0.01 s
Wall time: 0.05 s
1/4*x^4

I say we should press ahead and update maxima so that Sage compiles on Fedora 14. If the new maxima starts up slowly on your hardware then file a ticket and see if you can profile/fix it.

comment:57 follow-up: Changed 9 years ago by jpflori

On my Core 2 Quad? running Debian, here are the timings for the new Maxima:

sage: time integrate(x^3,x)
CPU times: user 0.02 s, sys: 0.01 s, total: 0.03 s
Wall time: 1.49 s
1/4*x^4
sage: time integrate(x^3,x)
CPU times: user 0.01 s, sys: 0.00 s, total: 0.01 s
Wall time: 0.06 s
1/4*x^4
sage: time integrate(x^3,x)
CPU times: user 0.01 s, sys: 0.00 s, total: 0.01 s
Wall time: 0.06 s
1/4*x^4

and with the old Maxima:

sage: time integrate(x^3,x)
CPU times: user 0.01 s, sys: 0.02 s, total: 0.03 s
Wall time: 1.42 s
1/4*x^4
sage: time integrate(x^3,x)
CPU times: user 0.01 s, sys: 0.00 s, total: 0.01 s
Wall time: 0.04 s
1/4*x^4
sage: time integrate(x^3,x)
CPU times: user 0.01 s, sys: 0.00 s, total: 0.01 s
Wall time: 0.04 s
1/4*x^4

On the Power G5 with the new Maxima:

sage: time integrate(x^3,x)
CPU times: user 0.04 s, sys: 0.06 s, total: 0.10 s
Wall time: 4.04 s
1/4*x^4
sage: time integrate(x^3,x)
CPU times: user 0.02 s, sys: 0.00 s, total: 0.02 s
Wall time: 0.03 s
1/4*x^4
sage: time integrate(x^3,x)
CPU times: user 0.02 s, sys: 0.01 s, total: 0.03 s
Wall time: 0.04 s
1/4*x^4

and the old Maxima:

age: time integrate(x^3,x)
CPU times: user 0.04 s, sys: 0.07 s, total: 0.10 s
Wall time: 4.87 s
1/4*x^4
sage: time integrate(x^3,x)
CPU times: user 0.02 s, sys: 0.00 s, total: 0.02 s
Wall time: 0.04 s
1/4*x^4
sage: time integrate(x^3,x)
CPU times: user 0.02 s, sys: 0.01 s, total: 0.03 s
Wall time: 0.04 s
1/4*x^4

So it was actually faster with the new one.

I managed to update Ecl and Maxima on top of sage 4.5.1 on a Sunblade 1500 (SPARC with Solaris 10), but doctesting maxima.py timedout or died eating all the memory.

comment:58 in reply to: ↑ 56 Changed 9 years ago by kcrisman

Replying to vbraun:

The new maxima certainly comes with a bigger library; Its inevitable that it is somewhat slower in exchange for being better at integration. I don't know why startup is so slow on kcrisman's antique computer, but stracing the maxima startup shows that it thrashes around quite a lot in the $SAGE_LOCAL/share/maxima/5.22.1/ directory tree. I say we should press ahead and update maxima so that Sage compiles on Fedora 14. If the new maxima starts up slowly on your hardware then file a ticket and see if you can profile/fix it.

Okay, but you'd still have to address the issue in Comment 52. The best option there is finding similar doctests that don't have different orders, since #9880 isn't necessarily going to be merged in the immediate future (see above).

comment:59 Changed 9 years ago by vbraun

OK lets mark the offending doctests # random output until #9880 is fixed. I'll attach a complimentary patch to #9880 that reverts trac_10187_mark_some_doctests_random_until_9880_gets_merged.patch.

comment:60 follow-up: Changed 9 years ago by jdemeyer

For tickets with many patches and multiple spkg's like this one, it would be very helpful to write in the description precisely which patches have to be applied.

comment:61 in reply to: ↑ 60 Changed 9 years ago by leif

Replying to jdemeyer:

For tickets with many patches and multiple spkg's like this one, it would be very helpful to write in the description precisely which patches have to be applied.

Dave, as you have admin rights on trac, you could delete the obsolete / redundant http://trac.sagemath.org/sage_trac/raw-attachment/ticket/10187/sage-maxima.lisp.patch . (It's a "duplicate" of http://trac.sagemath.org/sage_trac/raw-attachment/ticket/10187/trac_10187_sage-maxima_lisp.patch .)

comment:62 Changed 9 years ago by vbraun

  • Description modified (diff)

comment:63 follow-up: Changed 9 years ago by leif

The Maxima spkg IMHO needs work; not sure if on another ticket. (I guess the authors will say yes; I personally don't like postponing spkg changes very much, since this tends to make them never happen, or leads to concurrent, "incompatible" changes.)

Besides minor things (like unquoted SAGE_LOCAL), unsetting MAKE and using make is a very bad thing to do. This doesn't work as expected (or intended) with MAKE=make -j by the way, and prohibits the user to use different makes, or pass other options in it.

(Currently $SAGE_ROOT/spkg/install does similar if SAGE_PARALLEL_SPKG_BUILD is not "yes", but that's equally wrong, and that code isn't executed when doing e.g. ./sage -i ....)

Also, I see no reason for not removing GMP from the ECL source tree, which more than halves the size of the ECL spkg. This requires just a trivial change to configure s.t. a different copy of install-sh is used. (See #9493. To me it's pointless to create yet another spkg and bump the patch level for such a change when we're already at it, at least if it's likely to get merged into a different Sage release, which then causes completely useless downloads and rebuilds on upgrades.)

Such a patch to configure is a very good use case for the newly included GNU patch, since configure is huge while the patch is tiny, so putting a patched copy in patches/ would again - now really unnecessarily - increase the size of the resulting spkg.

I'm not sure if there are other changes from the previous attempt to update ECL not made in the current spkg here.

comment:64 Changed 9 years ago by leif

  * Removed comments about removing the src/contrib/encodings directory,
    as there is no such directory.
~/Sage/spkgs/ecl-10.4.1$ ls src/contrib/encodings/
ATARIST.BIN    DOS-CP855.BIN  DOS-CP865.BIN  ISO-8859-10.BIN  ISO-8859-2.BIN  ISO-8859-9.BIN  tools.lisp          WINDOWS-CP1256.BIN
CP-856.BIN     DOS-CP857.BIN  DOS-CP866.BIN  ISO-8859-11.BIN  ISO-8859-3.BIN  JISX0201.BIN    WINDOWS-CP1250.BIN  WINDOWS-CP1257.BIN
DOS-CP437.BIN  DOS-CP860.BIN  DOS-CP869.BIN  ISO-8859-13.BIN  ISO-8859-4.BIN  JISX0208.BIN    WINDOWS-CP1251.BIN  WINDOWS-CP1258.BIN
DOS-CP737.BIN  DOS-CP861.BIN  DOS-CP874.BIN  ISO-8859-14.BIN  ISO-8859-5.BIN  JISX0212.BIN    WINDOWS-CP1252.BIN  WINDOWS-CP932.BIN
DOS-CP775.BIN  DOS-CP862.BIN  generate.lisp  ISO-8859-15.BIN  ISO-8859-6.BIN  KOI8-R.BIN      WINDOWS-CP1253.BIN  WINDOWS-CP936.BIN
DOS-CP850.BIN  DOS-CP863.BIN  ISO-2022-JP    ISO-8859-16.BIN  ISO-8859-7.BIN  KOI8-U.BIN      WINDOWS-CP1254.BIN  WINDOWS-CP949.BIN
DOS-CP852.BIN  DOS-CP864.BIN  ISO-2022-JP-1  ISO-8859-1.BIN   ISO-8859-8.BIN  SHIFT-JIS.BIN   WINDOWS-CP1255.BIN  WINDOWS-CP950.BIN

comment:65 Changed 9 years ago by leif

  * Removed the 'patches' directory as there are no patches!
~/Sage/spkgs/ecl-10.4.1$ ls -l patches/
ls: cannot access patches/: No such file or directory
# Add a patch which works around a gcc bug on Solaris. Actually
# the code is not standard C, but a GCC extension, which does
# not work on Solaris, causing relocation issues - see 
# http://trac.sagemath.org/sage_trac/ticket/9840
# This is committed to the ECL trunk, so can be removed 
# whenever we update to a version of ECL newer than 10.4.1. 
cp patches/bytecodes.h src/src/h/bytecodes.h

comment:66 follow-up: Changed 9 years ago by vbraun

What is the correct way to force-disable parallel make, then?

Apart from that, do you have any other issue than the unquoted SAGE_LOCAL (which this ticked did not add) for the maxima spkg? I'm happy to fix "minor things", but at the very least you have to list them if you want them fixed.

I don't think that saving 1 MB is good enough a reason to justify patching ecl's configure and maintaining that patch in future versions. If you want to save space then there are much bigger spkgs that could be shrunk. Or, better yet, if you have the time then fix some real bugs...

comment:67 Changed 9 years ago by leif

There are other flaws in ECL's spkg-install; also, user-specified CFLAGS shouldn't get overridden unless necessary. (-O2 is appended rather than prepended.)

comment:68 in reply to: ↑ 66 ; follow-up: Changed 9 years ago by leif

Replying to vbraun:

What is the correct way to force-disable parallel make, then?

As long as we rely on a GNU make, the correct thing to do is

MAKE="$MAKE -j1" # force sequential build (1 job)

Otherwise we'd have to test if the actual $MAKE understands that.

Apart from that, do you have any other issue than the unquoted SAGE_LOCAL (which this ticked did not add) for the maxima spkg? I'm happy to fix "minor things", but at the very least you have to list them if you want them fixed.

:) I don't recall. A major clean-up could be done, but I won't insist on having it here.


I don't think that saving 1 MB is good enough a reason to justify patching ecl's configure and maintaining that patch in future versions. If you want to save space then there are much bigger spkgs that could be shrunk.

Yes, but they accumulate. I'm not sure if upstream would make such a patch.

Btw, it's more than 2 MB of the spkg, 13 MB uncompressed.

"Maintaining" a real patch is trivial. It will still apply or not, in which case an error message is given. The patch itself is also trivial.

Or, better yet, if you have the time then fix some real bugs.

comment:69 in reply to: ↑ 68 Changed 9 years ago by leif

Replying to leif:

Replying to vbraun:

I don't think that saving 1 MB is good enough a reason to justify patching ecl's configure and maintaining that patch in future versions. If you want to save space then there are much bigger spkgs that could be shrunk.

Yes, but they accumulate. I'm not sure if upstream would make such a patch.

Btw, it's more than 2 MB of the spkg, 13 MB uncompressed.

"Maintaining" a real patch is trivial. It will still apply or not, in which case an error message is given. The patch itself is also trivial.

P.S.: If someone fears patch, a trivial alternative is to simply keep the only file from there configure uses, which is a redundant copy anyway, so one could even create a symbolic link to one of the other instances.

~/Sage/spkgs/ecl-10.4.1$ find . -name install-sh -exec head -v {} \;
==> ./src/src/gc/install-sh <==
#!/bin/sh
#
# install - install a program, script, or datafile
# This comes from X11R5 (mit/util/scripts/install.sh).
#
# Copyright 1991 by the Massachusetts Institute of Technology
#
# Permission to use, copy, modify, distribute, and sell this software and its
# documentation for any purpose is hereby granted without fee, provided that
# the above copyright notice appear in all copies and that both that
==> ./src/src/gc/libatomic_ops-1.2/install-sh <==
#!/bin/sh
# install - install a program, script, or datafile

scriptversion=2005-05-14.22

# This originates from X11R5 (mit/util/scripts/install.sh), which was
# later released in X11R6 (xc/config/util/install.sh) with the
# following copyright and license.
#
# Copyright (C) 1994 X Consortium
==> ./src/src/gmp/install-sh <==
#!/bin/sh
# install - install a program, script, or datafile

scriptversion=2004-04-01.17

# This originates from X11R5 (mit/util/scripts/install.sh), which was
# later released in X11R6 (xc/config/util/install.sh) with the
# following copyright and license.
#
# Copyright (C) 1994 X Consortium

comment:70 follow-up: Changed 9 years ago by leif

With the two new spkgs and all four patches applied, I got:

sage -t  -long -force_lib devel/sage/sage/modular/modform/ambient.py
*** *** Error: TIMED OUT! PROCESS KILLED! *** ***

	 [1800.1 s]
 
----------------------------------------------------------------------

The following tests failed:

	sage -t  -long -force_lib devel/sage/sage/interfaces/r.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/modular/modform/ambient.py # Time out
----------------------------------------------------------------------
Total time for all tests: 3619.8 seconds
make: *** [ptestlong] Error 192

(Sage 4.6.1.alpha2 with MPIR 2.1.3.p1 and [GMP-]ECM 6.3.p2, Ubuntu 10.04 x86_64)

Rerunning the tests individually, everything passes:

~/Sage/sage-4.6.1.alpha2/devel/sage-10187$ ../../sage -t -long -force_lib sage/interfaces/r.py
sage -t -long -force_lib "devel/sage-10187/sage/interfaces/r.py"
         [23.5 s]
 
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 23.5 seconds
~/Sage/sage-4.6.1.alpha2/devel/sage-10187$ ../../sage -t -long -force_lib sage/modular/modform/ambient.py
sage -t -long -force_lib "devel/sage-10187/sage/modular/modform/ambient.py"
         [5.5 s]
 
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 5.5 seconds

I've run ptestlong many times on that installation (before testing #10187) without any errors... Quite weird.

comment:71 in reply to: ↑ 70 Changed 9 years ago by jdemeyer

Replying to leif:

The following tests failed:

sage -t -long -force_lib devel/sage/sage/interfaces/r.py # 1 doctests failed

This is probably #9970.

Changed 9 years ago by drkirkby

Remove the comment about not removing the 'endcodings' directory, as it should have been remove.d Removed attempt to patch on Solaris, as the patch did not work. This patch is ror review purposes only, as the changes are in ecl-10.4.1.spkg

comment:72 in reply to: ↑ 63 ; follow-up: Changed 9 years ago by drkirkby

Replying to leif:

The Maxima spkg IMHO needs work; not sure if on another ticket. (I guess the authors will say yes; I personally don't like postponing spkg changes very much, since this tends to make them never happen, or leads to concurrent, "incompatible" changes.)

A point that needs to be born in mind is that whilst the Maxima package may not be perfect (I've not looked myself), a failure to update ECL and Maxima will mean Sage will not build on the latest version of Fedora, which is the second most popular linux distro. So I feel there is some argument here for possibly accepting a package that could be improved. (I say this, despite having no involvement whatsoever in updating the Maxima package). But ECL and Maxima need to done together.

  • I've removed the src/contrib/encodings/ directory in the ECL package, and corrected comments in SPKG.txt about this.
  • I've removed the cp patches/bytecodes.h src/src/h/bytecodes.h in the ECL package, along iwth the comments attached to why the copy was being made. (Basically the patch suggested by the ECL developer did not work on the latest stable ECL - only on a CVS snapshot).
  • I've updated the ecl-10.4.1.spkg. The changes should be totally harmless - a copy which would never have occured, some changes of comments and the removal of some unused files. The md5 checksum should be 7a0d66ee152b847009b23b1dbbfeb75d

Such a patch to configure is a very good use case for the newly included GNU patch, since configure is huge while the patch is tiny, so putting a patched copy in patches/ would again - now really unnecessarily - increase the size of the resulting spkg.

Agreed, but it would be unwise to start using GNU patch here for several reasons.

  • We have not agreed on how its best used (that's the subject of #9419)
  • Patch is not in any stable release of Sage. It would be unwise to switch to using it extensively just now.
  • Robert Bradshaw stated we should not mix use the use of patch and cp in the same package. I'm not sure if I agree with that or not. I can see his logic, though I can see it could be an error prone process converting all the cp's to patches.

I'm not sure if there are other changes from the previous attempt to update ECL not made in the current spkg here.

I'm not aware of any.

Dave

comment:73 follow-up: Changed 9 years ago by vbraun

I've updated the maxima spkg to address the quotes and the MAKE issue. For review purposes see the maxima-5.22.1.patch.

comment:74 in reply to: ↑ 72 Changed 9 years ago by leif

Replying to drkirkby:

Replying to leif:

The Maxima spkg IMHO needs work; not sure if on another ticket. (I guess the authors will say yes; I personally don't like postponing spkg changes very much, since this tends to make them never happen, or leads to concurrent, "incompatible" changes.)

A point that needs to be born in mind is that whilst the Maxima package may not be perfect (I've not looked myself), a failure to update ECL and Maxima will mean Sage will not build on the latest version of Fedora, which is the second most popular linux distro. So I feel there is some argument here for possibly accepting a package that could be improved. (I say this, despite having no involvement whatsoever in updating the Maxima package). But ECL and Maxima need to done together.

If there's a chance of getting it into 4.6.1 (and the authors are willing to open follow-up tickets ;-) ), yes. I'm not the release manager, but other spkg updates / changes have been postponed to Sage 4.6.2. Clearly Maxima is also a "central" package in Sage, so any upgrade carries the risk to break potentially a lot, or at least more than other spkgs hardly used within Sage.


I'm not sure if there are other changes from the previous attempt to update ECL not made in the current spkg here.

I'm not aware of any.

Certainly not... :D

(Otherwise I would have expected you to incorporate them if necessary. But I see your ECL package is "fresh"; I first thought you just took some "old" one since there were not many comments on it on the ticket, and no patch or diff attached.)

comment:75 in reply to: ↑ 73 ; follow-up: Changed 9 years ago by leif

Replying to vbraun:

I've updated the maxima spkg to address the quotes and the MAKE issue. For review purposes see the maxima-5.22.1.patch.

Well, if you redefine MAKE rather than unsetting it, you should also use it... (i.e. call $MAKE instead of make).

comment:76 in reply to: ↑ 75 Changed 9 years ago by leif

Replying to leif:

Replying to vbraun:

I've updated the maxima spkg to address the quotes and the MAKE issue. For review purposes see the maxima-5.22.1.patch.

Well, if you redefine MAKE rather than unsetting it, you should also use it... (i.e. call $MAKE instead of make).

Of course you could also just do

  • spkg-install

    diff --git a/spkg-install b/spkg-install
    a b  
    1414
    1515cd src/
    1616
    17 unset MAKE  # this can break things
    18 
    1917check_error() {
    2018    if [ $? -ne 0 ]; then
    2119        echo "***********************************************************"
     
    3533   echo "you will not be able to see the output of the build"
    3634   echo "as it occurs.  Don't worry, the build process did"
    3735   echo "not hang."
    38    make >> "$CUR"/output.log  2>> "$CUR"/error.log
     36   $MAKE -j1 >> "$CUR"/output.log  2>> "$CUR"/error.log
    3937else
    40    make
     38   $MAKE -j1
    4139fi
    4240check_error "Failed to make Maxima."
    4341
    44 make install
     42$MAKE -j1 install
    4543check_error "Failed to install Maxima."
    4644
    4745echo "creating wrapper script with disabled readline"

P.S.: Did anybody recently test where and if parallel Maxima builds or installs fail? Building it sequentially takes a fair amount of time...

comment:77 follow-up: Changed 9 years ago by vbraun

Fixed the make->$MAKE, too. Too much jet lag ;-)

I haven't tried if parallel build suddenly works. But then, even if it were to compile in parallel on my machine, what do we learn?

comment:78 in reply to: ↑ 77 ; follow-up: Changed 9 years ago by leif

Replying to vbraun:

I haven't tried if parallel build suddenly works. But then, even if it were to compile in parallel on my machine, what do we learn?

Need not be suddenly.

If e.g. just make install fails with multiple jobs, we could just do $MAKE -j1 install.

Also, failures might be limited to the OS (e.g. MacOS X), in which case we shouldn't disable parallel builds on all platforms.

If parallel builds or installs just non-deterministically fail in, say, 5-10%, it would IMHO be acceptable to enable them though, since on a failure one simply has to rerun make or ./sage -i ....

comment:79 follow-up: Changed 9 years ago by leif

Looks as if enabling parallel make (in Maxima) doesn't make any difference, so rather a non-issue.

comment:80 in reply to: ↑ 79 Changed 9 years ago by drkirkby

Replying to leif:

Looks as if enabling parallel make (in Maxima) doesn't make any difference, so rather a non-issue.

I can't do it now, but later I will try my old trick of building it 1000 times, inserting random delays before invoking gcc. I don't think a few]build in parallel proves very much.

It took:

real    9m0.386s
user    5m4.518s
sys     0m43.198s
Successfully installed maxima-5.22.1

so I guess it would take about 6 minutes to build serially. So we will see see how long it takes to build in parallel and so how practical it is to test it 100 times or 1000 times.

Dave

Dave

Dave

comment:81 in reply to: ↑ 78 Changed 9 years ago by drkirkby

  • Description modified (diff)
  • Status changed from needs_review to needs_info

Replying to leif:

If parallel builds or installs just non-deterministically fail in, say, 5-10%, it would IMHO be acceptable to enable them though, since on a failure one simply has to rerun make or ./sage -i ....

I personally think it would be unwise to enable parallel builds unless we are confident they work reliably on every platform. It's conceivable a file may not be written fully, yet processed in some way, leading to undefined behavior.

I've asked on the Maxima mailing list if parallel builds are OK or not. So far there is no reply.

I've started a build of Maxima, inserting a random delay between 0 and 1 second before invoking gcc. The problem is, Maxima is now taking 11 and a half minutes to build on my machine.

real    11m33.576s
user    3m22.703s
sys     0m29.970s

as obviously delaying gcc on average by 500 ms slows down the build. So far it has only built twice, but both times it was OK.

I'll build a few more copies of Sage, then test Maxima on them too, since testing this even 100 times is going to take 19 hours.

I've put to needs info, since I think we need to verify if parallel builds are ok or not.

A new package, with parallel builds enabled, can be found at

http://boxen.math.washington.edu/home/kirkby/patches/maxima-5.22.1.spkg

Dave

comment:82 Changed 9 years ago by drkirkby

  • Description modified (diff)
  • Status changed from needs_info to needs_review

I tried to build Maxima in parallel a number of times. It failed once with the following, so I think it is unwise to attempt a parallel build.

This was without any random delays before invoking gcc, but on a heavily loaded system.

We should use http://www.stp.dias.ie/~vbraun/Sage/spkg/maxima-5.22.1.spkg and not my parallel version.

/usr/bin/ginstall -c -m 644 maxima-index.lisp "/export/home/drkirkby/1/sage-4.6.1.alpha2/local/info/maxima-index.lisp"
mkdir: /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html: [File exists]
make[6]: *** [install-maxima-html] Error 1
make[6]: *** Waiting for unfinished jobs....
 /usr/bin/ginstall -c -m 644 ./figures/contour1.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/contour1.gif
 /usr/bin/ginstall -c -m 644 ./figures/contour2.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/contour2.gif
 /usr/bin/ginstall -c -m 644 ./figures/contour3.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/contour3.gif
 /usr/bin/ginstall -c -m 644 ./figures/dynamics1.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/dynamics1.gif
 /usr/bin/ginstall -c -m 644 './maxima.info' '/export/home/drkirkby/1/sage-4.6.1.alpha2/local/info/maxima.info'
 /usr/bin/ginstall -c -m 644 './maxima.info-1' '/export/home/drkirkby/1/sage-4.6.1.alpha2/local/info/maxima.info-1'
 /usr/bin/ginstall -c -m 644 ./figures/dynamics2.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/dynamics2.gif
 /usr/bin/ginstall -c -m 644 './maxima.info-2' '/export/home/drkirkby/1/sage-4.6.1.alpha2/local/info/maxima.info-2'
 /usr/bin/ginstall -c -m 644 ./figures/dynamics3.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/dynamics3.gif
 /usr/bin/ginstall -c -m 644 ./figures/dynamics4.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/dynamics4.gif
 /usr/bin/ginstall -c -m 644 ./figures/dynamics5.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/dynamics5.gif
 /usr/bin/ginstall -c -m 644 ./figures/dynamics6.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/dynamics6.gif
 /usr/bin/ginstall -c -m 644 './maxima.info-3' '/export/home/drkirkby/1/sage-4.6.1.alpha2/local/info/maxima.info-3'
 /usr/bin/ginstall -c -m 644 ./figures/dynamics7.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/dynamics7.gif
 /usr/bin/ginstall -c -m 644 ./figures/dynamics8.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/dynamics8.gif
 install-info --info-dir='/export/home/drkirkby/1/sage-4.6.1.alpha2/local/info' '/export/home/drkirkby/1/sage-4.6.1.alpha2/local/info/maxima.info'
 /usr/bin/ginstall -c -m 644 ./figures/dynamics9.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/dynamics9.gif
 /usr/bin/ginstall -c -m 644 ./figures/dynamics10.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/dynamics10.gif
 /usr/bin/ginstall -c -m 644 ./figures/implicit_plot.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/implicit_plot.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotdf1.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotdf1.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotdf2.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotdf2.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotdf3.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotdf3.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotdf4.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotdf4.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotdf5.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotdf5.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotdf6.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotdf6.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting1.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting1.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting2.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting2.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting3.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting3.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting4.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting4.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting5.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting5.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting6.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting6.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting7.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting7.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting8.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting8.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting9.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting9.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting10.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting10.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting11.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting11.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting12.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting12.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting13.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting13.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting14.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting14.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting15.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting15.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting16.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting16.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting17.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting17.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting18.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting18.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting19.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting19.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting20.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting20.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting21.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting21.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting22.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting22.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting23.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting23.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting24.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting24.gif
 /usr/bin/ginstall -c -m 644 ./figures/plotting25.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/plotting25.gif
 /usr/bin/ginstall -c -m 644 ./figures/orthopoly1.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/orthopoly1.gif
 /usr/bin/ginstall -c -m 644 ./figures/graphs01.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/graphs01.gif
 /usr/bin/ginstall -c -m 644 ./figures/graphs02.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/graphs02.gif
 /usr/bin/ginstall -c -m 644 ./figures/graphs03.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/graphs03.gif
 /usr/bin/ginstall -c -m 644 ./figures/graphs04.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/graphs04.gif
 /usr/bin/ginstall -c -m 644 ./figures/graphs05.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/graphs05.gif
 /usr/bin/ginstall -c -m 644 ./figures/graphs06.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/graphs06.gif
 /usr/bin/ginstall -c -m 644 ./figures/graphs07.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/graphs07.gif
 /usr/bin/ginstall -c -m 644 ./figures/graphs08.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/graphs08.gif
 /usr/bin/ginstall -c -m 644 ./figures/graphs09.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/graphs09.gif
 /usr/bin/ginstall -c -m 644 ./figures/graphs10.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/graphs10.gif
 /usr/bin/ginstall -c -m 644 ./figures/graphs11.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/graphs11.gif
 /usr/bin/ginstall -c -m 644 ./figures/graphs12.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/graphs12.gif
 /usr/bin/ginstall -c -m 644 ./figures/graphs13.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/graphs13.gif
 /usr/bin/ginstall -c -m 644 ./figures/graphs14.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/graphs14.gif
 /usr/bin/ginstall -c -m 644 ./figures/graphs15.gif /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/figures/graphs15.gif
 /usr/bin/ginstall -c -m 644 ./contents.hhc /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/contents.hhc
 /usr/bin/ginstall -c -m 644 ./index.hhk /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/index.hhk
 /usr/bin/ginstall -c -m 644 ./header.hhp /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/header.hhp
 /usr/bin/ginstall -c -m 644 ./maxima.hhp /export/home/drkirkby/1/sage-4.6.1.alpha2/local/share/maxima/5.22.1/doc/html/maxima.hhp
make[6]: Leaving directory `/export/home/drkirkby/1/sage-4.6.1.alpha2/spkg/build/maxima-5.22.1/src/doc/info'
make[5]: *** [install-am] Error 2
make[5]: Leaving directory `/export/home/drkirkby/1/sage-4.6.1.alpha2/spkg/build/maxima-5.22.1/src/doc/info'
make[4]: *** [install-recursive] Error 1
make[4]: Leaving directory `/export/home/drkirkby/1/sage-4.6.1.alpha2/spkg/build/maxima-5.22.1/src/doc/info'
make[3]: *** [install-recursive] Error 1
make[3]: Leaving directory `/export/home/drkirkby/1/sage-4.6.1.alpha2/spkg/build/maxima-5.22.1/src/doc'
make[2]: *** [install-recursive] Error 1
make[2]: Leaving directory `/export/home/drkirkby/1/sage-4.6.1.alpha2/spkg/build/maxima-5.22.1/src'
***********************************************************
Failed to install Maxima.
***********************************************************

real	11m2.520s
user	5m0.763s
sys	0m33.530s
sage: An error occurred while installing maxima-5.22.1

comment:83 Changed 9 years ago by drkirkby

As seen above, the parallel build appeared to work, but the install failed. But I think we should leave this. I've reported the failure on the Maxima list.

I don't feel able to review the maxima package, and obviously can't the ECL one, as I wrote most of it.

But I'm fairly convinced these two packages are working ok together myself, based on the buildbot results.

Dave

comment:84 follow-up: Changed 9 years ago by drkirkby

This was posted on the Maxima list by Rupert Swarbrick, who is a Maxima developer.


Sorry, I think I was not clear. What I meant was that most of the time compiling maxima is spent compiling code written in lisp. This code is compiled in order using a "Makefile-like" process called asdf (or mk:defsystem, depending on which you prefer) and this is a serial process.

There are projects like XCVB that have tried to do this in parallel for building large lisp images, but it's not clear how much success they've had.

Rupert


So it looks like we wont gain much trying to build in parallel and as I've shown, at least installing in parallel is unsafe.

Dave

comment:85 in reply to: ↑ 57 ; follow-up: Changed 9 years ago by drkirkby

Replying to jpflori:

I managed to update Ecl and Maxima on top of sage 4.5.1 on a Sunblade 1500 (SPARC with Solaris 10), but doctesting maxima.py timedout or died eating all the memory.

On a Blade 1500, you will need to increase the values of the timeouts. I would suggest suitable values would be :

SAGE_TIMEOUT=600
SAGE_TIMEOUT_LONG=3000

I suspect the SAGE_TIMEOUT_LONG could actually be 2000 seconds, but I'd go for 6000 to be really safe.

I use:

SAGE_TIMEOUT=1000
SAGE_TIMEOUT_LONG=10000

on a dual processor 900 MHz Blade 1000. I think however the SAGE_TIMEOUT_LONG, which defaults to 1800 s, is probably not too far from ok, so I reckon 2000 s would be fine.

The default SAGE_TIMEOUT of 300 s will certainly be too short.

Dave

comment:86 Changed 9 years ago by drkirkby

What do we need for a positive review of this?

comment:87 in reply to: ↑ 85 ; follow-up: Changed 9 years ago by jpflori

Replying to drkirkby:

On a Blade 1500, you will need to increase the values of the timeouts. I would suggest suitable values would be : SAGE_TIMEOUT=600 SAGE_TIMEOUT_LONG=3000 I suspect the SAGE_TIMEOUT_LONG could actually be 2000 seconds, but I'd go for 6000 to be really safe. I use: SAGE_TIMEOUT=1000 SAGE_TIMEOUT_LONG=10000 on a dual processor 900 MHz Blade 1000. I think however the SAGE_TIMEOUT_LONG, which defaults to 1800 s, is probably not too far from ok, so I reckon 2000 s would be fine. The default SAGE_TIMEOUT of 300 s will certainly be too short. Dave

I did increase the timeout, to 20000 if I remember correctly.

However when I came back Sage had gotten out of memory (and the computer had to be restarted).

It could be because of the way I installed Sage (downloaded binary 4.5.1 package, updated the spkgs and could have used an old gcc at some point because the default one on the system is 3.4.something even if 4.something is available).

I should build everything from scratch but do not have the time right now.

comment:88 in reply to: ↑ 87 ; follow-up: Changed 9 years ago by drkirkby

Replying to jpflori:

I did increase the timeout, to 20000 if I remember correctly.

That is more than enough. Remember there are two timeout values. I think if you set just SAGE_TIMEOUT to 20000 seconds, but leave SAGE_TIMEOUT_LONG at the default 1800 seconds, then the long tests will just use 1800 seconds.

However when I came back Sage had gotten out of memory (and the computer had to be restarted).

I can build Sage and doctest it on a Sun Blade 1000 which has only 2 GB RAM.

It could be because of the way I installed Sage (downloaded binary 4.5.1 package, updated the spkgs and could have used an old gcc at some point because the default one on the system is 3.4.something even if 4.something is available).

That indeed would screw it up. We should run the 'prereq' script every time we run 'make' as that will stop some instances of people mess up, but I think installing individual packages with sage -f foobar, will be difficult to detect problems like a wrong gcc.

I should build everything from scratch but do not have the time right now.

It will take a long time on a blade 1500, so I suggest you start it asap.

Dave

comment:89 in reply to: ↑ 84 ; follow-up: Changed 9 years ago by leif

Replying to drkirkby:

This was posted on the Maxima list by Rupert Swarbrick, who is a Maxima developer.


Sorry, I think I was not clear. What I meant was that most of the time compiling maxima is spent compiling code written in lisp. This code is compiled in order using a "Makefile-like" process called asdf (or mk:defsystem, depending on which you prefer) and this is a serial process.

There are projects like XCVB that have tried to do this in parallel for building large lisp images, but it's not clear how much success they've had.

Rupert


So it looks like we wont gain much trying to build in parallel

... as I said a few comments above.

and as I've shown, at least installing in parallel is unsafe.

(If building Maxima in parallel would make [more] sense, we could still disable just parallel installation.)

comment:90 in reply to: ↑ 88 ; follow-up: Changed 9 years ago by leif

Replying to drkirkby:

Replying to jpflori:

I did increase the timeout, to 20000 if I remember correctly.

That is more than enough. Remember there are two timeout values. I think if you set just SAGE_TIMEOUT to 20000 seconds, but leave SAGE_TIMEOUT_LONG at the default 1800 seconds, then the long tests will just use 1800 seconds.

Unless someone has changed this recently, if you run some doctest(s) with -long, only SAGE_TIMEOUT_LONG will be used, regardless if the file(s) contain(s) doctests marked "long".

(And the timeout applies to the whole file being doctested, not individual doctests in it.)

If you run out of memory (i.e., the machine starts to swap), increasing the timeout doesn't help much; it's better to then reduce the number of doctests run in parallel ("NUM_THREADS" in the Makefile).

However when I came back Sage had gotten out of memory (and the computer had to be restarted).

I can build Sage and doctest it on a Sun Blade 1000 which has only 2 GB RAM.

comment:91 in reply to: ↑ 89 Changed 9 years ago by drkirkby

Replying to leif:

(If building Maxima in parallel would make [more] sense, we could still disable just parallel installation.)

Yes, but for various reasons, there's no point.

Dave

comment:92 in reply to: ↑ 90 Changed 9 years ago by drkirkby

Replying to leif:

Replying to drkirkby:

That is more than enough. Remember there are two timeout values. I think if you set just SAGE_TIMEOUT to 20000 seconds, but leave SAGE_TIMEOUT_LONG at the default 1800 seconds, then the long tests will just use 1800 seconds.

Unless someone has changed this recently, if you run some doctest(s) with -long, only SAGE_TIMEOUT_LONG will be used, regardless if the file(s) contain(s) doctests marked "long".

I was not aware of that.

(And the timeout applies to the whole file being doctested, not individual doctests in it.)

If you run out of memory (i.e., the machine starts to swap), increasing the timeout doesn't help much; it's better to then reduce the number of doctests run in parallel ("NUM_THREADS" in the Makefile).

However when I came back Sage had gotten out of memory (and the computer had to be restarted).

I know there are exceptions, but normally one can kill the process that's taking all the ram. If ulimit was set, this should not happen.

I can build Sage and doctest it on a Sun Blade 1000 which has only 2 GB RAM.

FWIW, that's running Solaris 10 03/2005, so is an almost 6 year old version of Solaris.

A later version of Solaris, if configured to use the 128-bit ZFS file system, then I doubt 2 GB would be sufficient. One could always use the old UFS file system, but ZFS has lots of advantages, but it tends to need more memory.

Dave

comment:93 follow-up: Changed 9 years ago by leif

Don't know if I mentioned this earlier, but in the ECL spkg user-specified CFLAGS (and CXXFLAGS) still get overridden even if SAGE_DEBUG is not "yes".

Are CXX and CXXFLAGS used by ECL at all? I don't think so.

comment:94 Changed 9 years ago by kcrisman

  • Reviewers changed from Karl-Dieter Crisman to Karl-Dieter Crisman, David Kirkby, Volker Braun

Okay, I was able (finally) to rebuild everything on OS X 10.4 PPC and now have no problems.

sage: time integrate(x^3,x)
CPU times: user 0.13 s, sys: 0.13 s, total: 0.26 s
Wall time: 14.62 s
1/4*x^4
sage: time integrate(x^3,x)
CPU times: user 0.07 s, sys: 0.01 s, total: 0.08 s
Wall time: 0.17 s
1/4*x^4
sage: time integrate(x^3,x)
CPU times: user 0.08 s, sys: 0.02 s, total: 0.10 s
Wall time: 0.19 s
1/4*x^4

As to review and Volker's comments on sage-devel, note that sometimes the people who can review the functioning of the patch/spkg are not the same as those who can review the actual changed code. That is probably a big piece of the problem.

Anyway, I cannot review the lisp-ish changes and whether they are needed, though I can look at the rest and of course verify that things seem to work.

However, you and David could review each others' spkgs in computer terms; I think that might be okay, n'est pas?

comment:95 Changed 9 years ago by kcrisman

  • Reviewers changed from Karl-Dieter Crisman, David Kirkby, Volker Braun to Karl-Dieter Crisman, David Kirkby, Volker Braun, Leif Leonhardy

comment:96 in reply to: ↑ 93 ; follow-up: Changed 9 years ago by drkirkby

Replying to leif:

Don't know if I mentioned this earlier, but in the ECL spkg user-specified CFLAGS (and CXXFLAGS) still get overridden even if SAGE_DEBUG is not "yes".

Are CXX and CXXFLAGS used by ECL at all? I don't think so.

CXX is definitely used. I just tried to specify CXX=/path/to/Sun/C++-compiler

and it used the Sun compiler.

I must admit, I'm not so convinced it is honoring CXXFLAGS, but it would be unwise to not set them. That's really an ECL bug if it ignore the flags.

Dave

comment:97 in reply to: ↑ 96 ; follow-up: Changed 9 years ago by leif

Replying to drkirkby:

Replying to leif:

Don't know if I mentioned this earlier, but in the ECL spkg user-specified CFLAGS (and CXXFLAGS) still get overridden even if SAGE_DEBUG is not "yes".

So did you fix that?


Are CXX and CXXFLAGS used by ECL at all? I don't think so.

CXX is definitely used. I just tried to specify CXX=/path/to/Sun/C++-compiler

and it used the Sun compiler.

:-) What for?

I must admit, I'm not so convinced it is honoring CXXFLAGS, but it would be unwise to not set them. That's really an ECL bug if it ignore the flags.

(At least I couldn't find CXXFLAGS being used in the build logs.)

comment:98 in reply to: ↑ 97 Changed 9 years ago by leif

Replying to leif:

Replying to drkirkby:

Replying to leif:

Are CXX and CXXFLAGS used by ECL at all? I don't think so.

CXX is definitely used. I just tried to specify CXX=/path/to/Sun/C++-compiler

and it used the Sun compiler.

:-) What for?

I just successfully installed ECL (and afterwards Maxima) with env CXX=/some/nonexistent/cxx ./sage -f ....

configure tests CXX, but it'll only be used if you also build the included GMP, with C++ support, I guess.

comment:99 follow-ups: Changed 9 years ago by drkirkby

  • Status changed from needs_review to needs_work

Some comments, having spent some time looking at the code others have written.

My ECL package

  • I don't see anything wrong with the way CXXFLAGS/CFLAGS is handled in the ECL package. It looks OK to me. It might not be 100% necessary, but in general we should set these flags and hope the package follows this.

Looking at the doc tests:

  • trac_10187_general_display_prefix_workaround.patch looks OK to me. So I'll positively review that part.
  • trac_10187_mark_some_doctests_random_until_9880_gets_merged.patch looks OK to me, So I'll positively review that part.
  • maxima-5.22.1.patch SPKG.txt looks OK, but you will need to find someone that know more about Lisp than me to tell you if that's correct. Ask on sage-devel to get someone who understands it, since I don't.
  • trac_10187_fix_easy_doctests.patch

1) There are these numerical results. Has anyone independently verified this are correct?

[2.6789385347..., -8.3905259853..., 26.662447494..., -80.683148377...] 

On what basis was the number of significant digits selected?

As a matter of interest, do we know if anyone every verify the original horrid looking symbolic integral is correct?

I have a dislike for tests where is appears we don't verify the result is correct, but assume it must be. Please justify why those results are correct. That should be in a comment in the code, not displayed.

2) Why was the doctest:

integral(abs(x), x, 0, a)

changed to:

integral(abs(x)*x, x, 0, a) 

It rather strikes me that we should show the example of how to do this calculation in the way Maxima now wants, rather than to simply change the integral to one which gives a simpler answer.

It so happens that Mathematica 7.0.1 can do the first inegral:

In[7]:= Integrate[Abs[x],{x,0,a}]

        a Abs[a]
Out[7]= --------
           2

I think it would be better to show the original integral, show the failure. We could even point out how to do the integral in Maxima, and get the hopefully correct result. Is rather strikes me that inputting something that was quite reasonable looking does not work. So we should explain why that reasonable thing is wrong, rather than just pick another example.

The Mathematica documentation often shows a section marked Possible issues and would explain problems one might get. Here it seems we should do this, rather than just pick another integral.

sage/plot/plot3d/transform.pyx Do we know if that horrible looking result is correct? I see it is a simple change from the previous horrible looking result, but how do we know if the previous one is correct? If, as I suspect, this has not been verified by hand or with another package, then we should write that fact.

Summary

  • The portability issues appears to have been resolved.
  • Someone needs to check my ECL package
  • trac_10187_general_display_prefix_workaround.patch is OK
  • trac_10187_mark_some_doctests_random_until_9880_gets_merged.patch is OK
  • trac_10187_fix_easy_doctests.patch Needs explanation.
  • maxima-5.22.1.patch Needs someone to review with a skill set different to me.

I'm changing to needs work, as some of these tests changes needs explanation.

Dave

comment:100 in reply to: ↑ 99 Changed 9 years ago by leif

Replying to drkirkby:

My ECL package

  • I don't see anything wrong with the way CXXFLAGS/CFLAGS is handled in the ECL package. It looks OK to me. It might not be 100% necessary, but in general we should set these flags and hope the package follows this.

If you change

if [ "x$SAGE_DEBUG" = xyes ] ; then
   CFLAGS="$CFLAGS -g -O0"
   CXXFLAGS="$CXXFLAGS -g -O0"
else
   CFLAGS="$CFLAGS -g -O2"
   CXXFLAGS="$CXXFLAGS -g -O2"
fi

to

if [ "x$SAGE_DEBUG" = xyes ] ; then
   CFLAGS="$CFLAGS -g -O0"
   CXXFLAGS="$CXXFLAGS -g -O0"
else
   CFLAGS="-g -O2 $CFLAGS"
   CXXFLAGS="-g -O2 $CXXFLAGS"
fi

and add a comment that neither CXX nor CXXFLAGS are used within Sage / the way we configure / install ECL, I'm ok with your package.

In the long run, CFLAG64 and CXXFLAG64 should be set to their defaults (if empty) in (e.g.) sage-env.

You could remove the superfluous

set +e

The comment

# 'export MAKE='make -j n' where n>1, breaks ECL builds, so unset make

isn't up-to-date (since we do no longer unset MAKE).

I have no idea what

# We clear MAKEFLAGS to fix building multiple spkgs in parallel on OS X.
export MAKEFLAGS=

is (now) intended to fix; MAKE="$MAKE -j1" should be sufficient. If it was really necessary on Darwin, it should only be done on Darwin, since it clears other MAKEFLAGS unrelated to parallel builds as well.

The following comment is a bit weird; ln -sf ... should do the right thing (without rm).

# Create symbolic link to lib/ecl-version directory.
# This is important when the Sage install is moved.
cd "$SAGE_LOCAL/lib/" && rm -f ecl && ln -s ecl-* ecl

The following won't work with spaces in $SAGE_LOCAL (currently a minor issue):

CPPFLAGS="$CPPFLAGS -I$SAGE_LOCAL/include"
LDFLAGS="$LDFLAGS -L$SAGE_LOCAL/lib"

(We could instead use --with-gmp-prefix="$SAGE_LOCAL", and similar for the Boehm garbage collector...)

Perhaps add a comment to SPKG.txt that the included Boehm GC is used on MacOS X (only), if this is still the case. (I think it is; ECL used to assume any other Boehm GC found on MacOS X must be Fink's broken one, so always used its own there.)

I don't agree with the comment regarding the removal of the GMP source tree; leaving just install-sh there is totally safe, without requiring any patch.


I'm ok with the Maxima spkg (IIRC ;-) ).

comment:101 in reply to: ↑ 99 Changed 9 years ago by vbraun

Replying to drkirkby:

  • maxima-5.22.1.patch SPKG.txt looks OK, but you will need to find someone that know more about Lisp than me to tell you if that's correct.

The only Lisp change is adding the one line (setf asdf::*user-cache* (truename "./lisp-cache")) to asdf (Lisp build system). This was actually written by Nils Bruin in #8645, I just collected the patch. So I'll positively review that line myself, if I may.

  • trac_10187_fix_easy_doctests.patch

1) There are these numerical results. Has anyone independently verified this are correct? [2.6789385347..., -8.3905259853..., 26.662447494..., -80.683148377...] On what basis was the number of significant digits selected?

The preceeding (long) symbolic expression changed in some trivial ways and I added the numerical evaluation to make sure that the old and new symbolic expression are the same.

As a matter of interest, do we know if anyone every verify the original horrid looking symbolic integral is correct?

I'm not the original author. All I know is that it is as correct in the patched doctest as it was before my patch.

2) Why was the doctest:

integral(abs(x), x, 0, a)

changed to:

integral(abs(x)*x, x, 0, a) 

The old maxima could not perform the old integral, but the new maxima version can do it. To keep testing the assume facility and maxima error reporting, I made the integral more difficult.

sage/plot/plot3d/transform.pyx Do we know if that horrible looking result is correct? I see it is a simple change from the previous horrible looking result, but how do we know if the previous one is correct? If, as I suspect, this has not been verified by hand or with another package, then we should write that fact.

The old and new maxima actually yield different expressions because of some branch choice issue with the square roots. I'm not the original author, but I think this doctest only shows that maxima doesn't blow up with horrible expressions. I don't think anyone verified the original version. If you want to go through all maxima doctests and annotate them with a manual computation then that is great, but maybe material for a different ticket.

comment:102 Changed 9 years ago by drkirkby

If you comment the doctest, to indicate that the numerical results have not been verified, explaining why, then I;d consider that OK. But I think to stick a test with numerical results, which have not been verified, needs to be noted.

I would also remark you are not the author of the original test, but don't think the integral has been verified.

If state the reasons for the numerical results in the doctest then

  • I'm happy with everything except the Lisp.
  • You copied the Lisp, but understand Lisp and know its correct.
  • Someone else can verify my ECL package

then I think that is all we need tor a positive review.

I must say, I think it's pretty unimpressive if we use as a test a result we have not verified. I would have thought it more sensible to use an example from a table of integrals from a book, verify it with Mathematica or Maple. But to base a test on something we don't even know is right is a very unfortunate state of affairs.

comment:103 Changed 9 years ago by drkirkby

When I say "verify it with Mathematica or Maple", I'm not saying they are a "gold standard". But if software developed independently of other software gives the same results, it would give increased confidence that the results are indeed correct.

Changed 9 years ago by drkirkby

clean up SPKG.txt and spkg-install on the ECL package

comment:104 follow-up: Changed 9 years ago by drkirkby

I've attached a cleanup patch, addressing most of Leif's comments. See 10187-cleanup.patch

I decided not to restrict the MAKEFLAGS to only OS X, since we don't know for sure it's not needed on other systems, and I don't want to take a chance.

There appears to be no option to specify the location of the garbidge collector - only whether it is a system one or not. This differs from gmp, where you can specify the location too.

The package has been updated. MD5 hecksum = a72f6967736c3d0dd9f69ab34a0cadb7

Dave

comment:105 in reply to: ↑ 104 Changed 9 years ago by leif

Replying to drkirkby:

I decided not to restrict the MAKEFLAGS to only OS X, since we don't know for sure it's not needed on other systems, and I don't want to take a chance.

I know... ;-) (at least if we rely on GNU `make`)

There appears to be no option to specify the location of the garbidge collector - only whether it is a system one or not. This differs from gmp, where you can specify the location too.

Yes, the apparently right one is unfortunately deprecated; I think we should report the Darwin issue upstream if ECL's behavior hasn't changed yet, since it is dumb to assume any "system" version there was broken.

comment:106 follow-up: Changed 9 years ago by leif

Dave, how does changing

FOO="$with_spaces/foo/bar"

to

FOO="${with_spaces}/foo/bar"

solve the problem with spaces?

SPKG.txt contains some unterminated sentence ("Added -with-gmp-prefix="$SAGE_LOCAL" --with-system-gmp=yes --enable-boehm=system to the line invoking the configure script. These..." ?)

Changed 9 years ago by vbraun

Updated patch

comment:107 in reply to: ↑ 106 Changed 9 years ago by drkirkby

Replying to leif:

Dave, how does changing

FOO="$with_spaces/foo/bar"

to

FOO="${with_spaces}/foo/bar"

solve the problem with spaces?

What's wrong with it?

drkirkby@laptop:~$ cat tt
#!/bin/sh
SAGE_LOCAL="/this path has/some spaces in/it"
FULL_PATH="${SAGE_LOCAL}/local/lib"
echo "$FULL_PATH"

drkirkby@laptop:~$ ./tt
/this path has/some spaces in/it/local/lib

As far as I can tell, that basically shows that what we have will permit spaces. If not, please explain why and provide an alternative.

SPKG.txt contains some unterminated sentence ("Added -with-gmp-prefix="$SAGE_LOCAL" --with-system-gmp=yes --enable-boehm=system to the line invoking the configure script. These..." ?)

I was finishing that at about 3 AM local time. I'll sort out that minor issue, and any others you can see, once we agree how to handle the spaces in the path

Dave

comment:108 follow-up: Changed 9 years ago by vbraun

I've added a comment to explain the numeric evaluation of the taylor series in trac_10187_fix_easy_doctests.patch and compared it with Maple 12.

About the spaces in paths: FOO="$with_spaces/foo/bar" and FOO="${with_spaces}/foo/bar" will assign the same value to FOO. You only need the curly brackets if then next character is a valid part of a variable name, i.e. FOO="${with_spaces}bar" works but FOO="$with_spacesbar" is wrong.

comment:109 in reply to: ↑ 108 Changed 9 years ago by drkirkby

Replying to vbraun:

I've added a comment to explain the numeric evaluation of the taylor series in trac_10187_fix_easy_doctests.patch and compared it with Maple 12.

Thank you. I'm ok with that then.

About the spaces in paths: FOO="$with_spaces/foo/bar" and FOO="${with_spaces}/foo/bar" will assign the same value to FOO. You only need the curly brackets if then next character is a valid part of a variable name, i.e. FOO="${with_spaces}bar" works but FOO="$with_spacesbar" is wrong.

So what bit of spkg-install will break with spaces in the SAGE_LOCAL? Leif commented the CPPFLAGS would not handle spaces, but I don't see why not. The braces should just make 100% sure not even a contrived example should break things.

Dave

comment:110 Changed 9 years ago by kcrisman

I just wanted to verify that all seems well, except possibly that the trac_10187_mark_some_doctests_random_until_9880_gets_merged.patch may not apply correctly after the latest change to clarify that the Taylor coefficients are correct. Certainly the #8645 changes are fine. So if that patch still applies (or is updated) and you guys figure out the spaces issue, this should definitely get positive review! Great work. Hopefully it will be merged before the *next* Maxima upgrade :)

comment:111 follow-up: Changed 9 years ago by vbraun

  • Status changed from needs_work to needs_review

I've rediffed trac_10187_mark_some_doctests_random_until_9880_gets_merged.patch to eliminate the fuzz, nothing fancy.

I think all that remains is to give a positive review to ECL; Leif, are you satisfied?

comment:112 in reply to: ↑ 111 Changed 9 years ago by drkirkby

Replying to vbraun:

I think all that remains is to give a positive review to ECL;

I agree.

Leif, are you satisfied?

Leif found a couple of errors in the documentation I put in SPKG.txt. I found two myself, but perhaps Leif found others. If he can list them, along with some suggestions to get around what he perceives as a problem with spaces in filenames, I'll address all issues at once.

Then this should get a positive review I think.

Dave

comment:113 Changed 9 years ago by leif

I had ("in principle"TM) already been ok with Dave's ECL package, then he made further changes.

As said before, I consider the "spaces in SAGE_ROOT" issue a currently rather minor one (as long as our spkg-install doesn't fail in an unexpected manner), since I doubt upstream (at least some other packages) will support that.

However, Dave claimed he had fixed that (with a change that effectively changes nothing ;-) ). While echo $CPPFLAGS will almost always work, running gcc with spaces in the include path won't as is. So if at all, I'd change it to

CPPFLAGS="$CPPFLAGS -I\"$SAGE_LOCAL/include\""
LDFLAGS="$LDFLAGS -L\"$SAGE_LOCAL/lib\""

which at least gcc does support, but not necessarily other parts of configure etc.


There are some typos in SPKG.txt (IMHO minor as well):

  • s/safter/safer/ (plus spurious ":w" ?)
  • s/user defined/user-defined/
  • s/overwritten, as they would have done/overridden, as they would have been/
  • s/it will causes/it will cause/
  • s/Remove/Removed/

But SPKG.txt is not the User or Developer's Guide...


I cannot guarantee (nor test) the new parameters to configure work on all platforms as expected: --with-system-gmp=yes is IMHO (at least) superfluous if we also pass --with-gmp-prefix="$SAGE_LOCAL".


I'd prefer all error messages (in spkg-install) starting with "Error" to ease grepping for such, consistent with other packages.

comment:114 follow-up: Changed 9 years ago by leif

Can someone test if / verify that ECL meanwhile allows us to use Sage's Boehm GC on Darwin as well, rather than the one shipped with ECL?

comment:115 in reply to: ↑ 114 Changed 9 years ago by leif

Replying to leif:

Can someone test if / verify that ECL meanwhile allows us to use Sage's Boehm GC on Darwin as well, rather than the one shipped with ECL?

(If not, I think we should mention this in the Special Update/Build? Instructions, and report this upstream as a bug.)

comment:116 follow-up: Changed 9 years ago by drkirkby

I take your point Leif, you can't check the extra options I added to the configure script. I think they were helpful and safe. Given that, and to stop this ticket dragging on any longer, they are best removed. It's one less thing to worry about.

If I remove those options to the configured script, and implement your changes to CPPFLAGS and LDFLAGS, it actually break the code, as the Boehm header files are no longer found. (I cut and pasted your changes.) As soon as I undo your changes, so the header files are found.

Given this, I've made some more changes

  • Put CPPFLAGS and LDFLAGS back to how they were in the previous ECL package.
  • Removed the options to configure, so they are the same as the previous ECL package
  • Corrected the typos.

If you want to sort out the problem with spaces in the filenames, then edit the ticket and add yourself as an author. But I don't want spend any longer on this.

I've attched another patch, which reverses some changes, and corrects the typos. A new package can be found at the old place. The MD5 checksum is 8f60b1b4453a277d33178683ef532a93 Dave

Changed 9 years ago by drkirkby

Further clean up SPKG.txt and spkg-install on the ECL package. Reverse some previous changes

comment:117 in reply to: ↑ 116 Changed 9 years ago by leif

Replying to drkirkby:

I take your point Leif, you can't check the extra options I added to the configure script. I think they were helpful and safe. Given that, and to stop this ticket dragging on any longer, they are best removed. It's one less thing to worry about.

If I remove those options to the configured script, and implement your changes to CPPFLAGS and LDFLAGS, it actually break the code, as the Boehm header files are no longer found. (I cut and pasted your changes.) As soon as I undo your changes, so the header files are found.

Yes, with

CPPFLAGS="$CPPFLAGS -I\"${SAGE_LOCAL}/include\""
LDFLAGS="$LDFLAGS -L\"${SAGE_LOCAL}/lib\""

and

   ./configure --prefix="$SAGE_LOCAL" --with-gmp-prefix="$SAGE_LOCAL" # --enable-boehm=system

configure behaves weird (and finally the Lisp build fails for me):

...
checking for ld flags when building shared libraries... -shared  -L"/home/leif/Sage/sage-4.6.1.alpha2/local/lib"
...
checking gmp.h usability... yes
checking gmp.h presence... yes
checking for gmp.h... yes
...
checking for GC_malloc in -lgc... yes
checking whether we can use the existing Boehm-Weiser library ... yes
checking for GC_malloc in -lgc... (cached) yes
checking if we need to copy GC private headers ... checking if we use Boehm-Demers-Weiser precise garbage collector... no
...
checking for ffi_call in -lffi... no
/home/leif/Sage/sage-4.6.1.alpha2/spkg/build/ecl-10.4.1/src/src/configure: line
9924: test: =: unary operator expected
checking whether we can dynamically build calls to C functions... yes
...
checking gc.h usability... yes
checking gc.h presence... no
configure: WARNING: gc.h: accepted by the compiler, rejected by the preprocessor
!
configure: WARNING: gc.h: proceeding with the compiler's result
checking for gc.h... yes
configure: creating ./config.status
...
Configuration complete. To build ECL, issue make in this directory.
...
;;; About to load cmp/load.lsp
Too many arguments supplied to a macro or a destructuring-bind form.

No restarts available.

Broken at SI::TPL.
 File: #P"/home/leif/Sage/sage-4.6.1.alpha2/spkg/build/ecl-10.4.1/src/src/lsp/to
p.lsp" (Position #20393)
C>>

I'd like to see such documented in SPKG.txt (Special Update/Build? Instructions) and / or spkg-install (e.g. # *don't* quote SAGE_LOCAL here, since...), to avoid others touching this spkg running into the same problem(s).

Clearly there are bugs in configure, which might get fixed in future releases though.

comment:118 Changed 9 years ago by drkirkby

I will attach yet another patch, documenting the problem as requested. Hopefully this is sufficient to get a positive review. The MD5 checksum is now d162c7fa859b7f894bdd6a6cfa061359

BTW Leif, I'd appreciate your input on sage-devel on the thread When is a test not a valid test? It's clear different Sage developers have very different views on quality control.

Dave

Changed 9 years ago by drkirkby

Document that quoting SAGE_LOCAL can cause problems. Correct a spelling error too.

comment:119 Changed 9 years ago by drkirkby

Since Leif's last comment was he wanted the issue of quoting of SAGE_LOCAL documented in SPKG.txt and spkg-install, and I've now added those comments, I believe this ticket should now get a positive review. I hope Leif and/or others agree!! It would be good to see it merged into 4.6.1, which will hopefully address the Fedora 14 issues.

Dave

comment:120 follow-up: Changed 9 years ago by jdemeyer

Please confirm that the links to the spkgs and patches in the ticket description are up-to-date, then I will do some testing.

comment:121 in reply to: ↑ 120 ; follow-up: Changed 9 years ago by leif

Replying to jdemeyer:

Please confirm that the links to the spkgs and patches in the ticket description are up-to-date, then I will do some testing.

Currently myself doing "final" testing... ;-)

Dave: You can still delete the redundant http://trac.sagemath.org/sage_trac/raw-attachment/ticket/10187/maxima-5.22.1.patch .

comment:122 in reply to: ↑ 121 ; follow-up: Changed 9 years ago by leif

Replying to leif:

Dave: You can still delete the redundant http://trac.sagemath.org/sage_trac/raw-attachment/ticket/10187/maxima-5.22.1.patch .

Ooops, forget that.

Confused it with the (now deleted) redundant patch to Sage scripts.

comment:123 in reply to: ↑ 122 Changed 9 years ago by drkirkby

Replying to leif:

Replying to leif:

Dave: You can still delete the redundant http://trac.sagemath.org/sage_trac/raw-attachment/ticket/10187/maxima-5.22.1.patch .

Ooops, forget that.

Confused it with the (now deleted) redundant patch to Sage scripts.

Ah, we have a problem. I see your message, checked the description which no longer mentions that patch, so I have deleted it. There is no way to retrieve it from trac either.

Dave

comment:124 follow-up: Changed 9 years ago by drkirkby

Leif, since you are now doing the final testing, I assume you have downloaded the patch. Can you put it back.

There's no mention in the description of applying the patch either. Is it in the .spkg, in which case it was only needed for review purposes, and can be retried with

hg export tip

Dave

comment:125 follow-up: Changed 9 years ago by leif

Ok, except for the now obsolete changelog entry

  * Changed method of setting CPPFLAGS and LDFLAGS so they would work with
    a space in $SAGE_LOCAL.

the ECL spkgs looks clean (though there's work for follow-up tickets).

The patch recently deleted by Dave was just the (cumulative) Maxima spkg patch for reference / review, so can easily be restored.

comment:126 in reply to: ↑ 124 Changed 9 years ago by leif

Replying to drkirkby:

There's no mention in the description of applying the patch either. Is it in the .spkg, in which case it was only needed for review purposes, and can be retried with

hg export tip

You'd have to export changesets 20, 21 and 22:

changeset:   22:e8822a6306c4
tag:         tip
user:        Volker Braun <vbraun@stp.dias.ie>
date:        Fri Nov 26 17:46:01 2010 +0100
files:       spkg-install
description:
trac 10187: use MAKE variable


changeset:   21:2f6e663e08f6
user:        Volker Braun <vbraun@stp.dias.ie>
date:        Fri Nov 26 16:12:35 2010 +0100
files:       spkg-install
description:
trac 10187: minor changes


changeset:   20:1b2e265d49d2
user:        Volker Braun <vbraun@stp.dias.ie>
date:        Fri Oct 29 13:40:03 2010 +0100
files:       SPKG.txt spkg-install
description:
updated to latest upstream; rewritten build for maxima.fas (see spgk-install)

Changed 9 years ago by vbraun

Re-uploaded diff for the maxima spkg (for review purposes only)

comment:127 Changed 9 years ago by vbraun

I have re-uploaded the diff for the maxima spkg. It is only for historical purposes right now as we have already reviewed the new maxima spkg.

comment:128 Changed 9 years ago by leif

  • Description modified (diff)

Minor correction, description is (and was) up-to-date.

comment:129 Changed 9 years ago by drkirkby

Thank you for uploading the Maxima patch again. Perhaps you could explain how one would export one patch containing those 3 changesets, as I don't know how to do that, but have often felt it would be useful.

OK, hopefully after Leif's FinalTM testing, this is ready to go, with all the correct library patches and .spkg files now in place.

At least that's my understanding of the situation. Can anyone else confirm this?

Dave

comment:130 in reply to: ↑ 125 ; follow-up: Changed 9 years ago by drkirkby

Replying to leif:

Ok, except for the now obsolete changelog entry

  * Changed method of setting CPPFLAGS and LDFLAGS so they would work with
    a space in $SAGE_LOCAL.

the ECL spkgs looks clean (though there's work for follow-up tickets).

I'll address that. Give me 10 minutes and I'll upload a new .spkg with that entry in SPKG.txt removed.

comment:131 in reply to: ↑ 130 Changed 9 years ago by leif

Replying to drkirkby:

Replying to leif: I'll address that. Give me 10 minutes and I'll upload a new .spkg with that entry in SPKG.txt removed.

I thought we were in "feature" freeze. ;-)

As mentioned before, positive review from me for both spkgs (and the scripts patch, haven't really looked at the others).

Final test on a slow machine will take hours; I think Jeroen is also going to test on the farm.

I'll report back then.

comment:132 Changed 9 years ago by drkirkby

A new ECL package, with an MD5 cheskcum of 7c612a32fef87878cf686f9c35858e15 has been uploaded. That has the inaccurate comment removed. I'll attach a patch in a minute or two, but there's no need for Jeroen to wait - he can test this now, as the package is fully upto date.

Dave

comment:133 Changed 9 years ago by drkirkby

Oops, forget my comment. I failed to cmmmit the change.

Give me a bit longer!!!

Dave

comment:134 Changed 9 years ago by drkirkby

  • Owner changed from tbd to drkirkby

OK, new package with a checksum of 452b5481d6e9dfd47d276cda89cebe7d is now in place. Patch will follow, but it only removes two lines from SPKG.txt

Dave

comment:135 Changed 9 years ago by vbraun

For the record, here is how to generate a patch for a range of changesets:

hg diff -r 19:tip > maxima-5.22.1.patch

Applying the resulting patch is equivalent to applying the output of

hg export -r 19:tip

but the hg diff patch is not broken down into the individual changesets.

Changed 9 years ago by drkirkby

Removes an inaccurate comment from SPKG.txt

comment:136 Changed 9 years ago by drkirkby

All ready to be tested now.

Dave

comment:137 Changed 9 years ago by leif

Just for the record:

I do get doctest errors with Sage 4.6.1.alpha2 (+ #8664 + #5847) running ptestlong on Ubuntu 9.04 x86 (Pentium 4); have to analyze later...

comment:138 follow-up: Changed 9 years ago by leif

I hope my installation is somehow broken:

The following tests failed:

	sage -t  -long -force_lib devel/sage/doc/en/a_tour_of_sage/index.rst # 3 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/bordeaux_2008/nf_introduction.rst # 8 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/constructions/interface_issues.rst # 4 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/constructions/linear_algebra.rst # 5 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/constructions/polynomials.rst # 1 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/constructions/calculus.rst # 31 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/constructions/plotting.rst # 3 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/installation/source.rst # 1 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/tutorial/interfaces.rst # 18 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/tutorial/latex.rst # 3 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/tutorial/tour_rings.rst # 2 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/tutorial/tour_algebra.rst # 21 doctests failed
	sage -t  -long -force_lib devel/sage/doc/fr/tutorial/interfaces.rst # 18 doctests failed
	sage -t  -long -force_lib devel/sage/doc/fr/a_tour_of_sage/index.rst # 3 doctests failed
	sage -t  -long -force_lib devel/sage/doc/fr/tutorial/tour_algebra.rst # 20 doctests failed
	sage -t  -long -force_lib devel/sage/sage/combinat/perfect_matching.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/combinat/combinat.py # 10 doctests failed
	sage -t  -long -force_lib devel/sage/sage/combinat/words/word_generators.py # 11 doctests failed
	sage -t  -long -force_lib devel/sage/sage/geometry/lattice_polytope.py # 3 doctests failed
	sage -t  -long -force_lib devel/sage/sage/geometry/polyhedra.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/schemes/elliptic_curves/constructor.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/schemes/elliptic_curves/heegner.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/schemes/elliptic_curves/ell_generic.py # 16 doctests failed
	sage -t  -long -force_lib devel/sage/sage/misc/latex.py # 3 doctests failed
	sage -t  -long -force_lib devel/sage/sage/misc/preparser.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/misc/functional.py # 34 doctests failed
	sage -t  -long -force_lib devel/sage/sage/misc/citation.pyx # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/graphs/graph_generators.py # 6 doctests failed
	sage -t  -long -force_lib devel/sage/sage/gsl/integration.pyx # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/modules/free_module_element.pyx # 39 doctests failed
	sage -t  -long -force_lib devel/sage/sage/matrix/matrix1.pyx # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/matrix/matrix_sparse.pyx # 6 doctests failed
	sage -t  -long -force_lib devel/sage/sage/matrix/matrix2.pyx # 7 doctests failed
	sage -t  -long -force_lib devel/sage/sage/matrix/matrix_symbolic_dense.pyx # 63 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/relation.py # 72 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/power_helper.pyx # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/pynac.pyx # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/function.pyx # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/function_factory.py # 4 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/ring.pyx # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/random_tests.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/constants_c.pyx # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/maxima_wrapper.py # 8 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/expression_conversions.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/constants.py # 16 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/assumptions.py # 51 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/expression.pyx # 255 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/integration/external.py # 3 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/integration/integral.py # 64 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/infinity.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/fraction_field_element.pyx # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/ring.pyx # 5 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/residue_field.pyx # 15 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/misc.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/contfrac.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/integer_ring.pyx # 5 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/polynomial/multi_polynomial.pyx # 4 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/polynomial/polynomial_element.pyx # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/polynomial/polynomial_rational_flint.pyx # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/number_field/order.py # 3 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/number_field/galois_group.py # 8 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/number_field/number_field_element_quadratic.pyx # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/number_field/number_field_element.pyx # 12 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/number_field/number_field_base.pyx # 13 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/number_field/number_field_rel.py # 10 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/functions.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/calculus.py # 127 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/desolvers.py # 70 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/tests.py # 27 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/functional.py # 45 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/test_sympy.py # 7 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/wester.py # 49 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/special.py # 69 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/log.py # 8 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/wigner.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/min_max.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/hyperbolic.py # 9 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/transcendental.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/piecewise.py # 65 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/orthogonal_polys.py # 42 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/other.py # 19 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/trig.py # 17 doctests failed
	sage -t  -long -force_lib devel/sage/sage/plot/line.py # 3 doctests failed
	sage -t  -long -force_lib devel/sage/sage/plot/plot.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/plot/arc.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/plot/plot3d/plot3d.py # 3 doctests failed
	sage -t  -long -force_lib devel/sage/sage/plot/plot3d/transform.pyx # 4 doctests failed
	sage -t  -long -force_lib devel/sage/sage/structure/parent.pyx # 5 doctests failed
	sage -t  -long -force_lib devel/sage/sage/structure/element.pyx # 5 doctests failed
	sage -t  -long -force_lib devel/sage/sage/structure/formal_sum.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/interfaces/expect.py # 26 doctests failed
	sage -t  -long -force_lib devel/sage/sage/interfaces/tests.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/interfaces/quit.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/interfaces/maxima.py # 218 doctests failed
	sage -t  -long -force_lib devel/sage/sage/algebras/group_algebra.py # 6 doctests failed
	sage -t  -long -force_lib devel/sage/sage/algebras/quaternion_algebra_element.py # 4 doctests failed
	sage -t  -long -force_lib devel/sage/sage/algebras/quatalg/quaternion_algebra_element.pyx # 36 doctests failed
	sage -t  -long -force_lib devel/sage/sage/algebras/quatalg/quaternion_algebra.py # 8 doctests failed
	sage -t  -long -force_lib devel/sage/sage/modular/overconvergent/genus0.py # 19 doctests failed
	sage -t  -long -force_lib devel/sage/sage/modular/modform/ambient_R.py # 5 doctests failed
	sage -t  -long -force_lib devel/sage/sage/tensor/differential_forms.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/tensor/differential_form_element.py # 43 doctests failed
----------------------------------------------------------------------
Total time for all tests: 13327.9 seconds
make: *** [ptestlong] Error 128

Going to retry, which again will take hours...

comment:139 Changed 9 years ago by leif

Yes, make reverted both ECL and Maxima to the previous versions.

Another ptestlong in progress...

comment:140 follow-up: Changed 9 years ago by drkirkby

Am I right in assuming all these patches are against sage-4.6.1.alpha1 ?

I just built sage-4.6.1.alpha2, but some of these patches are applied, and some are not. So it looks like I'm going to have to build sage-4.6.1.alpha1. I expect it will take me a couple of hours to build it, patch it, then doctest it

Dave

comment:141 in reply to: ↑ 140 Changed 9 years ago by leif

Replying to drkirkby:

Am I right in assuming all these patches are against sage-4.6.1.alpha1 ?

I just built sage-4.6.1.alpha2, but some of these patches are applied, and some are not. So it looks like I'm going to have to build sage-4.6.1.alpha1. I expect it will take me a couple of hours to build it, patch it, then doctest it

Shouldn't be necessary.

I've previously tested the spkgs and patches successfully (ptestlong) multiple times with Sage 4.6.1.alpha2 (in addition with the new MPIR and ECM, #8664 and #5847, respectively) on Ubuntu 10.04 x86_64.

On the 32-bit machine, I so far again got 47 doctest errors with the same though.

comment:142 follow-up: Changed 9 years ago by leif

Hmmm, maybe a Mercurial issue?

Patches apply apparently clean on vanilla alpha2 as well (with an old Mercurial version), but I also get doctest errors on Ubuntu 9.04 x86_64.

comment:143 in reply to: ↑ 142 Changed 9 years ago by leif

Replying to leif:

Hmmm, maybe a Mercurial issue?

Patches apply apparently clean on vanilla alpha2 as well (with an old Mercurial version), but I also get doctest errors on Ubuntu 9.04 x86_64.

Same with Sage's Mercurial version (1.6.x), i.e. patches apply cleanly.

Now at 199+ doctest errors on the 32-bit machine... :(

comment:144 Changed 9 years ago by leif

Ubuntu 9.04 x86_64 (gcc 4.3.3), vanilla Sage 4.6.1.alpha2 with #10187:

The following tests failed:

	sage -t  -long -force_lib devel/sage/doc/en/installation/source.rst # 1 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/a_tour_of_sage/index.rst # 3 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/constructions/polynomials.rst # 1 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/constructions/linear_algebra.rst # 5 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/constructions/calculus.rst # 31 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/constructions/interface_issues.rst # 4 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/constructions/plotting.rst # 3 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/tutorial/interfaces.rst # 18 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/tutorial/latex.rst # 3 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/tutorial/tour_algebra.rst # 21 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/bordeaux_2008/nf_introduction.rst # 8 doctests failed
	sage -t  -long -force_lib devel/sage/doc/fr/a_tour_of_sage/index.rst # 3 doctests failed
	sage -t  -long -force_lib devel/sage/doc/fr/tutorial/interfaces.rst # 18 doctests failed
	sage -t  -long -force_lib devel/sage/doc/fr/tutorial/tour_algebra.rst # 20 doctests failed
	sage -t  -long -force_lib devel/sage/sage/matrix/matrix2.pyx # 5 doctests failed
	sage -t  -long -force_lib devel/sage/sage/matrix/matrix1.pyx # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/matrix/matrix_symbolic_dense.pyx # 40 doctests failed
	sage -t  -long -force_lib devel/sage/sage/combinat/combinat.py # 10 doctests failed
	sage -t  -long -force_lib devel/sage/sage/combinat/perfect_matching.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/structure/element.pyx # 5 doctests failed
	sage -t  -long -force_lib devel/sage/sage/structure/formal_sum.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/expression_conversions.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/ring.pyx # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/function.pyx # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/random_tests.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/maxima_wrapper.py # 8 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/pynac.pyx # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/relation.py # 66 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/constants.py # 16 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/assumptions.py # 51 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/power_helper.pyx # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/function_factory.py # 4 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/constants_c.pyx # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/integration/external.py # 3 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/integration/integral.py # 64 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/expression.pyx # 247 doctests failed
	sage -t  -long -force_lib devel/sage/sage/algebras/quaternion_algebra_element.py # 4 doctests failed
	sage -t  -long -force_lib devel/sage/sage/algebras/group_algebra.py # 6 doctests failed
	sage -t  -long -force_lib devel/sage/sage/algebras/quatalg/quaternion_algebra_element.pyx # 36 doctests failed
	sage -t  -long -force_lib devel/sage/sage/algebras/quatalg/quaternion_algebra.py # 8 doctests failed
	sage -t  -long -force_lib devel/sage/sage/geometry/lattice_polytope.py # 3 doctests failed
	sage -t  -long -force_lib devel/sage/sage/interfaces/quit.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/geometry/polyhedra.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/interfaces/tests.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/interfaces/maxima.py # 224 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/misc.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/ring.pyx # 5 doctests failed
	sage -t  -long -force_lib devel/sage/sage/interfaces/expect.py # 26 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/infinity.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/fraction_field_element.pyx # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/contfrac.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/residue_field.pyx # 15 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/integer_ring.pyx # 5 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/number_field/number_field_element_quadratic.pyx # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/number_field/galois_group.py # 8 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/number_field/number_field_base.pyx # 13 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/number_field/order.py # 3 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/number_field/number_field_element.pyx # 12 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/number_field/number_field_rel.py # 10 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/polynomial/polynomial_element.pyx # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/schemes/elliptic_curves/heegner.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/schemes/elliptic_curves/ell_generic.py # 15 doctests failed
	sage -t  -long -force_lib devel/sage/sage/plot/arc.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/plot/line.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/plot/plot.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/plot/plot3d/plot3d.py # 3 doctests failed
	sage -t  -long -force_lib devel/sage/sage/plot/plot3d/transform.pyx # 4 doctests failed
	sage -t  -long -force_lib devel/sage/sage/misc/citation.pyx # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/misc/functional.py # 29 doctests failed
	sage -t  -long -force_lib devel/sage/sage/misc/latex.py # 3 doctests failed
	sage -t  -long -force_lib devel/sage/sage/misc/preparser.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/modular/modform/ambient_R.py # 5 doctests failed
	sage -t  -long -force_lib devel/sage/sage/modular/overconvergent/genus0.py # 19 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/trig.py # 17 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/other.py # 19 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/piecewise.py # 65 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/special.py # 69 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/transcendental.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/log.py # 8 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/min_max.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/wigner.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/hyperbolic.py # 9 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/orthogonal_polys.py # 42 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/functional.py # 45 doctests failed
	sage -t  -long -force_lib devel/sage/sage/graphs/graph_generators.py # 6 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/calculus.py # 127 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/test_sympy.py # 7 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/functions.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/desolvers.py # 70 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/tests.py # 24 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/wester.py # 45 doctests failed
	sage -t  -long -force_lib devel/sage/sage/gsl/integration.pyx # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/modules/free_module_element.pyx # 8 doctests failed
----------------------------------------------------------------------
Total time for all tests: 2000.6 seconds
make: *** [ptestlong] Error 128

Currently no idea what happens there...

comment:145 in reply to: ↑ 138 Changed 9 years ago by leif

Replying to leif:

Going to retry, which again will take hours...

Now I get (Ubuntu 9.04 x86, gcc 4.3.3):

The following tests failed:

	sage -t  -long -force_lib devel/sage/doc/en/a_tour_of_sage/index.rst # 3 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/bordeaux_2008/nf_introduction.rst # 8 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/constructions/interface_issues.rst # 4 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/constructions/linear_algebra.rst # 5 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/constructions/polynomials.rst # 1 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/constructions/calculus.rst # 31 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/constructions/plotting.rst # 3 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/installation/source.rst # 1 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/tutorial/interfaces.rst # 18 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/tutorial/latex.rst # 3 doctests failed
	sage -t  -long -force_lib devel/sage/doc/en/tutorial/tour_algebra.rst # 21 doctests failed
	sage -t  -long -force_lib devel/sage/doc/fr/tutorial/interfaces.rst # 18 doctests failed
	sage -t  -long -force_lib devel/sage/doc/fr/a_tour_of_sage/index.rst # 3 doctests failed
	sage -t  -long -force_lib devel/sage/doc/fr/tutorial/tour_algebra.rst # 20 doctests failed
	sage -t  -long -force_lib devel/sage/sage/combinat/perfect_matching.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/combinat/combinat.py # 10 doctests failed
	sage -t  -long -force_lib devel/sage/sage/geometry/lattice_polytope.py # 3 doctests failed
	sage -t  -long -force_lib devel/sage/sage/geometry/polyhedra.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/schemes/elliptic_curves/heegner.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/schemes/elliptic_curves/ell_generic.py # 15 doctests failed
	sage -t  -long -force_lib devel/sage/sage/misc/latex.py # 3 doctests failed
	sage -t  -long -force_lib devel/sage/sage/misc/preparser.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/misc/functional.py # 29 doctests failed
	sage -t  -long -force_lib devel/sage/sage/misc/citation.pyx # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/graphs/graph_generators.py # 6 doctests failed
	sage -t  -long -force_lib devel/sage/sage/gsl/integration.pyx # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/modules/free_module_element.pyx # 8 doctests failed
	sage -t  -long -force_lib devel/sage/sage/matrix/matrix1.pyx # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/matrix/matrix2.pyx # 5 doctests failed
	sage -t  -long -force_lib devel/sage/sage/matrix/matrix_symbolic_dense.pyx # 40 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/power_helper.pyx # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/relation.py # 66 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/pynac.pyx # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/function.pyx # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/function_factory.py # 4 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/ring.pyx # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/random_tests.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/constants_c.pyx # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/maxima_wrapper.py # 8 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/expression_conversions.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/constants.py # 16 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/assumptions.py # 51 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/expression.pyx # 247 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/integration/integral.py # 64 doctests failed
	sage -t  -long -force_lib devel/sage/sage/symbolic/integration/external.py # 3 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/infinity.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/fraction_field_element.pyx # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/ring.pyx # 5 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/residue_field.pyx # 15 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/misc.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/contfrac.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/integer_ring.pyx # 5 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/polynomial/polynomial_element.pyx # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/number_field/order.py # 3 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/number_field/galois_group.py # 8 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/number_field/number_field_element_quadratic.pyx # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/number_field/number_field_element.pyx # 12 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/number_field/number_field_base.pyx # 13 doctests failed
	sage -t  -long -force_lib devel/sage/sage/rings/number_field/number_field_rel.py # 10 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/functions.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/calculus.py # 127 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/desolvers.py # 70 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/tests.py # 24 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/functional.py # 45 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/test_sympy.py # 7 doctests failed
	sage -t  -long -force_lib devel/sage/sage/calculus/wester.py # 45 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/special.py # 69 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/log.py # 8 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/wigner.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/min_max.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/hyperbolic.py # 9 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/transcendental.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/piecewise.py # 65 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/orthogonal_polys.py # 42 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/other.py # 19 doctests failed
	sage -t  -long -force_lib devel/sage/sage/functions/trig.py # 17 doctests failed
	sage -t  -long -force_lib devel/sage/sage/plot/line.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/plot/plot.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/plot/arc.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/plot/plot3d/plot3d.py # 3 doctests failed
	sage -t  -long -force_lib devel/sage/sage/plot/plot3d/transform.pyx # 4 doctests failed
	sage -t  -long -force_lib devel/sage/sage/structure/element.pyx # 5 doctests failed
	sage -t  -long -force_lib devel/sage/sage/structure/formal_sum.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/interfaces/expect.py # 26 doctests failed
	sage -t  -long -force_lib devel/sage/sage/interfaces/tests.py # 1 doctests failed
	sage -t  -long -force_lib devel/sage/sage/interfaces/maxima.py # 224 doctests failed
	sage -t  -long -force_lib devel/sage/sage/interfaces/quit.py # 2 doctests failed
	sage -t  -long -force_lib devel/sage/sage/algebras/group_algebra.py # 6 doctests failed
	sage -t  -long -force_lib devel/sage/sage/algebras/quaternion_algebra_element.py # 4 doctests failed
	sage -t  -long -force_lib devel/sage/sage/algebras/quatalg/quaternion_algebra_element.pyx # 36 doctests failed
	sage -t  -long -force_lib devel/sage/sage/algebras/quatalg/quaternion_algebra.py # 8 doctests failed
	sage -t  -long -force_lib devel/sage/sage/modular/overconvergent/genus0.py # 19 doctests failed
	sage -t  -long -force_lib devel/sage/sage/modular/modform/ambient_R.py # 5 doctests failed
----------------------------------------------------------------------
Total time for all tests: 13357.2 seconds

which at first glance doesn't look much different; 525 doctest errors.

comment:146 Changed 9 years ago by leif

(Note that on all systems vanilla Sage 4.6.1.alpha2 passes all tests.)

comment:147 Changed 9 years ago by vbraun

If you would tell us what the error is then that would be much more useful than a list of failed tests. All of these work fine on Fedora 14 with sage-4.6.1.alpha2 and the new maxima/ecl. My first guess would be that you forgot to apply one of the patches (been there, done that), can you double-check?

comment:148 Changed 9 years ago by leif

Well, I didn't attach the logs since they're huge.

I've reverted all changes and started from scratch, and now all tests pass on the Ubuntu 9.04 x86_64 machine. I have absolutely no idea what went wrong in the first two attempts; in the succeeding one I used Sage's hg to apply the patches, after installing the spkgs, and afterwards ran ./sage -ba-force (rather than just ./sage -b), which IMHO shouldn't be necessary.

I did apply all patches in the right order in all cases, so something else must be wrong. Never experienced similar before...

comment:149 follow-up: Changed 9 years ago by drkirkby

The patches on this ticket are not so easy to apply as usual, as part of them is already in Sage 4.6.1.alpha2. What we don't have on this ticket are changes relative to 4.6.1.alpha2.

I've built sage 4.6.1.alpha1 with the patches applied and the new .spkg files. This is on a Sun Ultra 27, 3.33 GHz Xeon processor running OpenSolaris 06/2009.

----------------------------------------------------------------------
All tests passed!
Total time for all tests: 1560.4 seconds
drkirkby@hawk:~/new/sage-4.6.1.alpha1$ 

I wonder if Leif's problems had anything to the patch process going wrong?

Dave

comment:150 in reply to: ↑ 149 ; follow-ups: Changed 9 years ago by jdemeyer

Replying to drkirkby:

The patches on this ticket are not so easy to apply as usual, as part of them is already in Sage 4.6.1.alpha2.

Really? If that's true, it was unintentional. Which patches are in sage-4.6.1.alpha2?

comment:151 in reply to: ↑ 150 ; follow-ups: Changed 9 years ago by leif

Replying to jdemeyer:

Replying to drkirkby:

The patches on this ticket are not so easy to apply as usual, as part of them is already in Sage 4.6.1.alpha2.

Really? If that's true, it was unintentional. Which patches are in sage-4.6.1.alpha2?

I don't think so. I've applied all patches multiple times with -v and different Mercurial versions; no rejects, no moved hunks.


One problem I ran into seems to be that when running make with SAGE_UPGRADING=yes to (safely) install spkgs the Sage library install completely ignores the current branch, and switches back to sage-main, such that previously applied patches "vanish".

Another odd thing I discovered is that at least on Ubuntu 9.04, gcc run from Sage picks up Sage's MPFR (and GMP/MPIR).

comment:152 in reply to: ↑ 151 Changed 9 years ago by leif

Replying to leif:

One problem I ran into seems to be that when running make with SAGE_UPGRADING=yes to (safely) install spkgs the Sage library install completely ignores the current branch, and switches back to sage-main, such that previously applied patches "vanish".

... which means one has to apply patches to the Sage library after all spkgs have been installed that way.

Also, there seems to be a missing dependency for libcsage, i.e. SCons doesn't always rebuild it when necessary, so one has to run ./sage -ba-force.

comment:153 in reply to: ↑ 150 Changed 9 years ago by drkirkby

  • Status changed from needs_review to positive_review

Replying to jdemeyer:

Replying to drkirkby:

The patches on this ticket are not so easy to apply as usual, as part of them is already in Sage 4.6.1.alpha2.

Really? If that's true, it was unintentional. Which patches are in sage-4.6.1.alpha2?

I guess I must have been mistaken - perhaps my alpha2 was not "clean", though I thought I'd extracted a fresh tarball.

Anyway, several others have got all passes, and I got all passes if I apply this to sage-4.6.1.alpha1.

I would say I'd try again with sage-4.6.1.alpha2, but it is late here, and I'm not going to start a build now, as you wont get the results until I wake in the morning. I'm pretty convinced this is OK. To my knowledge, the only thing was some minor changes Leif wanted to the ECL package, and those have been addressed. Leif seemed happy, so I'm setting this to positive.

comment:154 in reply to: ↑ 151 ; follow-up: Changed 9 years ago by drkirkby

Replying to leif:

Another odd thing I discovered is that at least on Ubuntu 9.04, gcc run from Sage picks up Sage's MPFR (and GMP/MPIR).

I'm not surprised by that, as LD_LIBRARY_PATH gets $SAGE_LOCAL/lib prepended to it when sage is run..

When building gcc, the configure options like --with-gmp=/path/to/gmp only tell the first stage build of the compiler where to find the libraries. They do not hard-code the path of the libraries, though there are ways of doing that if you wish when building gcc. If your not careful, it is easy to build gcc where the first stage of the build works, as it finds the GMP and MPFR libraries, but the second stage fails. That's hit many people on Solaris systems, a they often will not have suitable versions of the libraries on the system.

Dave

Dave

comment:155 Changed 9 years ago by drkirkby

  • Description modified (diff)
  • Summary changed from Update ecl and maxima to Update ECL to 10.4.1 and Maxima to 5.22.1 - currently the latest releases.

I'm just changing the ticket title, to be more explicit.

comment:156 in reply to: ↑ 154 Changed 9 years ago by leif

Replying to drkirkby:

Replying to leif:

Another odd thing I discovered is that at least on Ubuntu 9.04, gcc run from Sage picks up Sage's MPFR (and GMP/MPIR).

I'm not surprised by that, as LD_LIBRARY_PATH gets $SAGE_LOCAL/lib prepended to it when sage is run..

Yes, but I would have expected the Debian or Ubuntu developers to set the RPATH.

Not many people like setting LD_LIBRARY_PATH (which is empty outside Sage btw.).

comment:157 Changed 9 years ago by leif

Finally passed all tests (ptestlong) with Sage 4.6.1.alpha2 on

  • Ubuntu 9.04 x86 (gcc 4.3.3),
  • Ubuntu 9.04 x86_64 (gcc 4.3.3), with and without the new MPIR and GMP-ECM (#8664 / #5847),
  • Ubuntu 10.04 x86_64 (gcc 4.4.3) with the new MPIR and GMP-ECM.

Sorry for the noise.

comment:158 Changed 9 years ago by jdemeyer

  • Merged in set to sage-4.6.1.alpha3
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:159 Changed 9 years ago by jason

I just wanted to say "Bravo!" to everyone that has diligently worked on this. Thanks for your work!

comment:160 Changed 9 years ago by mvngu

  • Description modified (diff)
Note: See TracTickets for help on using tickets.