Update scipy to 0.16.1

[jpflori]

Upgrade to scipy 0.16.1 this is intended to be done at the same time as numpy 1.10.1 and matplotlib 1.5.0 as noted in the ticket dependencies.

​http://www.lmona.de/files/sage/scipy-0.16.1.tar.gz

[fbissey]

Didn't know it was out. Not in the Gentoo tree yet so I have no idea so far.

[fbissey]

scipy 0.16.0 is out, one doctest broken:

sage -t --long usr/share/doc/sage/en/faq/faq-usage.rst
Running doctests with ID 2015-08-05-11-07-08-20037f0b.
Using --optional=optional,sage
Doctesting 1 file.
sage -t --long --warn-long 64.8 usr/share/doc/sage/en/faq/faq-usage.rst
**********************************************************************
File "usr/share/doc/sage/en/faq/faq-usage.rst", line 351, in faq-usage
Failed example:
    stats.ttest_ind(list([1,2,3,4,5]),list([2,3,4,5,.6]))
Expected:
    (array(0.07675295564533369), 0.94070490247380478)
Got:
    Ttest_indResult(statistic=array(0.07675295564533369), pvalue=0.94070490247380478)
**********************************************************************
1 item had failures:
   1 of  36 in faq-usage
    [24 tests, 1 failure, 0.68 s]
----------------------------------------------------------------------
sage -t --long --warn-long 64.8 usr/share/doc/sage/en/faq/faq-usage.rst  # 1 doctest failed
----------------------------------------------------------------------

this is on top of 6.9.beta0.

[fbissey]

New commits:

​05c84c1	upgrade to numpy 1.10.1, updating the package including a few cherrypicked patch from master
​de8f740	Merge branch 'develop' into numpy-1.10
​ebdf899	Fix to get rid of the problem with result_type. Pending upstream approval or suggestion of an alternative.
​d12591f	Upgarde matplotlib to 1.5.0
​f3ff139	Fix the doctest warning by adding it to the expected out. I am expecting the warning to originate from matplotlib and to be

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​31669f2

Correct further warnings

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​b21be98

Removing line numbers from doctest

[fbissey]

New branch compatible with the new push at #19556 coming soon.

[fbissey]

New commits:

​7177516	Upgrade matplotlib to 1.5.0
​7ce23d0	Fix deprecation warnings in doctests.
​c7ac0a7	Updated numpy patch according to feedback from upstream
​52a45f4	Merge branch 'numpy-1.10' into mpl-1.5.0
​a733692	remove test for a limitation that doesn't exist anymore.
​6d59bad	Upgrade to scipy 0.16.1 and fix affected doctest

[vdelecroix]

Just a confirmation that with the three tickets applied all tests pass (debian 64bits).

Could you replace

if np.array_equal(vert1, tpath.vertices) and np.array_equal(code1, tpath.codes):
     return True
else:
     return False

with

return np.array_equal(vert1, tpath.vertices) and np.array_equal(code1, tpath.codes)

Or there is a reason to let it as you did?

[fbissey]

No reason I didn't do that. I was so focused in finding a replacement for that particular line (it took some effort for me) that I didn't realise the code could be streamlined. I will make the change your suggest in a few hours. Would you otherwise be happy to review positively the tickets?

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​04997f6	Simplify flow in arrow.py
​8b383de	Merge branch 'mpl-1.5.0' into scipy0.16.1

[fbissey]

Replying to fbissey:

I will make the change your suggest in a few hours. Would you otherwise be happy to review positively the tickets?

All done.

[vbraun]

Fails on arm

gfortran: error: unrecognized command line option '-msse2'
error: Command "gfortran -Wall -g -ffixed-form -fno-second-underscore -fPIC -O2 -march=x86-64 -DMS_WIN64 -mtune=generic -msse2 -I/home/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/numpy/core/include -c -c scipy/fftpack/src/dfftpack/dfftf1.f -o build/temp.linux-armv7l-2.7/scipy/fftpack/src/dfftpack/dfftf1.o" failed with exit status 1

​http://build.sagedev.org/release/builders/%20%20slow%20Oxford%20ARM%20%28Ubuntu%2012.10%29%20incremental/builds/327/steps/compile/logs/scipy

[fbissey]

Serious fail from distutils

customize Gnu95FCompiler
customize Gnu95FCompiler using build_clib
building 'dfftpack' library
compiling Fortran sources
Fortran f77 compiler: gfortran -Wall -g -ffixed-form -fno-second-underscore -fPIC -O2 -march=x86-64 -DMS_WIN64 -mtune=generic -msse2
Fortran f90 compiler: gfortran -Wall -g -fno-second-underscore -fPIC -O2 -march=x86-64 -DMS_WIN64 -mtune=generic -msse2
Fortran fix compiler: gfortran -Wall -g -ffixed-form -fno-second-underscore -Wall -g -fno-second-underscore -fPIC -O2 -march=x86-64 -DMS_WIN64 -mtune=generic -msse2

arm being mistaken for windows64?

[fbissey]

Looking around distros that provide arm support I am wondering how it was building on arm before. I think none of fedora, debian or gentoo have trouble with arm but they all pass config_fc --noarch to setup.py which was done at some point in sage but has been removed a while ago.

I note from ​https://pkgs.fedoraproject.org/cgit/numpy.git/tree/numpy.spec

* Wed Oct 14 2015 Thomas Spura <tomspur@fedoraproject.org> - 1:1.10.1-3
- Remove fortran flags or arm would build with -march=x86-64

I am guessing from this that the bug is recent but because everyone but sage passes --noarch it did go unnoticed in scipy. I'll add it and put this back to need_review.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​d40c0e9

Adding "config_fc --noarch" to setup.py call so we don't get miscompilation on ARM.

[vbraun]

Can you report it somewhere upstream?

[fbissey]

I'll look into it. I am guessing it is technically a bug in numpy's distutils rather than scipy but I have no idea how to fix it or how it happens.

[fbissey]

It looks like someone already acted in upstream numpy, I guess an option is to try to backport the commit in the numpy we ship. ​https://github.com/numpy/numpy/commit/a08437e12d6b6796f4b0beda49bce9adc37db63d

[jpflori]

I think I might have removed the no_arch stuff when I began to clean up the spkg mess. Backporting the fix looks a good idea for a further ticket. Or we just wait for a numpy version bump.

[fbissey]

Replying to jpflori:

Backporting the fix looks a good idea for a further ticket. Or we just wait for a numpy version bump.

The danger with those two scenario is that the removal will be forgotten. However considering that everyone else has it with some variations (Gentoo has noopt and fedora has fcompiler=gunf95 [or something]) I guess it doesn't hurt to keep it.

[fbissey]

I must say I spent a bit of my night worrying that it was the wrong fix. Good that it really works.

[vbraun]

Argh still doesn't work... ​http://build.sagedev.org/release/builders/%20%20slow%20Oxford%20ARM%20%28Ubuntu%2012.10%29%20incremental/builds/330/steps/compile/logs/scipy

---

Last 10 new commits:

​ebdf899	Fix to get rid of the problem with result_type. Pending upstream approval or suggestion of an alternative.
​7177516	Upgrade matplotlib to 1.5.0
​7ce23d0	Fix deprecation warnings in doctests.
​c7ac0a7	Updated numpy patch according to feedback from upstream
​52a45f4	Merge branch 'numpy-1.10' into mpl-1.5.0
​a733692	remove test for a limitation that doesn't exist anymore.
​6d59bad	Upgrade to scipy 0.16.1 and fix affected doctest
​04997f6	Simplify flow in arrow.py
​8b383de	Merge branch 'mpl-1.5.0' into scipy0.16.1
​d40c0e9	Adding "config_fc --noarch" to setup.py call so we don't get miscompilation on ARM.

[fbissey]

OK so I was right to worry it was the wrong fix. I'll remove the last commit on the scipy branch and add the upstream commit in numpy. I guess if I had gone for noopt instead of noarch it would have worked.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​93a8c3d

Revert "Adding "config_fc --noarch" to setup.py call so we don't get miscompilation on ARM."

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​9c6e94c	Add upstream patch to reverse inclusion of mingwin64 support. It cause ARM to be taken for win64!
​414fccf	Merge branch 'numpy-1.10' into mpl-1.5.0
​cc6accc	Merge branch 'mpl-1.5.0' into scipy0.16.1