Numerical noise on OS X 10.7

[jdemeyer]

When running Sage on OS X 10.7, there is numerical noise in some doctests.

[jdemeyer]

How to deal with this:

sage -t  -force_lib devel/sage/sage/matrix/matrix2.pyx
**********************************************************************
File "/Users/jdemeyer/sage-5.0.beta1-gcc/devel/sage-main/sage/matrix/matrix2.pyx", line 7983:
    sage: G.round(6)
Expected:
    [-0.422243 - 0.490087*I  0.566698 - 0.097416*I -0.500882 + 0.002251*I]
    [-0.057002 - 0.495035*I  -0.35059 - 0.625323*I  0.255514 - 0.415284*I]
    [ 0.394105 - 0.421778*I -0.392266 - 0.039345*I  -0.352905 + 0.62195*I]
Got:
    [ 0.422243 + 0.490087*I -0.566698 + 0.097416*I  0.500882 - 0.002251*I]
    [ 0.057002 + 0.495035*I   0.35059 + 0.625323*I -0.255514 + 0.415284*I]
    [-0.394105 + 0.421778*I  0.392266 + 0.039345*I   0.352905 - 0.62195*I]
**********************************************************************
File "/Users/jdemeyer/sage-5.0.beta1-gcc/devel/sage-main/sage/matrix/matrix2.pyx", line 7987:
    sage: M.round(6).zero_at(10^-6)
Expected:
    [             -1.528503                      0                      0]
    [  0.459974 - 0.40061*I              -1.741233                      0]
    [-0.934304 + 0.148868*I   0.54833 + 0.073202*I              -0.550725]
Got:
    [             1.528503                     0                     0]
    [-0.459974 + 0.40061*I              1.741233                     0]
    [0.934304 - 0.148868*I -0.54833 - 0.073202*I              0.550725]
**********************************************************************
File "/Users/jdemeyer/sage-5.0.beta1-gcc/devel/sage-main/sage/matrix/matrix2.pyx", line 8031:
    sage: G.round(6).zero_at(10^-6)
Expected:
    [-0.408248 -0.408248 -0.816497]
    [ 0.707107 -0.707107       0.0]
    [ -0.57735  -0.57735   0.57735]
Got:
    [ 0.408248  0.408248  0.816497]
    [ 0.707107 -0.707107       0.0]
    [ -0.57735  -0.57735   0.57735]
**********************************************************************
File "/Users/jdemeyer/sage-5.0.beta1-gcc/devel/sage-main/sage/matrix/matrix2.pyx", line 8035:
    sage: M.round(10).zero_at(10^-10)
Expected:
    [-2.4494897428           0.0           0.0]
    [-3.6742346142  0.7071067812           0.0]
    [-4.8989794856  1.4142135624           0.0]
Got:
    [2.4494897428          0.0          0.0]
    [3.6742346142 0.7071067812          0.0]
    [4.8989794856 1.4142135624          0.0]
**********************************************************************

[robertwb]

I think a lot of these could be fixed without marking them as random (essentially disabling the test). E.g. for

sage: B.is_hermitian(algorithm='naive', tol=1.0e-17)

we could try increasing the tolerance, and for the sign issues, we could write

G.round(6) * sgn(G[0,0])

to force the top-left entry to be positive.

[jdemeyer]

Replying to robertwb:

I think a lot of these could be fixed without marking them as random (essentially disabling the test). E.g. for

sage: B.is_hermitian(algorithm='naive', tol=1.0e-17)

we could try increasing the tolerance

Doesn't work. On OS X 10.7, the matrix actually is Hermitian to infinite precision.

G.round(6) * sgn(G[0,0])

to force the top-left entry to be positive.

Doesn't work either, since you would have to do this on a row-by-row or column-by-column basis for the Gram Schmidt tests. I think this would make things too complicated. Besides, the important thing to check are the defining properties of the Gram-Schmidt reduction, which is checked in these doctests.

[jhpalmieri]

Looks good to me, and passes doctests on several platforms (despite what the patchbot thinks).