Opened 15 months ago

Last modified 3 months ago

#30163 needs_work enhancement

cygwin: Try again to use system BLAS and system R

Reported by: mkoeppe Owned by:
Priority: major Milestone: sage-9.5
Component: porting: Cygwin Keywords:
Cc: embray, slelievre Merged in:
Authors: Matthias Koeppe Reviewers: github.com/mkoeppe/sage/actions/runs/207840100, github.com/mkoeppe/sage/actions/runs/207840103
Report Upstream: N/A Work issues:
Branch: u/mkoeppe/cygwin__try_again_to_use_system_blas_and_system_r (Commits, GitHub, GitLab) Commit: 7f8ede4439787684d16f39e8d4d42937993efff2
Dependencies: #30157, #30149, #29441, #30127, #30277 Stopgaps:

Status badges

Description (last modified by mkoeppe)

... they were disabled in spkg-configure for 9.1 in #29538 ...

Previous system BLAS tickets:

Change History (15)

comment:1 Changed 14 months ago by mkoeppe

  • Dependencies changed from #30157, #30149, #29441 to #30157, #30149, #29441, #30127, #30277

comment:2 Changed 14 months ago by mkoeppe

  • Branch set to u/mkoeppe/cygwin__try_again_to_use_system_blas_and_system_r

comment:3 Changed 14 months ago by mkoeppe

  • Authors set to Matthias Koeppe
  • Commit set to 7f8ede4439787684d16f39e8d4d42937993efff2
  • Reviewers set to github.com/mkoeppe/sage/actions/runs/207840100, github.com/mkoeppe/sage/actions/runs/207840103
  • Status changed from new to needs_review

Last 10 new commits:

b8e366ebuild/pkgs/sagelib/spkg-src: chmod +x
0473ef3Merge branch 't/21559/change-src-bin-installation' into t/29950/build_sagelib_using_installed_sage_setup
7244371Merge branch 't/29701/replace_use_of_module_list_optionalextension' into t/29950/build_sagelib_using_installed_sage_setup
4344f89Merge branch 't/21559/change-src-bin-installation' into t/29950/build_sagelib_using_installed_sage_setup
01b96b0Merge branch 't/29701/replace_use_of_module_list_optionalextension' into t/29950/build_sagelib_using_installed_sage_setup
2818739Merge branch 't/29950/build_sagelib_using_installed_sage_setup' into t/30277/remove_src_module_list_py
8a19fe2build/make/Makefile.in (sagelib-clean): Clean the new build location
ccc67b0src/sage_setup: Update cythonized_dir in doctests
df38027Merge branch 't/29950/build_sagelib_using_installed_sage_setup' into t/30277/remove_src_module_list_py
7f8ede4Merge branch 't/30277/remove_src_module_list_py' into t/30163/cygwin__try_again_to_use_system_blas_and_system_r

comment:4 Changed 14 months ago by mkoeppe

cygwin-standard (https://github.com/mkoeppe/sage/runs/983346777?check_suite_focus=true) maplotlib build fails:

      File "/tmp/pip-req-build-ixb64a6o/setupext.py", line 461, in add_numpy_flags
        import numpy as np
      File "/cygdrive/d/a/sage/sage/local/lib/python3.7/site-packages/numpy/__init__.py", line 143, in <module>
        from . import lib
      File "/cygdrive/d/a/sage/sage/local/lib/python3.7/site-packages/numpy/lib/__init__.py", line 25, in <module>
        from .index_tricks import *
      File "/cygdrive/d/a/sage/sage/local/lib/python3.7/site-packages/numpy/lib/index_tricks.py", line 11, in <module>
        import numpy.matrixlib as matrixlib
      File "/cygdrive/d/a/sage/sage/local/lib/python3.7/site-packages/numpy/matrixlib/__init__.py", line 4, in <module>
        from .defmatrix import *
      File "/cygdrive/d/a/sage/sage/local/lib/python3.7/site-packages/numpy/matrixlib/defmatrix.py", line 11, in <module>
        from numpy.linalg import matrix_power
      File "/cygdrive/d/a/sage/sage/local/lib/python3.7/site-packages/numpy/linalg/__init__.py", line 73, in <module>
        from .linalg import *
      File "/cygdrive/d/a/sage/sage/local/lib/python3.7/site-packages/numpy/linalg/linalg.py", line 33, in <module>
        from numpy.linalg import lapack_lite, _umath_linalg
    ImportError: No such file or directory

also scipy fails with the same error.

r system package is not used because prereq curl is not accepted:

Checking whether SageMath should install SPKG curl...
checking for curl 7.22... /usr/bin/curl
checking for gawk... (cached) gawk
checking for curl-config... /usr/bin/curl-config
checking for the version of libcurl... 7.71.1
checking for libcurl >= version 7.22... yes
checking whether libcurl is usable... no
checking whether programs linking to libcurl can be executed... no
configure: no suitable system package found for SPKG curl

and then building r also fails:

Warning in solve.default(rgb) :
  unable to load shared object '/cygdrive/d/a/sage/sage/local/var/tmp/sage/build/r-3.6.3/src/modules//lapack.dll':
  No such file or directory
Error in solve.default(rgb) : LAPACK routines cannot be loaded
Error: unable to load R code in package 'grDevices'
Execution halted
make[7]: *** [../../../share/make/lazycomp.mk:9: ../../../library/grDevices/R/grDevices.rdb] Error 1
make[7]: Target 'mklazycomp' not remade because of errors.
make[7]: Leaving directory '/cygdrive/d/a/sage/sage/local/var/tmp/sage/build/r-3.6.3/src/src/library/grDevices'
make[6]: *** [Makefile:28: all] Error 2
make[6]: Leaving directory '/cygdrive/d/a/sage/sage/local/var/tmp/sage/build/r-3.6.3/src/src/library/grDevices'
make[5]: *** [Makefile:39: R] Error 1
make[5]: Leaving directory '/cygdrive/d/a/sage/sage/local/var/tmp/sage/build/r-3.6.3/src/src/library'

comment:6 Changed 14 months ago by mkoeppe

From the numpy build log:

blas_opt_info:
blas_mkl_info:
Disabled blas_mkl_info: (MKLROOT is None)
Disabled blas_mkl_info: (MKLROOT is None)
customize UnixCCompiler
  libraries mkl_rt not found in []
  NOT AVAILABLE

blis_info:
  libraries blis not found in ['/cygdrive/d/a/sage/sage/local/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/']
  NOT AVAILABLE

openblas_info:
Disabled openblas_info: (OPENBLAS is None)
  libraries openblas not found in []
  NOT AVAILABLE

atlas_3_10_blas_threads_info:
Disabled atlas_3_10_blas_threads_info: (PTATLAS is None)
  libraries tatlas not found in []
  NOT AVAILABLE

atlas_3_10_blas_info:
Disabled atlas_3_10_blas_info: (ATLAS is None)
  libraries satlas not found in []
  NOT AVAILABLE

atlas_blas_threads_info:
Disabled atlas_blas_threads_info: (PTATLAS is None)
  libraries ptf77blas,ptcblas,atlas not found in []
  NOT AVAILABLE

atlas_blas_info:
Disabled atlas_blas_info: (ATLAS is None)
  libraries f77blas,cblas,atlas not found in []
  NOT AVAILABLE

accelerate_info:
  NOT AVAILABLE

blas_info:
C compiler: gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -ggdb -O2 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector-strong --param=ssp-buffer-size=4 -fdebug-prefix-map=/cygdrive/d/cyg_pub/devel/python/python37/python37-3.7.7-1.x86_64/build=/usr/src/debug/python37-3.7.7-1 -fdebug-prefix-map=/cygdrive/d/cyg_pub/devel/python/python37/python37-3.7.7-1.x86_64/src/Python-3.7.7=/usr/src/debug/python37-3.7.7-1 -ggdb -O2 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector-strong --param=ssp-buffer-size=4 -fdebug-prefix-map=/cygdrive/d/cyg_pub/devel/python/python37/python37-3.7.7-1.x86_64/build=/usr/src/debug/python37-3.7.7-1 -fdebug-prefix-map=/cygdrive/d/cyg_pub/devel/python/python37/python37-3.7.7-1.x86_64/src/Python-3.7.7=/usr/src/debug/python37-3.7.7-1

creating /tmp/tmp7np5_kr4/tmp
creating /tmp/tmp7np5_kr4/tmp/tmp7np5_kr4
compile options: '-I/usr/include -I/usr/include/suitesparse -I/cygdrive/d/a/sage/sage/local/include -c'
gcc: /tmp/tmp7np5_kr4/source.c
gcc /tmp/tmp7np5_kr4/tmp/tmp7np5_kr4/source.o -L/usr/lib -lblas -llapack -o /tmp/tmp7np5_kr4/a.out.exe
  FOUND:
    libraries = ['blas', 'lapack', 'blas']
    library_dirs = ['/usr/lib']
    include_dirs = ['/usr/include', '/usr/include/suitesparse', '/cygdrive/d/a/sage/sage/local/include']
    language = c
    define_macros = [('HAVE_CBLAS', None)]

  FOUND:
    define_macros = [('NO_ATLAS_INFO', 1), ('HAVE_CBLAS', None)]
    libraries = ['blas', 'lapack', 'blas']
    library_dirs = ['/usr/lib']
    include_dirs = ['/usr/include', '/usr/include/suitesparse', '/cygdrive/d/a/sage/sage/local/include']
    language = c

lapack_opt_info:
lapack_mkl_info:
Disabled lapack_mkl_info: (MKLROOT is None)
Disabled lapack_mkl_info: (MKLROOT is None)
  libraries mkl_rt not found in []
  NOT AVAILABLE

openblas_lapack_info:
Disabled openblas_lapack_info: (OPENBLAS is None)
  libraries openblas not found in []
  NOT AVAILABLE

openblas_clapack_info:
Disabled openblas_clapack_info: (OPENBLAS is None)
  libraries openblas,lapack not found in []
  NOT AVAILABLE

flame_info:
  libraries flame not found in ['/cygdrive/d/a/sage/sage/local/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/']
  NOT AVAILABLE

atlas_3_10_threads_info:
Disabled atlas_3_10_threads_info: (PTATLAS is None)
<class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
  NOT AVAILABLE

atlas_3_10_info:
Disabled atlas_3_10_info: (ATLAS is None)
<class 'numpy.distutils.system_info.atlas_3_10_info'>
  NOT AVAILABLE

atlas_threads_info:
Disabled atlas_threads_info: (PTATLAS is None)
<class 'numpy.distutils.system_info.atlas_threads_info'>
  NOT AVAILABLE

atlas_info:
Disabled atlas_info: (ATLAS is None)
<class 'numpy.distutils.system_info.atlas_info'>
  NOT AVAILABLE

lapack_info:
  FOUND:
    libraries = ['blas', 'lapack', 'lapack']
    library_dirs = ['/usr/lib']
    language = f77

  FOUND:
    libraries = ['blas', 'lapack', 'lapack', 'blas', 'lapack', 'blas']
    library_dirs = ['/usr/lib']
    language = c
    define_macros = [('NO_ATLAS_INFO', 1), ('HAVE_CBLAS', None)]
    include_dirs = ['/usr/include', '/usr/include/suitesparse', '/cygdrive/d/a/sage/sage/local/include']

.....





/cygdrive/d/a/sage/sage/local/var/tmp/sage/build/numpy-1.19.1/src/numpy/distutils/system_info.py:1914: UserWarning: 
    Optimized (vendor) Blas libraries are not found.
    Falls back to netlib Blas library which has worse performance.
    A better performance should be easily gained by switching
    Blas library.
  if self._calc_info(blas):

comment:7 Changed 14 months ago by mkoeppe

  • Description modified (diff)
  • Status changed from needs_review to needs_work

comment:8 Changed 14 months ago by mkoeppe

  • Milestone changed from sage-9.2 to sage-9.3

comment:9 Changed 14 months ago by mkoeppe

Hi Erik, do you have time to look into this ticket?

comment:10 Changed 14 months ago by embray

This looks like a good start so far and your analysis of how to detect openblas makes sense. I'll look into it when I get back to my Windows machine.

comment:11 Changed 8 months ago by mkoeppe

  • Milestone changed from sage-9.3 to sage-9.4

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

comment:12 Changed 8 months ago by embray

It looks like one of the first things that needs to be fixed is properly detecting the system curl on Cygwin. I remember having trouble with this before. I thought I looked into it at one point, and I think even fixed it somehow, but now I don't remember for sure, or why the fix hasn't made it into Sage if indeed I did fix it...

comment:13 Changed 7 months ago by mkoeppe

Version 0, edited 7 months ago by mkoeppe (next)

comment:14 Changed 7 months ago by embray

Ok, great. I just mentioned it since one of your earlier comments on this thread said as much. But that was also several months ago. I could have sworn it was working now.

comment:15 Changed 3 months ago by mkoeppe

  • Milestone changed from sage-9.4 to sage-9.5
Note: See TracTickets for help on using tickets.