Opened 14 months ago

Last modified 7 weeks ago

#31539 new defect

conda-forge-standard (linux): python3 spkg-configure.m4 rejects conda's python3

Reported by: mkoeppe Owned by:
Priority: major Milestone: sage-9.7
Component: build: configure Keywords:
Cc: isuruf, vdelecroix, dimpase Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description (last modified by mkoeppe)

Checking whether SageMath should install SPKG python3...
checking whether any of bzip2 xz libffi is installed as or will be installed as SPKG... no
checking for python3 >= 3.6.0, < 3.10.0 with modules sqlite3, ctypes, math, hashlib, crypt, readline, socket, zlib, distutils.core... 
checking ... whether /opt/conda/bin/python3 is good... no, this is a misconfigured Python whose sysconfig compiler/linker flags contain -I or -L options, which may cause wrong versions of libraries to leak into the build of Python packages - see https://trac.sagemath.org/ticket/31132; to use it anyway, use ./configure --with-python=/opt/conda/bin/python3
checking ... whether /opt/conda/bin/python3 is good... no, this is a misconfigured Python whose sysconfig compiler/linker flags contain -I or -L options, which may cause wrong versions of libraries to leak into the build of Python packages - see https://trac.sagemath.org/ticket/31132; to use it anyway, use ./configure --with-python=/opt/conda/bin/python3

configure: to try to use a different system python, use ./configure --with-python=/path/to/python
configure: no suitable system package found for SPKG python3

Critical for Sage 9.3 because it is a regression of platform support

Change History (15)

comment:1 Changed 14 months ago by mkoeppe

  • Description modified (diff)

comment:2 Changed 14 months ago by mkoeppe

On this system, python3 -m sysconfig gives LDFLAGS = "-Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,/opt/conda/lib -Wl,-rpath-link,/opt/conda/lib -L/opt/conda/lib -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,/opt/conda/lib -Wl,-rpath-link,/opt/conda/lib -L/opt/conda/lib"

It is the -L/opt/conda/lib that our configure is unhappy about.

comment:3 Changed 13 months ago by mkoeppe

  • Milestone changed from sage-9.3 to sage-9.4

Moving to 9.4, as 9.3 has been released.

comment:4 Changed 11 months ago by mkoeppe

  • Cc vdelecroix added

comment:5 Changed 11 months ago by mkoeppe

@isuruf wrote in https://groups.google.com/g/sage-devel/c/ZCjFpJGD-JA/m/5hOi6hcZBwAJ:

Looks like the check for misconfigured python is a little bit too extreme. If sysconfig's CFLAGS includes any path in the environment's CFLAGS, then it shouldn't be considered misconfigured.

From the viewpoint of user installable Python packages, it is actually misconfigured, for the same reasons that led to the introduction of this check in #31132. The -L options are in the front of the command line, and take precedence over any package-provided -L options that come in from the use of setuptools.

Last edited 11 months ago by slelievre (previous) (diff)

comment:6 Changed 10 months ago by mkoeppe

  • Milestone changed from sage-9.4 to sage-9.5

comment:7 Changed 8 months ago by mkoeppe

From isuruf in https://trac.sagemath.org/ticket/31132#comment:77:

-I can't be stripped, but -L can be stripped by setting LDSHARED env variable. This would fix the conda-forge issue as conda-forge only use -L and doesn't use -I.

comment:8 Changed 8 months ago by mkoeppe

A previous attempt to use LDSHARED were not successful in #29408. But of course in the meantime the Xcode python3 has changed (it no longer seems to use xcrun) so this may be worth trying if it can be done without breaking another platform.

Has there been a discussion in conda-forge regarding this misconfiguration of python3?

comment:9 Changed 8 months ago by isuruf

Has there been a discussion in conda-forge regarding this misconfiguration of python3?

Nope, but the use-case we have in sage is really a rare case because we usually don't want to override a library in a conda environment. Removing -L will cause more harm than good.

Something like below should give the LDSHARED to use and ldflags to append to LDFLAGS.

import sysconfig
import os
import shlex

orig_ldshared = sysconfig.get_config_var("LDSHARED")
orig_cc = sysconfig.get_config_var("CC")
if 'LDSHARED' in os.environ:
    orig_ldshared = os.environ['LDSHARED']
elif 'CC' in os.environ and orig_ldshared.startswith(orig_cc):
    orig_ldshared = os.environ['CC'] + orig_ldshared[len(orig_cc):]

orig_ldshared = shlex.split(orig_ldshared)
new_ldshared = shlex.join(filter(lambda x: not x.startswith("-L"), orig_ldshared))
append_ldflags = shlex.join(filter(lambda x: x.startswith("-L"), orig_ldshared))

print(new_ldshared)
print(append_ldflags)

comment:10 Changed 8 months ago by isuruf

@mkoeppe, would you be able to help with this? I don't have experience with writing autoconf scripts. Only experience in patching them. ;)

comment:11 Changed 8 months ago by mkoeppe

  • Cc dimpase added

comment:12 follow-up: Changed 8 months ago by dimpase

Not clear from the ticket description whether this is a failure to accept a correct configuration, or there is more to fix than just this.

If the former, we can just waive Conda's Python3 python through, if we knew how to distinguish it.

Is https://wiki.sagemath.org/Conda up to date?

comment:13 in reply to: ↑ 12 Changed 7 months ago by mkoeppe

Replying to dimpase:

Not clear from the ticket description

... it should be clear from the comments above

comment:14 Changed 5 months ago by mkoeppe

  • Milestone changed from sage-9.5 to sage-9.6
  • Priority changed from critical to major

comment:15 Changed 7 weeks ago by mkoeppe

  • Milestone changed from sage-9.6 to sage-9.7
Note: See TracTickets for help on using tickets.