Opened 10 months ago

Last modified 4 weeks ago

#25816 new defect

git broken in ./sage -sh on Cygwin

Reported by: embray Owned by:
Priority: major Milestone: sage-pending
Component: porting: Cygwin Keywords:
Cc: Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: u/mstipulanti/adding_a_note_in_the_doc_of_the_method_delta___available_in_combinat_words_finite_word (Commits) Commit: 6398d0e3721a1ef19450ed55fe250b9dffb151f3
Dependencies: Stopgaps:

Description

This is a problem that has been plaguing me for a few months but I haven't taken much time to work out the source of the problem yet.

It might also be dependent on the package version of Cygwin's git pacakge, but when using the system git from Cygwin in a ./sage -sh shell, it's broken. A workaround is to put PATH=/usr/bin:$PATH in front. So it seems maybe some library installed as part of Sage is incompatible with the system git.

This is a problem, in particular, for pip installing packages from a git repo, for example.

Change History (11)

comment:1 Changed 10 months ago by embray

In the process of git clone it forks and spawns /usr/libexec/git-core/git-remote-https.exe. This in turn loads:

--- Process 1812 loaded C:\Windows\System32\ntdll.dll at 00007FFEA4BF0000
--- Process 1812 loaded C:\Windows\System32\kernel32.dll at 00007FFEA3D30000
--- Process 1812 loaded C:\Windows\System32\KernelBase.dll at 00007FFEA1B90000
--- Process 1812 thread 13644 created
--- Process 1812 thread 17792 created
--- Process 1812 thread 29828 created
--- Process 1812 loaded C:\cygwin64\bin\cygcurl-4.dll at 00000003F6D60000
--- Process 1812 loaded C:\cygwin64\bin\cygwin1.dll at 0000000180040000
--- Process 1812 loaded C:\cygwin64\bin\cygintl-8.dll at 00000003ECE50000
--- Process 1812 loaded C:\Users\Erik M. Bray\Home\src\sagemath\sage\local\bin\cygiconv-2.dll at 00000003CB550000
--- Process 1812 loaded C:\Users\Erik M. Bray\Home\src\sagemath\sage\local\bin\cygpcre-1.dll at 00000003C6FC0000
--- Process 1812 loaded C:\cygwin64\bin\cygz.dll at 00000003E8DC0000
--- Process 1812 loaded C:\cygwin64\bin\cygcrypto-1.0.0.dll at 00000003F5180000
--- Process 1812 loaded C:\cygwin64\bin\cyggssapi_krb5-2.dll at 00000003F2EF0000
--- Process 1812 loaded C:\cygwin64\bin\cygldap-2-4-2.dll at 00000003F26A0000
--- Process 1812 loaded C:\cygwin64\bin\cygidn2-0.dll at 00000003F2BC0000
--- Process 1812 loaded C:\cygwin64\bin\cyglber-2-4-2.dll at 00000003F2750000
--- Process 1812 loaded C:\cygwin64\bin\cygnghttp2-14.dll at 00000003ECF70000
--- Process 1812 loaded C:\cygwin64\bin\cygpsl-5.dll at 00000003EBEA0000
--- Process 1812 loaded C:\cygwin64\bin\cygssh2-1.dll at 00000003EB010000
--- Process 1812 loaded C:\cygwin64\bin\cygssl-1.0.0.dll at 00000003EAFA0000
--- Process 1812, exception c0000139 at 00007FFEA4C89968
--- Process 1812 loaded C:\cygwin64\bin\cygkrb5-3.dll at 00000003EC900000
--- Process 1812 loaded C:\cygwin64\bin\cygk5crypto-3.dll at 00000003EC9E0000
--- Process 1812 thread 21968 exited with status 0xc0000139
--- Process 1812 thread 17792 exited with status 0xc0000139
--- Process 1812 thread 29828 exited with status 0xc0000139
--- Process 1812 exited with status 0xc0000139

and then dies. It's odd that it loads iconv and pcre from Sage. Rather, it's expected in a way, but where it gets strange is that git.exe also loads these DLLs but it uses the versions from the system:

--- Process 9444 created
--- Process 9444 loaded C:\Windows\System32\ntdll.dll at 00007FFEA4BF0000
--- Process 9444 loaded C:\Windows\System32\kernel32.dll at 00007FFEA3D30000
--- Process 9444 loaded C:\Windows\System32\KernelBase.dll at 00007FFEA1B90000
--- Process 9444 thread 9700 created
--- Process 9444 thread 4776 created
--- Process 9444 thread 1016 created
--- Process 9444 loaded C:\cygwin64\bin\cygwin1.dll at 0000000180040000
--- Process 9444 loaded C:\cygwin64\bin\cygiconv-2.dll at 00000003F2530000
--- Process 9444 loaded C:\cygwin64\bin\cygintl-8.dll at 00000003ECE50000
--- Process 9444 loaded C:\cygwin64\bin\cygpcre-1.dll at 00000003EC540000
--- Process 9444 loaded C:\cygwin64\bin\cygz.dll at 00000003E8DC0000

It could be some subtlety about the DLL order? That remains unclear at the moment. I need to take a closer look at the DLL dependency trees.

comment:2 follow-up: Changed 10 months ago by embray

I think I know the reason. It's because DLL search order normally looks first in the same directory as the executable loading the DLL, then resorts to $PATH. Since git.exe is in /usr/bin/ it has most of its dependent DLLs right beside it. But executables installed under /usr/libexec/ do not, so they find these DLLs first in $SAGE_LOCAL/bin.

This is definitely a bit of an oddity to git, since so much of it is installed user /usr/libexec/git-core.

I'm not sure much can be done about this from the Sage end of things. Perhaps for the Sage-for-Windows build I can ensure that some DLLs are copied into /usr/libexec/git-core/ in order for git to function properly.

comment:3 Changed 10 months ago by embray

Turns out this only affects those two DLLs:

$ cygcheck.exe /usr/libexec/git-core/*.exe | grep 'sage\\local' | sed 's/^ *//' | sort | uniq
C:\Users\Erik M. Bray\Home\src\sagemath\sage\local\bin\cygiconv-2.dll
C:\Users\Erik M. Bray\Home\src\sagemath\sage\local\bin\cygpcre-1.dll

comment:4 Changed 10 months ago by embray

This works as a general solution. I'll add it to the Cygwin64Port wiki page:

for dllname in $(cygcheck.exe /usr/libexec/git-core/*.exe | grep 'sage\\local' | sed 's|.*\\\(.\+\.dll\)|\1|' | sort | uniq); do
    cp /usr/bin/$dllname /usr/libexec/git-core/
done

This might also be worth mentioning upstream in the Cygwin git package, since this is a bit fragile.

comment:5 in reply to: ↑ 2 Changed 10 months ago by embray

Replying to embray:

I'm not sure much can be done about this from the Sage end of things. Perhaps for the Sage-for-Windows build I can ensure that some DLLs are copied into /usr/libexec/git-core/ in order for git to function properly.

Well, one thing we could do is prefer not to build those libraries in the first place. That, of course, is being worked on...

comment:6 Changed 9 months ago by mstipulanti

  • Branch set to u/mstipulanti/adding_a_note_in_the_doc_of_the_method_delta___available_in_combinat_words_finite_word

comment:7 Changed 9 months ago by git

  • Commit set to 6398d0e3721a1ef19450ed55fe250b9dffb151f3

Branch pushed to git repo; I updated commit sha1. New commits:

6398d0eagreeing with comment 6

comment:8 Changed 9 months ago by embray

  • Milestone changed from sage-8.3 to sage-8.4

comment:9 Changed 6 months ago by embray

  • Milestone changed from sage-8.4 to sage-8.5

comment:10 Changed 4 months ago by embray

  • Milestone changed from sage-8.5 to sage-8.7

Retargeting some of my tickets (somewhat optimistically for now).

comment:11 Changed 4 weeks ago by embray

  • Milestone changed from sage-8.7 to sage-pending

Removing most of the rest of my open tickets out of the 8.7 milestone, which should be closed.

Note: See TracTickets for help on using tickets.