Opened 5 years ago

Closed 5 years ago

Last modified 5 years ago

#17365 closed defect (fixed)

ATLAS does not build shared lib on Cygwin64

Reported by: jpflori Owned by:
Priority: major Milestone: sage-6.5
Component: porting: Cygwin Keywords:
Cc: tscrim, vbraun, gouezel Merged in:
Authors: Jean-Pierre Flori Reviewers: Volker Braun, Sebastien Gouezel
Report Upstream: Reported upstream. No feedback yet. Work issues:
Branch: 0b0f078 (Commits) Commit:
Dependencies: Stopgaps:

Description

It tries to use some MSVCRT thread functions instead of Cygwin's ones and fails at link time because of undefined references. (And later on numpy fails to build because of the same undefined references.)

See:

Attachments (3)

cygwin_threads.patch (1.0 KB) - added by jpflori 5 years ago.
atlas-3.10.2.log (17.1 KB) - added by tscrim 5 years ago.
atlas-jp.log (30.1 KB) - added by jpflori 5 years ago.

Download all attachments as: .zip

Change History (33)

Changed 5 years ago by jpflori

comment:1 Changed 5 years ago by jpflori

Attached patch should treat Cygwin and Cygwin64 as Cygwin used to be treated already.

comment:2 Changed 5 years ago by jpflori

(Not tested yet, I want to reproduce a failed build first as I did not have access to the first failed build log and it takes ages...)

comment:3 Changed 5 years ago by jpflori

Tested now and works ok. Or rather ATLAS now builds correctly, I don't promise it is functional at this point.

comment:4 Changed 5 years ago by jpflori

  • Authors set to Jean-Pierre Flori
  • Branch set to u/jpflori/ticket/17365
  • Commit set to f3402576d2a6cecf79856d5821a324c28cfa8d45
  • Status changed from new to needs_review

New commits:

f340257Let ATLAS use Cygwin thread functions on Cygwin64.

comment:5 Changed 5 years ago by git

  • Commit changed from f3402576d2a6cecf79856d5821a324c28cfa8d45 to 0b0f0785e23437afa504bd537fb69df974a7c8c5

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

0b0f078Let ATLAS use Cygwin thread functions on Cygwin64.

comment:6 Changed 5 years ago by jpflori

  • Cc vbraun added

comment:7 Changed 5 years ago by tscrim

I will try to get to this today or tomorrow.

comment:8 Changed 5 years ago by vbraun

Looks good, did you send your patch upstream?

comment:9 Changed 5 years ago by jpflori

Nope, I wanted to have a look at the devel version of ATLAS first (as it would also be a good thing to use it in Sage if working and more or less stabilized, the version we ship right now is very suboptimal or broken on some platforms), but found no time yet...

comment:10 Changed 5 years ago by jpflori

  • Report Upstream changed from N/A to Reported upstream. No feedback yet.

Changed 5 years ago by tscrim

comment:11 Changed 5 years ago by tscrim

I'm unable to build ATLAS with this branch on the latest develop + #15015 (after running a make distclean). I've attached the build log and I was compiling on a single thread with Win 8 on a Intel quad-core. I also ran a rebase and still got the same error - it stems from a permission denied error in part of ATLAS's source code. Is this an issue with my setup?

comment:12 Changed 5 years ago by jpflori

I'd say it comes from your setup.

I'll have a closer look right now.

comment:13 Changed 5 years ago by jpflori

The problematic files have suspicious timestamps:

...atlconf_misc.c: Timestamp out of range; substituting 2514-05-29 18:53:03.999999999
...atlconf_misc.c' has modification time 15759751875 s in the future

and end up with permission errors:

...atlconf_misc.c: Permission denied

Can you have a closer look at these files in the build directory?

But this has nothing to do with the patch here. Maybe with our repackaged ATLAS tarball, but that was not changed here.

comment:14 Changed 5 years ago by tscrim

Here's some info on the file

$ stat atlconf_misc.c
  File: ‘atlconf_misc.c’
  Size: 25087           Blocks: 28         IO Block: 65536  regular file
Device: f458da8eh/4099463822d   Inode: 19140298416521874  Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1001/  Travis)   Gid: (  513/    None)
Access: 2015-01-01 05:44:33.963182200 -0800
Modify: 2014-07-10 09:22:02.000000000 -0700
Change: 2015-01-01 05:44:33.963182200 -0800
 Birth: 2015-01-01 05:44:33.963182200 -0800

This is consistent with other files that I randomly stat'ed, including the file permissions. However I'm not quite sure what I should be looking for...

comment:15 Changed 5 years ago by jpflori

  • Cc gouezel added

@Sébastien: any report on this one?

@Travis: I don't have any clue yet for your problem...

comment:16 Changed 5 years ago by gouezel

With the patch (and SAGE_ATLAS_ARCH=base, so maybe some part of the configuration process was skipped), ATLAS compiles fine for me (Win 7, 4 cores, compiling on a single thread).

comment:17 Changed 5 years ago by jpflori

We would need to make sure that it works for a multi threaded build (e.g. with autotuning or maybe with SAGE_ATLAS_ARCH=fast if that works on Cygwin), otherwise the problematic code in ATLAS is not used IIRC. In particular I used to build on Cygwin64 with SAGE_ATLAS_ARCH=base as well and did not have this problem until I stopped using SAGE_ATLAS_ARCH=base and that triggered a completely autotuned build.

comment:18 Changed 5 years ago by jpflori

Seems to work fine for me once again... The compilation did not finish yet so I can not ensure that the thread fix is working once again, but I don't get Travis' problem.

My stat output looks similar, I'll quickly attach the (start of the) log of the ongoing build.

$ stat atlconf_misc.c
  Fichier : « atlconf_misc.c »
   Taille : 25087       Blocs : 28         Blocs d'E/S : 65536  fichier
Périphérique : 784a2af6h/2018126582d    Inœud : 16044073672726054  Liens : 1
Accès : (0644/-rw-r--r--)  UID : ( 1001/      jp)   GID : (  513/    None)
 Accès : 2015-01-13 06:46:54.816090400 +0100
Modif. : 2014-07-10 18:22:02.000000000 +0200
Changt : 2015-01-13 06:46:54.819090600 +0100
  Créé : 2015-01-13 06:46:54.816090400 +0100

Changed 5 years ago by jpflori

comment:19 Changed 5 years ago by jpflori

Note that I don't have timestamps issues.

comment:20 Changed 5 years ago by tscrim

My Win8 laptop tends to make things behave like their multithreaded...at least they seem to jump from processor to processor...so maybe that's the issue? I'll try it with SAGE_ATLAS_ARCH=base and/or SAGE_ATLAS_ARCH=fast to see if that works, and perhaps with a variety of other build flags.

comment:21 Changed 5 years ago by jpflori

Note that SAGE_ATLAS_ARCH=base explicitely disable threads so that won't actually test that the patch here is working... The fast setting should use threads (if your CPU has more than one logical core).

Whatsoever, your compilation fails very early and it really looks like it is a filesystem issue. Did you test installing ATLAS without this branch as well?

comment:22 Changed 5 years ago by gouezel

I am confused: I compiled with SAGE_ATLAS_ARCH=fast, and it compiles fine with and without the patch. Build time multiplied by 10 compared to SAGE_ATLAS_ARCH=base, so the flag is taken into account, but it seems from CPU use that the build was using only one core (on a CPU with 4 cores). The build was started with ./sage -f atlas, what else should I do to build it multithreaded and trigger the bug?

comment:23 Changed 5 years ago by tscrim

I don't remember exactly fit worked or not, but I know it didn't fail at this point. I will try it without this branch when I get back from the JMM tomorrow.

Last edited 5 years ago by tscrim (previous) (diff)

comment:24 Changed 5 years ago by jpflori

Just look in the log and see at the end if it built a shared lib or not. To detect if not, you can look for "undefined ref". Or just post your logs here and I'll tell!

comment:25 Changed 5 years ago by gouezel

I looked at the proper place in my logs. Compilation with SAGE_ATLAS_ARCH=fast

  • with the patch, the library is correctly built
  • without the patch, it is not built

So, the patch works for me. No problem such as Travis'.

comment:26 Changed 5 years ago by jpflori

By the way, the atlas/cblas linking situation is very broken on cygwin anyway. See #17630.

comment:27 Changed 5 years ago by tscrim

I get the same error now based on develop, so it must be something specific to my Win8 machine (which is also a dual boot with Ubuntu). So don't let me hold this up.

comment:28 Changed 5 years ago by jpflori

  • Reviewers set to Volker Braun, Sébastien Gouezel
  • Status changed from needs_review to positive_review

So I guess that Sébastien and Volker reviews are enough.

comment:29 Changed 5 years ago by vbraun

  • Branch changed from u/jpflori/ticket/17365 to 0b0f0785e23437afa504bd537fb69df974a7c8c5
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:30 Changed 5 years ago by kcrisman

  • Commit 0b0f0785e23437afa504bd537fb69df974a7c8c5 deleted
  • Reviewers changed from Volker Braun, Sébastien Gouezel to Volker Braun, Sebastien Gouezel

(I'm making the change in name, but also it would be okay to change all previous instances to the one with the accent aigu.)

Note: See TracTickets for help on using tickets.