Opened 22 months ago

Last modified 19 months ago

#28906 needs_work defect

generate libpng.pc, zlib.pc if needed

Reported by: dimpase Owned by:
Priority: major Milestone: sage-wishlist
Component: build: configure Keywords:
Cc: fbissey, embray, isuruf, mkoeppe Merged in:
Authors: Dima Pasechnik Reviewers:
Report Upstream: N/A Work issues:
Branch: u/dimpase/packages/missingpcs (Commits, GitHub, GitLab) Commit: ba7d9632b368790279a890e225eb98e37c678cf5
Dependencies: Stopgaps:

Status badges

Description (last modified by dimpase)

libpng.pc, gdlib.pc, zlib.pcetc are needed in the calls of pkgconfig.parse in src/module_list.py

some perfectly fine implementations of zlib etc. don't install zlib.pc etc. (e.g. MacOS does not).

Since #26286 this problem was sleeping, until we updated pkgconfig on #28883, which made it throw an error on absense of zlib.pc.

This ticket provides generation of zlib.pc and libpng.pc, among with some fly-by refactoring.

Change History (23)

comment:1 Changed 22 months ago by dimpase

  • Description modified (diff)
  • Summary changed from generate zlib.pc if needed, or avoid using it. to generate libpng.pc, gdlib.pc, zlib.pc if needed

comment:2 Changed 22 months ago by dimpase

  • Description modified (diff)
  • Summary changed from generate libpng.pc, gdlib.pc, zlib.pc if needed to generate libpng.pc, zlib.pc if needed

comment:3 Changed 22 months ago by dimpase

  • Cc fbissey embray isuruf added
  • Status changed from new to needs_info

The biggest question I here is how to find the full path to libz located by AC_CHECK_LIB or other similar macro. This path has to be written into zlib.pc.

I don't quite know: I can link an executable with libz and analyse it with readelf or otool or a similar thing, but it's not platform independent, and I could not find a ready-made macro (I could change AC_CHECK_LIB, sure...)

comment:4 Changed 22 months ago by dimpase

  • Authors set to Dima Pasechnik
  • Branch set to u/dimpase/packages/missingpcs
  • Commit set to 73d06199a6be95858da159036b15e6c99f57b61a

New commits:

ad511a1simplify some spkg-configures (use new macro)
73d0619refactor zlib's spkg-configure, start on a macro for zlib.pc

comment:5 Changed 22 months ago by isuruf

You can use a minimal pkgconfig file like below which doesn't require the full path to the library.

Name: Foo
Description: description for foo
Version: 1.0.0
Libs: -lfoo

comment:6 follow-up: Changed 22 months ago by dimpase

well, a minimal pkgconfig like this would be just a bug in waiting to bite.

I can think of one platform-independent way (well, it needs dladdr, which is non-POSIX, but BSD, something that is available on most Unices, certainly on Linux and MacOS)

Write a C function that calls dlopen() on the library in question, and then calls dladdr() on a symbol from the library; one of parts of the returned data is the full path to the shared object where the symbol is located. That's a hell of a macro call to AC_RUN_IFELSE though :-)

comment:7 Changed 22 months ago by dimpase

  • Status changed from needs_info to needs_work

comment:8 follow-up: Changed 22 months ago by dimpase

  • Cc mkoeppe added

Could someone please try if the following works on MacOS, and if yes, what's the output

/* save into zt.c and run as follows:
  $ cc -o zt zt.c -ldl && ./zt

  On my Debian Linux box it outputs
  /usr/lib/x86_64-linux-gnu/libz.so
*/
#include <stdio.h>
#define __USE_GNU
#include <dlfcn.h>
#ifdef __APPLE__
#define EXT "dylib"
#else
#define EXT "so"
#endif
int main() {
void *z;
Dl_info  info;
int res;
void *ha;
z = dlopen("libz."EXT,  RTLD_LAZY); /* load zlib */
ha = dlsym(z, "inflateEnd");       /* get address of inflateEnd in libz */
res = dladdr(ha, &info);           /* get info for the function */
printf("%s\n", info.dli_fname);
return res;
}

comment:9 in reply to: ↑ 8 Changed 22 months ago by fbissey

Replying to dimpase:

Could someone please try if the following works on MacOS, and if yes, what's the output

fbissey@itsv-frb15 GitHub % uname -a
Darwin itsv-frb15.lan 19.2.0 Darwin Kernel Version 19.2.0: Sat Nov  9 03:47:04 PST 2019; root:xnu-6153.61.1~20/RELEASE_X86_64 x86_64
fbissey@itsv-frb15 GitHub % cc -o zt zt.c -ldl && ./zt
/usr/lib/libz.1.dylib

comment:10 Changed 22 months ago by isuruf

I suggest linking the executable zt to libz so that the correct shared library is loaded.

comment:11 Changed 22 months ago by git

  • Commit changed from 73d06199a6be95858da159036b15e6c99f57b61a to 6bb85c4d4866c0fa4cce7a40ec9d0017603513be

Branch pushed to git repo; I updated commit sha1. New commits:

6bb85c4implemented generating input data for zlib.pc

comment:12 Changed 22 months ago by git

  • Commit changed from 6bb85c4d4866c0fa4cce7a40ec9d0017603513be to 80a23ac3f187d4cc30c7618a2e6dd22d6c7beef2

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

80a23acimplemented generating zlib.pc

comment:13 Changed 22 months ago by dimpase

  • Status changed from needs_work to needs_review

This branch is so far only for zlib.pc

Any obvious things I missed?

Naturally, I'd appreciate it tested on MacOS.

For libpng.pc I'll probably refactor that, to make computing abstolute path to libpng/libz/libWhatever a macro.

comment:14 Changed 22 months ago by git

  • Commit changed from 80a23ac3f187d4cc30c7618a2e6dd22d6c7beef2 to ba7d9632b368790279a890e225eb98e37c678cf5

Branch pushed to git repo; I updated commit sha1. New commits:

2d4a1cdrefactor finding absolute path to dylib into macro
ba7d963do not try to recongnise libpng without pkg-config

comment:15 Changed 22 months ago by dimpase

  • Milestone changed from sage-9.0 to sage-9.1

this is a result - ugly as hell. I am not sure whether this should get in.

comment:16 Changed 22 months ago by dimpase

  • Milestone changed from sage-9.1 to sage-wishlist

comment:17 in reply to: ↑ 6 ; follow-up: Changed 22 months ago by embray

Replying to dimpase:

well, a minimal pkgconfig like this would be just a bug in waiting to bite.

Would it though? Why exactly do we require pkgconfig here at all? It's a convenience sure, but on systems that don't install .pc files for some libraries we can still get the required information another way...

I mean, if autoconf could detect a system zlib in the first place, then I don't see why would need the absolute path to the SO file itself. All that should be needed to link against that library are whatever compiler/linker flags (e.g. -lz) that autoconf used to detect usability of the library in the first place.

comment:18 in reply to: ↑ 17 Changed 22 months ago by dimpase

Replying to embray:

Replying to dimpase:

well, a minimal pkgconfig like this would be just a bug in waiting to bite.

Would it though? Why exactly do we require pkgconfig here at all? It's a convenience sure, but on systems that don't install .pc files for some libraries we can still get the required information another way...

I am thinking of a library trying to detect zlib by calling pkg-config rather than using autoconf.

IMHO it's better not to install a pc file at all rather than install a rather bogus one. That's why I prefer how it's done on #28883.

I mean, if autoconf could detect a system zlib in the first place, then I don't see why would need the absolute path to the SO file itself. All that should be needed to link against that library are whatever compiler/linker flags (e.g. -lz) that autoconf used to detect usability of the library in the first place.

comment:19 Changed 22 months ago by embray

It's not necessarily "bogus". But I agree--I think what you did in #28883 is more in line with what I was trying to say here.

comment:20 follow-up: Changed 22 months ago by isuruf

I don't see the difference in #28883 and a minimal .pc file as in https://trac.sagemath.org/ticket/28906#comment:5 . They do the same thing.

comment:21 in reply to: ↑ 20 Changed 22 months ago by dimpase

Replying to isuruf:

I don't see the difference in #28883 and a minimal .pc file as in https://trac.sagemath.org/ticket/28906#comment:5 . They do the same thing.

yes, but they do so by different means, and a pc file is much more likely to trip up someone.

comment:22 Changed 19 months ago by mkoeppe

  • Status changed from needs_review to needs_work

I don't know if this ticket is still needed; but if it is, then it should be revised along the lines of #29082 -- do not write into SAGE_LOCAL at configure time; instead generate a rule in build/make/Makefile that generates the pc file there.

comment:23 Changed 19 months ago by dimpase

I don't recall right now all the details, why I thought it's needed. Anyhow, perhaps as a concept of how to generate *.pc files, e.g. for openblas.

Note: See TracTickets for help on using tickets.