Opened 10 years ago

Closed 10 years ago

Last modified 7 months ago

#8567 closed defect (fixed)

Change iconv so it builds on Cygwin and Solaris only

Reported by: drkirkby Owned by: drkirkby
Priority: major Milestone: sage-4.3.5
Component: build Keywords:
Cc: kcrisman Merged in: sage-4.3.5
Authors: David Kirkby Reviewers: Jaap Spies, Florent Hivert
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by drkirkby)

Since iconv was added to Sage (#8432) as a standard package (needed due to Karl-Dieter Crisman updating R in #6532 whilst overlooking the warning message from the configure script and without testing on Solaris), there have been some problems on Fedora and OpenSuse which look to be related to problems building gd when there are two copies of iconv to be found on the system. Since there is an option to gd's configure script specify the location of the iconv that seemed the most logical option. However, that did not work, so changing iconv to only build on Cygwin and Solaris seemed the most logical.

Dave

Attachments (1)

iconv.patch (1.8 KB) - added by drkirkby 10 years ago.
Mercurial patch to ensure iconv builds on Solaris and Cygwin only

Download all attachments as: .zip

Change History (43)

comment:1 follow-up: Changed 10 years ago by jsp

--with-libiconv-prefix=SAGE_LOCAL

does not work for me on Fedora 12 64 bit.

It was created by GD configure 2.0.35, which was
generated by GNU Autoconf 2.61.  Invocation command line was

  $ ./configure --prefix=/home/jaap/downloads/sage-4.3.4.rc0/local --without-jpeg --with-libiconv-prefix=/home/jaap/downloads/sage-4.3.4.rc0/local --without-x --with-zlib=/home/jaap/downloads/sage-4.3.4.rc0/local --with-freetype=/home/jaap/downloads/sage-4.3.4.rc0/local --without-xpm

gcc -fPIC -g -I/home/jaap/downloads/sage-4.3.4.rc0/local/include -I/home/jaap/downloads/sage-4.3.4.rc0/local/include/freetype2/ -Wl,--rpath -Wl,/home/jaap/downloads/sage-4.3.4.rc0/local/lib -o .libs/annotate annotate.o  -L/home/jaap/downloads/sage-4.3.4.rc0/local/lib ./.libs/libgd.so -lfontconfig /home/jaap/downloads/sage-4.3.4.rc0/local/lib/libfreetype.so /home/jaap/downloads/sage-4.3.4.rc0/local/lib/libpng12.so -lz -lm -Wl,--rpath -Wl,/home/jaap/downloads/sage-4.3.4.rc0/local/lib
./.libs/libgd.so: undefined reference to `libiconv'
./.libs/libgd.so: undefined reference to `libiconv_close'
./.libs/libgd.so: undefined reference to `libiconv_open'
collect2: ld returned 1 exit status

Jaap

comment:2 follow-up: Changed 10 years ago by jsp

moving /usr/bin/iconv and /usr/include/iconv.h out of the way made it possible to build gd

So nothing wrong with the libiconv in SAGE_LOCAL

Jaap

comment:3 in reply to: ↑ description ; follow-ups: Changed 10 years ago by hivert

Replying to drkirkby:

Since iconv was added to Sage (#8432) as a standard package (needing due to someone updating R without checking it fully), there have been some problems on Fedora and OpenSuse which look to be related to problems building gd when there are two copies of iconv to be found on the system.

First of all I have the same problem on Gentoo. Ther is also a report of someone having the same problem on Mandriva.

I manage to get it work by the following workaround: I replaced

$MAKE

by

LIBICONV="$SAGE_LOCAL"/lib/libiconv.so $MAKE

in spkg-install. I've no idea how robust this workaround is. I posted a spkg with the fix.

Florent

comment:4 Changed 10 years ago by jhpalmieri

By the way, in spkg-install, I think it should say "--with-x=no" rather than "--without-x", and the same for jpeg and xpm.

comment:5 in reply to: ↑ 3 Changed 10 years ago by hivert

Replying to hivert:

First of all I have the same problem on Gentoo. Ther is also a report of someone having the same problem on Mandriva.

I manage to get it work by the following workaround: I replaced

$MAKE

by

LIBICONV="$SAGE_LOCAL"/lib/libiconv.so $MAKE

in spkg-install. I've no idea how robust this workaround is. I posted a spkg with the fix.

Sorry ! This spkg is completely broken... I'm trying to put a working one.

Florent

comment:6 follow-up: Changed 10 years ago by jsp

Don't put an spkg as attachment! Please give a link to the spkg.

Jaap

comment:7 in reply to: ↑ 6 Changed 10 years ago by hivert

Replying to jsp:

Don't put an spkg as attachment! Please give a link to the spkg.

Jaap

Oops sorry ! My workaround does work actually. I find out that python distutils have the same problems much later...

comment:8 Changed 10 years ago by jhpalmieri

This doesn't help on these systems (output from uname -a):

Linux eno 2.6.31.12-174.2.22.fc12.x86_64 #1 SMP Fri Feb 19 18:55:03 UTC 2010 x86_64 x86_64 x86_64 GNU/Linux
Linux lena 2.6.31.12-174.2.19.fc12.x86_64 #1 SMP Thu Feb 11 07:07:16 UTC 2010 x86_64 x86_64 x86_64 GNU/Linux

I also don't know why it's about 50% bigger than the previous spkg.

comment:9 Changed 10 years ago by jsp

In Fedora 12 x86_64 the installation of libiconv is not complete:

[root@vrede jaap]# find /usr/bin -name "*iconv*"
/usr/bin/iconv.bak
/usr/bin/piconv
[root@vrede jaap]# find /usr/include -name "*iconv*"
/usr/include/evolution-data-server-2.28/camel/camel-iconv.h
/usr/include/evolution-data-server-2.28/libedataserver/e-iconv.h
/usr/include/Qt3Support/q3iconview.h
/usr/include/c++/4.4.3/gnu/gcj/convert/Output_iconv.h
/usr/include/c++/4.4.3/gnu/gcj/convert/Input_iconv.h
/usr/include/gtk-2.0/gtk/gtkiconview.h
/usr/include/Qt/q3iconview.h
/usr/include/xulrunner-sdk-1.9.1/system_wrappers_js/iconv.h
/usr/include/xulrunner-sdk-1.9.1/system_wrappers/iconv.h
/usr/include/iconv.h.bak
[root@vrede jaap]# find /usr/lib -name "*iconv*"
/usr/lib/ruby/1.8/xsd/iconvcharset.rb
[root@vrede jaap]# 

Jaap

comment:10 Changed 10 years ago by hivert

I maybe manage to have something working by a hack. Could someone try the following spkg: http://sage.math.washington.edu/home/hivert/spkg/gd-2.0.35.p4.spkg

comment:11 Changed 10 years ago by jhpalmieri

That appears to work on the two systems I mentioned before. I'll know for sure when a full Sage build is completed...

comment:12 in reply to: ↑ 3 Changed 10 years ago by drkirkby

Replying to hivert:

Replying to drkirkby:

Since iconv was added to Sage (#8432) as a standard package (needing due to someone updating R without checking it fully), there have been some problems on Fedora and OpenSuse which look to be related to problems building gd when there are two copies of iconv to be found on the system.

First of all I have the same problem on Gentoo. Ther is also a report of someone having the same problem on Mandriva.

I manage to get it work by the following workaround: I replaced

$MAKE

by

LIBICONV="$SAGE_LOCAL"/lib/libiconv.so $MAKE

in spkg-install. I've no idea how robust this workaround is. I posted a spkg with the fix.

Florent

I've got no idea if setting LIBICOV shoudl be set, but if it is, the way to do it should be:

LIBICONV="$SAGE_LOCAL/lib/libiconv.so"

and not

LIBICONV="$SAGE_LOCAL"/lib/libiconv.so

Dave

comment:13 in reply to: ↑ 1 Changed 10 years ago by drkirkby

Replying to jsp:

--with-libiconv-prefix=SAGE_LOCAL

does not work for me on Fedora 12 64 bit.

Jaap

Jaap,

is that a typo in the trac ticket, or was it in the package? There should be a $ sign in front if SAGE_LOCAL.

comment:14 Changed 10 years ago by drkirkby

  • Cc kcrisman added
  • Description modified (diff)
  • Owner changed from GeorgSWeber to drkirkby

If this gets too much of a mess, we could consider creating a ticket to get the R package reverted to the older version. I've cc'ed Karl-Dieter Crisman who updated R, whilst ignoring the warning from the configure script and not testing on Solaris.

Hopefully iconv can be sorted out, but if it can't be done properly, then reverting the R version is always an option, then iconv will not be needed. We could change iconv's spkg-install to simply exit with an exit code of 0. That would make it relatively easy to do, whilst leaving iconv in Sage to allow it to be sorted out properly prior to updating R.

Dave

comment:15 follow-up: Changed 10 years ago by jhpalmieri

For me, the Sage build completed, but the documentation won't build because Sage doesn't start successfully:

Traceback (most recent call last):
  File "/home/palmieri/lena/sage-4.3.4.alpha1/local/bin/sphinx-build", line 6, in <module>
    import sage.all
  File "/home/palmieri/lena/sage-4.3.4.alpha1/local/lib/python2.6/site-packages/sage/all.py", line 73, in <module>
    from sage.matrix.all     import *
  File "/home/palmieri/lena/sage-4.3.4.alpha1/local/lib/python2.6/site-packages/sage/matrix/all.py", line 1, in <module>
    from matrix_space import MatrixSpace, is_MatrixSpace
  File "/home/palmieri/lena/sage-4.3.4.alpha1/local/lib/python2.6/site-packages/sage/matrix/matrix_space.py", line 40, in <module>
    import matrix_mod2_dense
ImportError: /home/palmieri/lena/sage-4.3.4.alpha1/local/lib/libgd.so.2: undefined symbol: libiconv

Running

export LIBICONV="$SAGE_LOCAL/lib/libiconv.so"

right before running Sage doesn't help.

comment:16 in reply to: ↑ 2 ; follow-up: Changed 10 years ago by drkirkby

Replying to jsp:

moving /usr/bin/iconv and /usr/include/iconv.h out of the way made it possible to build gd

So nothing wrong with the libiconv in SAGE_LOCAL

Jaap

I had a discussion with a libtool developer about a similar issue once before. He said it is wrong to have two copies of a library, so the problem may be more fundamental. However, it is strange gd ignores the option to look for iconv in $SAGE_LOCAL.

One disadvantage of reverting R and not installing iconv is that iconv is needed to build gd on Cygwin, and there is effort into porting Sage to Cygwin.

We could consider only installing iconv on Solaris and Cygwin, though I'm tempted to suggest reverting R and getting the problem solved properly at a later date, rather than a hack now.

Dave

comment:17 in reply to: ↑ 15 Changed 10 years ago by hivert

Replying to jhpalmieri:

For me, the Sage build completed, but the documentation won't build because Sage doesn't start successfully:

I got the same behavior on openSuSE 11.1 and Gentoo 1.12.9 both x86_64. The build appears to be complete but sage itself is broken.

I don't know what to do and I've no more time to investigate. My vote is to revert R, remove gd and make as soon as possible a fix release until this is sorted out. We can't seriously make a release which is broken on half the distro.

comment:18 in reply to: ↑ 16 Changed 10 years ago by jsp

Replying to drkirkby:

I had a discussion with a libtool developer about a similar issue once before. He said it is wrong to have two copies of a library, so the problem may be more fundamental. However, it is strange gd ignores the option to look for iconv in $SAGE_LOCAL.

I'm not so sure anymore I didn't make another typo in spkg-install.

I'll try again.

Jaap

comment:19 follow-ups: Changed 10 years ago by drkirkby

Another way to solve the fallout from updating R, will probably be to add

if [ "x$UNAME" != xSunOS ] && [ "x$UNAME" != xCYGWIN ] then ; 
  exit 0
fi

to the top of the *iconv* spkg-install file. That's probably the simplest hack. This is untested and I don't have time to do this until around 2100 GMT today, so if someone beats me to it, go ahead and try it. Make sure it is tested on both Solaris and Linux. It should build iconv on Solaris, but not on Linux

Dave

comment:20 Changed 10 years ago by jsp

About iconv(): according to the man pages the function is part of glibc

VERSIONS
       This function is available in glibc since version 2.1.

CONFORMING TO
       POSIX.1-2001.

SEE ALSO
       iconv_close(3), iconv_open(3)


So for most systems there is no need to have a local libiconv! Let's say only Solaris and Cygwin.

Jaap

comment:21 in reply to: ↑ 19 Changed 10 years ago by jsp

Replying to drkirkby:

Another way to solve the fallout from updating R, will probably be to add

if [ "x$UNAME" != xSunOS ] && [ "x$UNAME" != xCYGWIN ] then ; 
  exit 0
fi

This probably will cause trouble on Open Solaris.

Jaap

comment:22 in reply to: ↑ 19 ; follow-up: Changed 10 years ago by GeorgSWeber

Replying to drkirkby:

Another way to solve the fallout from updating R, will probably be to add

if [ "x$UNAME" != xSunOS ] && [ "x$UNAME" != xCYGWIN ] then ; 
  exit 0
fi

to the top of the *iconv* spkg-install file. That's probably the simplest hack. This is untested and I don't have time to do this until around 2100 GMT today, so if someone beats me to it, go ahead and try it. Make sure it is tested on both Solaris and Linux. It should build iconv on Solaris, but not on Linux

Dave

Yes, I definitely prefer this way, too. The OpenSolaris? topic may be solved later (what does $UNAME give there as output?), and it might be nice to remove any leftovers in "$SAGE_ROOT/bin/" from earlier (especially v3.4.3) Sage installs. Which might be tricky in a "Sage upgrade" use case. But never mind, the road to go is clear now!

Cheers, Georg

comment:23 in reply to: ↑ 22 Changed 10 years ago by jsp

Replying to GeorgSWeber:

Replying to drkirkby:

Another way to solve the fallout from updating R, will probably be to add

if [ "x$UNAME" != xSunOS ] && [ "x$UNAME" != xCYGWIN ] then ; 
  exit 0
fi

Yes, I definitely prefer this way, too. The OpenSolaris? topic may be solved later (what does $UNAME give there as output?)

SunOS

Removing local/bin/iconv, local/lib/libiconv.* and local/include/iconv.h made the gd spkg work on Fedora 12.

Jaap

comment:24 follow-up: Changed 10 years ago by drkirkby

Jaap,

I don't have time to test it now (really am tied up), but if you can test on OpenSolaris, then make a decision whether iconv needs installing or not.

drkirkby@kestrel:~$ uname -r
5.10

on Solaris 10, and

drkirkby@hawk:~$ uname -r
5.11

on OpenSolaris (aka Solaris 11). So just put an additional clause if needed.

The -r option to uname is POSIX, so we can rely on that. Don't use 'uname -p' since

  • The i386 processor is used on Solaris 10 too.
  • -p is not a POSIX option to uname, so can cause hassles on other systems.

Also worth checking, in preference to this are whether the version of 'gd' in Sage is old and furthermore the 'gd' developers have fixed the bug that prevents '--with-libiconv-prefix' working properly. If so, consider updating gd.

Had R been properly tested, none of this would have been necessary.

When I wrote the iconv package, it clears out debris, like libraries and binaries, so there should be no need to do that again - just make sure the cleaning up is done before exiting.

Since Jaap and I are looking to port Sage to OpenSolaris, we are not keen to introduce more problems for the future. Let's sort it out now.

Dave

comment:25 in reply to: ↑ 24 Changed 10 years ago by jsp

Replying to drkirkby:

Jaap,

I don't have time to test it now (really am tied up), but if you can test on OpenSolaris, then make a decision whether iconv needs installing or not.

Opensolaris has it's own iconv included in glibc.

on OpenSolaris (aka Solaris 11). So just put an additional clause if needed.

Yes we should put an extra test in

The -r option to uname is POSIX, so we can rely on that. Don't use 'uname -p' since

Ok.

  • The i386 processor is used on Solaris 10 too.
  • -p is not a POSIX option to uname, so can cause hassles on other systems.

Also worth checking, in preference to this are whether the version of 'gd' in Sage is old and furthermore the 'gd' developers have fixed the bug that prevents '--with-libiconv-prefix' working properly. If so, consider updating gd.

Let's go for the proposed option and leave alone the gd package for the moment.

Had R been properly tested, none of this would have been necessary.

You are repeating your self :-)!

My proposal is you update the iconv spkg and I and evt. others review it.

I think we can wait some more hours.

Jaap

comment:26 in reply to: ↑ 19 ; follow-up: Changed 10 years ago by jhpalmieri

Replying to drkirkby:

Another way to solve the fallout from updating R, will probably be to add

if [ "x$UNAME" != xSunOS ] && [ "x$UNAME" != xCYGWIN ] then ; 
  exit 0
fi

to the top of the *iconv* spkg-install file.

Informally (that is, I threw together a quick spkg), this works on at least one of the linux boxes I had troubles with before, and it also works on sage.math. I haven't tested it on Solaris, and I probably won't have time to; I don't have access to a Windows/Cygwin? machine. I would suggest that the spkg-install file actually say

if [ "x$UNAME" != xSunOS ] && [ "x$UNAME" != xCYGWIN ] ; then
  echo "Not installing iconv -- on systems other than Solaris and Cygwin, use the system's iconv"
  exit 0
fi

Or something like that -- I'm not sure exactly what the message should say.

comment:27 in reply to: ↑ 26 Changed 10 years ago by hivert

Replying to jhpalmieri:

Replying to drkirkby:

Another way to solve the fallout from updating R, will probably be to add

if [ "x$UNAME" != xSunOS ] && [ "x$UNAME" != xCYGWIN ] then ; 
  exit 0
fi

The ";" should be before "then".. .

if [ "x$UNAME" != xSunOS ] && [ "x$UNAME" != xCYGWIN ] ; then
  echo "Not installing iconv -- on systems other than Solaris and Cygwin, use the system's iconv"
  exit 0
fi

Or something like that -- I'm not sure exactly what the message should say.

I'm trying this on gentoo and it seems to work... I'm waiting for the build to finish.

Florent

comment:28 Changed 10 years ago by drkirkby

  • Authors set to David Kirkby
  • Description modified (diff)
  • Summary changed from Specify the location of iconv for 'gd' to Change iconv so it builds on Cygwin and Solaris only

I've change the title and description and are in the process of creating a package for others to review.

comment:30 in reply to: ↑ 29 Changed 10 years ago by jsp

Replying to drkirkby:

I've stuck a package here:

http://sage.math.washington.edu/home/kirkby/iconv/iconv-1.13.1.p0.spkg

I'll try it asap.

Jaap

Changed 10 years ago by drkirkby

Mercurial patch to ensure iconv builds on Solaris and Cygwin only

comment:31 Changed 10 years ago by drkirkby

  • Status changed from new to needs_review

I've not tested this extensively, but it builds on Solaris and hopefully does not on Linux. I don't have time to do much testing, so others can.

I don't have a Cygwin system set up.

comment:32 Changed 10 years ago by jsp

Removing old iconv files if they exist iconv will not be installed, as it is only installed on Solaris and Cygwin - see: http://trac.sagemath.org/sage_trac/ticket/8567

real 0m0.078

gcc version 4.4.3 20100127 (Red Hat 4.4.3-4) (GCC) 
****************************************s
user	0m0.008s
sys	0m0.015s
Successfully installed iconv-1.13.1.p0

echo "Removing old iconv files if they exist"
# If iconv is updated, please double-check these are still necessary
# and that there are no extra files added.

rm -f $SAGE_LOCAL/bin/iconv  $SAGE_LOCAL/lib/charset.alias
rm -f $SAGE_LOCAL/lib/*libiconv*  $SAGE_LOCAL/lib/libcharset*
rm -f $SAGE_LOCAL/include/iconv.h  $SAGE_LOCAL/include/libcharset.h
rm -f $SAGE_LOCAL/include/localcharset.h
rm -rf  $SAGE_LOCAL/share/doc/libiconv
rm -f $SAGE_LOCAL/share/man/man1/iconv* $SAGE_LOCAL/share/man/man3/iconv*


This looks ok for me on Fedora (Linux in general?)

The Opensolaris issue is not resolved, but I think this deserves a positive review.

Jaap

comment:33 Changed 10 years ago by jsp

  • Reviewers set to Jaap Spies

comment:34 follow-up: Changed 10 years ago by hivert

  • Reviewers changed from Jaap Spies to Jaap Spies, Florent Hivert

With the spkg here, from a fresh Sage source:

  • iconv build is successfully skipped,
  • gd is successfully build

on openSuSE 11.1 and Gentoo 1.12.9. I'm waiting the whole build to finish, but it should be ok... I'll then agree with Jaap on positive review.

Thanks

comment:35 in reply to: ↑ 34 ; follow-up: Changed 10 years ago by jsp

Replying to hivert:

With the spkg here, from a fresh Sage source:

  • iconv build is successfully skipped,
  • gd is successfully build

on openSuSE 11.1 and Gentoo 1.12.9. I'm waiting the whole build to finish, but it should be ok... I'll then agree with Jaap on positive review.

Thanks

My build finished successful, so let's do it! I leave it to you to give the positive review.

Jaap

comment:36 in reply to: ↑ 35 ; follow-up: Changed 10 years ago by hivert

Replying to jsp:

My build finished successful, so let's do it! I leave it to you to give the positive review.

Mine too on openSuSE and gentoo. I'm ready to give positive review but should it be tested on Cygwin ? Unfortunately, I've no access to this kind of machine.

Cheers,

Florent

comment:37 in reply to: ↑ 36 Changed 10 years ago by jsp

Replying to hivert:

Replying to jsp:

My build finished successful, so let's do it! I leave it to you to give the positive review.

Mine too on openSuSE and gentoo. I'm ready to give positive review but should it be tested on Cygwin ? Unfortunately, I've no access to this kind of machine.

Cheers,

Florent

Please do! There is no change in SunOS and Cygwin. In both cases iconv spkg will be build as before.

Jaap

comment:38 Changed 10 years ago by hivert

  • Status changed from needs_review to positive_review

comment:39 Changed 10 years ago by was

  • Resolution set to fixed
  • Status changed from positive_review to closed

Merged into sage-4.3.5.

comment:40 Changed 10 years ago by mvngu

  • Merged in set to sage-4.3.5

comment:41 Changed 10 years ago by mvngu

  • Milestone changed from sage-4.4 to sage-4.3.5

comment:42 Changed 10 years ago by bascorp2

[deleted spam]

Last edited 7 months ago by embray (previous) (diff)
Note: See TracTickets for help on using tickets.