Opened 12 years ago

Last modified 12 years ago

#9530 closed defect

Improve/fix readline workarounds for Arch Linux and openSuSE, again broken on OpenSuSE 11.2 and 11.3 — at Version 24

Reported by: Thomas Bächler Owned by: Georg S. Weber
Priority: blocker Milestone: sage-4.6
Component: build Keywords: Arch Linux SuSE readline
Cc: Leif Leonhardy, David Kirkby, Mitesh Patel, Florent Hivert Merged in:
Authors: Thomas Bächler Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description (last modified by Mitesh Patel)

Sage currently has workarounds against building an incompatible libreadline on openSuSE 11.1 and Arch Linux - instead of building it, the system version is copied.

The SuSE workaround should be applied to openSuSE 11.2 and 11.3, too. Furthermore, SuSE offers libreadline.so.5 in addition to .so.6, so we only copy .6.

The workaround on Arch Linux is unreliable, as it detects Arch Linux by "-ARCH" in the kernel version (which is the default, but might be different). Instead, the presence of /etc/arch-release should be used to determine that.

I am attaching a patch that fixes those problems and changes 'cp' to 'cp -a' to not turn a symlink into a file. Without this patch, sage fails to build on my SuSE or Arch systems.

New spkg:

http://sage.math.washington.edu/home/mpatel/trac/9530/readline-6.0.p3.spkg

Change History (26)

Changed 12 years ago by Thomas Bächler

Attachment: readline-6.0.p2.spkg.patch added

comment:1 Changed 12 years ago by Carl Witty

Is it actually worth fixing the workarounds? It seems like upgrading readline (#9523) wouldn't be much more work, and then we could probably just remove the workarounds.

comment:2 Changed 12 years ago by Thomas Bächler

It might fix the problem for now. And when the system readline is upgraded again, everything is broken again.

I don't like that situation that sage wants to build its own version of libreadline - in my opinion it causes more problems than it solves. As long as sage wants to use its own readline version in favor of the system library, this problem will never fully go away.

comment:3 Changed 12 years ago by todd

What if you used symlinks to the native libreadline instead of copying it?

So instead of:

cp -a /lib64/libreadline.so.6* "$SAGE_LOCAL"/lib cp -a /lib/libreadline.so.6* "$SAGE_LOCAL"/lib cp -r /usr/include/readline "$SAGE_LOCAL"/include cp /lib/libreadline.so.* "$SAGE_LOCAL"/lib cp -a /lib/libreadline.so.6* "$SAGE_LOCAL"/lib

It does:

ln -s /lib64/libreadline.so.6* "$SAGE_LOCAL"/lib ln -s /lib/libreadline.so.6* "$SAGE_LOCAL"/lib ln -s /usr/include/readline "$SAGE_LOCAL"/include ln -s /lib/libreadline.so.* "$SAGE_LOCAL"/lib ln -s /lib/libreadline.so.6* "$SAGE_LOCAL"/lib

This should avoid the problem with upgrades changing libreadline again, and will save a little space.

comment:4 Changed 12 years ago by todd

The formatting got messed up.  Here is an example:

This:

cp -a /lib64/libreadline.so.6* "$SAGE_LOCAL"/lib

becomes:

ln -s /lib64/libreadline.so.6* "$SAGE_LOCAL"/lib

comment:5 Changed 12 years ago by Leif Leonhardy

Cc: Leif Leonhardy added

comment:6 Changed 12 years ago by todd

Since the update for readline has been pushed back to at least 4.6.1, can we get this workaround in 4.6? We have a patch that fixes the problem, is small, and is low-risk. With the switch from 4.5 to 4.6 I think it is important that it successfully compiles on significant distributions like openSUSE and Arch.

comment:7 Changed 12 years ago by Leif Leonhardy

Cc: David Kirkby Mitesh Patel added
Status: newneeds_review

Though the patch is not a Mercurial changeset...

If we keep using the system's readline on OpenSuSE 11[.1], we could incorporate the patch into #9523 (upgrade to readline 6.1).

comment:8 Changed 12 years ago by todd

I was thinking this made more as temporary workaround until 9523 render the workarounds unnecessary.

I should add that I tried this patch and it worked.

comment:9 Changed 12 years ago by Leif Leonhardy

Cc: Florent Hivert added

Well, the milestone of #9523 has been set to Sage 4.6.1.

Also, it's not clear to me if we should use the system's readline on OpenSuSE 11.1, and Sage's new readline 6.1 on OpenSuSE >11.1.

comment:10 in reply to:  9 ; Changed 12 years ago by todd

Replying to leif:

Well, the milestone of #9523 has been set to Sage 4.6.1.

Yes, that is the problem. As it stand now, Sage 4.6 will be released without being able to build as-is on a major distribution and won't build reliably on another. We have a small, simple, low-risk fix available right now. I think it would be good to incorporate the fix now rather than pushing it back again.

I think having it build successfully on the current version of major distributions would be a worthwhile goal for 4.6. Optimizing that could wait until later.

And that is assuming #9523 isn't bushed back again. It was originally a 4.6 blocker.

comment:11 in reply to:  10 ; Changed 12 years ago by Leif Leonhardy

Replying to TheBlackCat:

Replying to leif:

Well, the milestone of #9523 has been set to Sage 4.6.1.

Yes, that is the problem. As it stand now, Sage 4.6 will be released without being able to build as-is on a major distribution and won't build reliably on another. We have a small, simple, low-risk fix available right now. I think it would be good to incorporate the fix now rather than pushing it back again.

That's what I meant or intended: include #9530 into 4.6 (which requires a positive review), but also merge (some of) the changes here into #9530.

comment:12 in reply to:  11 Changed 12 years ago by Leif Leonhardy

Replying to leif:

... include #9530 into 4.6 (which requires a positive review)

... and also a patched new spkg. I do not even know the name of the author, baechler.

comment:13 in reply to:  10 ; Changed 12 years ago by David Kirkby

Replying to TheBlackCat:

Replying to leif:

Well, the milestone of #9523 has been set to Sage 4.6.1.

Yes, that is the problem. As it stand now, Sage 4.6 will be released without being able to build as-is on a major distribution and won't build reliably on another. We have a small, simple, low-risk fix available right now. I think it would be good to incorporate the fix now rather than pushing it back again.

Despite trying - see for example

http://wiki.sagemath.org/suggested-for-supported-platforms

I've never managed to get any agreement on what is "supported". But it we believe README.txt, both ArchLinux and !OpenSUSE are supported. However, in my opinion we should not hold a release of Sage just because on Linux distro has bought at yet another backwards incompatible version. But if a change in Sage has broken one of these systems, then I think it should be addressed in 4.6. But if the breakages are caused by updated versions of Linux, then I think these systems will just have to accept that.

comment:14 in reply to:  13 ; Changed 12 years ago by Thomas Bächler

Replying to drkirkby:

However, in my opinion we should not hold a release of Sage just because on Linux distro has bought at yet another backwards incompatible version.

You don't understand the problem at all. Sage overrides the system libreadline with an older version, one that lacks features that the system shell requires. This is a problem that affects EVERY Linux distribution that is not terribly outdated (if I would say "it doesn't build on Ubuntu", you would probably suddenly take this seriously, but as long as it works on Ubuntu, it can't be not broken, right? It's not Ubuntu, so I have to keep repeating myself, until the problem eventually also affects Ubuntu)

As long as sage does not build its own shell for the build process, overriding the system libreadline is simply plain stupid, and will keep breaking again and again. In my opinion, sage should not ship its own libreadline if the system already provides readline version 6 or newer - eliminating all the mentioned problems.

Replying to leif:

... and also a patched new spkg. I do not even know the name of the author, baechler.

Since when does that matter? If you must know: http://wwwb.math.rwth-aachen.de/Mitarbeiter/baechler.php?lang=eng http://www.archlinux.org/developers/#thomas

comment:15 Changed 12 years ago by todd

Replying to drkirkby:

Yes, that is the problem. As it stand now, Sage 4.6 will be released without being able to build as-is on a major distribution and won't build reliably on another. We have a small, simple, low-risk fix available right now. I think it would be good to incorporate the fix now rather than pushing it back again.

I've never managed to get any agreement on what is "supported". But it we believe README.txt, both ArchLinux and !OpenSUSE are supported. However, in my opinion we should not hold a release of Sage just because on Linux distro has bought at yet another backwards incompatible version. But if a change in Sage has broken one of these systems, then I think it should be addressed in 4.6. But if the breakages are caused by updated versions of Linux, then I think these systems will just have to accept that.

I agree we should not hold a release because of the lack of a fix for a new break caused by the distribution. However, neither is the case here.

The breakage first appeared in openSUSE 11.2, which was released almost a year ago. openSUSE 11.3, which also has the problem, was released over 3 months ago. So it is a long-standing issue.

Further, a fix is available, and has been for over 3 months now (shortly after openSUSE 11.3 was released). The fix was published several weeks before sage 4.5.2 was released, over a month and a half before 4.5.3.

So this is a long-standing issue for which a fix has been available for a while. Given that, I do not think holding 4.6 until the existing fix has been incorporated is outlandish.

comment:16 in reply to:  14 ; Changed 12 years ago by Leif Leonhardy

Replying to baechler:

Replying to leif:

... and also a patched new spkg. I do not even know the name of the author, baechler.

Since when does that matter?

It's just because we put the real names into the Authors field of the ticket, the changelogs of the spkgs and usually also into the Mercurial change sets.

If you must know: http://wwwb.math.rwth-aachen.de/Mitarbeiter/baechler.php?lang=eng http://www.archlinux.org/developers/#thomas

You can add your name to http://trac.sagemath.org/sage_trac/wiki#AccountNamesmappedtoRealNames

If you'd been listed there, I would perhaps have created a new spkg from your patch.

Replying to TheBlackCat:

Further, a fix is available, and has been for over 3 months now (shortly after openSUSE 11.3 was released). The fix was published several weeks before sage 4.5.2 was released, over a month and a half before 4.5.3.

Unfortunately, the ticket hadn't been set to "needs review", there are apparently only a few Sage developers dealing with OpenSuSE, and there hasn't been activity on the ticket for a while.

comment:17 in reply to:  14 Changed 12 years ago by David Kirkby

Replying to baechler:

Replying to drkirkby:

However, in my opinion we should not hold a release of Sage just because on Linux distro has bought at yet another backwards incompatible version.

You don't understand the problem at all. Sage overrides the system libreadline with an older version, one that lacks features that the system shell requires. This is a problem that affects EVERY Linux distribution that is not terribly outdated (if I would say "it doesn't build on Ubuntu", you would probably suddenly take this seriously, but as long as it works on Ubuntu, it can't be not broken, right? It's not Ubuntu, so I have to keep repeating myself, until the problem eventually also affects Ubuntu)

I suggest you do some homework on what operating systems I use, then you might draw a different conclusion about me and Ubunta.

Dave

Dave

comment:18 in reply to:  16 ; Changed 12 years ago by todd

Replying to leif:

Replying to TheBlackCat:

Further, a fix is available, and has been for over 3 months now (shortly after openSUSE 11.3 was released). The fix was published several weeks before sage 4.5.2 was released, over a month and a half before 4.5.3.

Unfortunately, the ticket hadn't been set to "needs review", there are apparently only a few Sage developers dealing with OpenSuSE,

That indicates that something is wrong with the patch handling process. It is one thing if developers do not fix a problem because they cannot reproduce it. But the patch was available and just sat here. Further, it is a patch for a "major" issue. Shouldn't such patches receive particular attention from developers?

and there hasn't been activity on the ticket for a while.

Yes, that is exactly the problem. Why did did it take this long for such a simple patch for a major issue to receive any attention? If I hadn't posted the comment, how long would it have taken for developers to act on it?

comment:19 in reply to:  18 Changed 12 years ago by Leif Leonhardy

Replying to TheBlackCat:

Replying to leif:

Unfortunately, the ticket hadn't been set to "needs review", there are apparently only a few Sage developers dealing with OpenSuSE,

That indicates that something is wrong with the patch handling process.

If somebody provides a patch, the status should be changed from "new" to "needs review" (by the author / the one that uploaded a patch).

It is one thing if developers do not fix a problem because they cannot reproduce it. But the patch was available and just sat here. Further, it is a patch for a "major" issue.

"major" is the default priority. It should perhaps have been at least critical. Also, the title doesn't suggest higher importance (it's "improve...", which doesn't sound like fixing a major bug).

Shouldn't such patches receive particular attention from developers?

It's more likely that people search for tickets e.g. needing review, or "new" ones that are marked "crtitical" or "blocker".

and there hasn't been activity on the ticket for a while.

Yes, that is exactly the problem. Why did did it take this long for such a simple patch for a major issue to receive any attention? If I hadn't posted the comment, how long would it have taken for developers to act on it?

See above. Posting just "ping" on the ticket also causes attention by those people involved (owner, reporter, cc'ed people and everyone who commented on a ticket, at least if they have e-mail notification enabled, which should be the case for most, if not all).

Also, I haven't seen recent build [failure] reports regarding OpenSuSE and that issue on sage-release, which is another way to bring back attention.

Florent's post on sage-release (and incidentally? Dave's activity on #9530) brought me back to this.

comment:20 Changed 12 years ago by Leif Leonhardy

comment:21 Changed 12 years ago by Leif Leonhardy

Priority: majorcritical
Summary: Improve/fix readline workarounds for Arch Linux and openSuSEImprove/fix readline workarounds for Arch Linux and openSuSE, again broken on OpenSuSE 11.2 and 11.3

comment:22 Changed 12 years ago by Leif Leonhardy

Work issues: Provide a readline-6.0.p3 spkg with the patch applied

comment:23 Changed 12 years ago by Mitesh Patel

Would a readline-6.0.p3.spkg with the attached patch definitely fix the broken builds on Arch Linux and/or openSUSE, at least in the short term? I'm willing to merge this into 4.6.

comment:24 Changed 12 years ago by Mitesh Patel

Authors: Thomas Bächler
Description: modified (diff)
Work issues: Provide a readline-6.0.p3 spkg with the patch applied

See the description for a link to a p3 spkg with Thomas' patch. I'll attach a Mercurial spkg patch shortly. Note: I have not tested the new package. Please do this!

Changed 12 years ago by Mitesh Patel

SPKG Mercurial patch of changes from p2 to p3

Note: See TracTickets for help on using tickets.