Opened 6 years ago

Last modified 6 years ago

#20190 closed enhancement

Upgrade R to 3.2.4 — at Version 23

Reported by: charpent Owned by:
Priority: minor Milestone: sage-7.2
Component: packages: standard Keywords: r-project
Cc: leif, jdemeyer, vbraun Merged in:
Authors: Emmanuel Charpentier Reviewers:
Report Upstream: N/A Work issues:
Branch: u/charpent/upgrade_r_to_3_2_4 (Commits, GitHub, GitLab) Commit: 385c51d8fc95817eec10780fa40723d9f02865e0
Dependencies: Stopgaps:

Status badges

Description (last modified by charpent)

It's this time of the year again...

Original tarball is here : https://cran.r-project.org/src/base/R-3/R-3.2.4.tar.gz.

"Revised" tarball (long story, see below) here.

Change History (26)

comment:1 Changed 6 years ago by charpent

  • Component changed from PLEASE CHANGE to packages: standard

comment:2 Changed 6 years ago by charpent

  • Branch set to u/charpent/upgrade_r_to_3_2_4

comment:3 Changed 6 years ago by charpent

  • Authors set to Emmanuel Charpentier
  • Commit set to 385c51d8fc95817eec10780fa40723d9f02865e0
  • Status changed from new to needs_review

Passes ptestlong with no error nor glitch ==> needs_review

Changed 6 years ago by tscrim

comment:4 Changed 6 years ago by tscrim

  • Status changed from needs_review to needs_work

I could not build it on Ubuntu 14.04 LTS by applying this on top of 7.1.rc0 and running make build. Log is attached.

My guess is it is related to the fuzz from applying the patches.

comment:5 Changed 6 years ago by tscrim

  • Cc leif jdemeyer vbraun added

From building old versions of R, those logs say the command should be rm -f liblzma.a (which it what I was thinking it should be). I suspect it has to do with the dont_lose_libz_libbz2_in_configure.patch, but I'm not sure what to look for. Do one of you know what needs to be done (and do you also get a build failure)?

comment:6 Changed 6 years ago by charpent

Works for me on Debian (three instances of testing/amd64 w. 6, 7 and 16 Gb RAM : 7 Gb seems the bare acceptable minimum to install/run rstan under R (The 6 Gb machine starts to swap/trash on this one...)

I'll have a look tonight (maybe tomorrow), but on a smallish machine (Ubuntu 15.10/amd64) not well suited to debuging sage (4 GB RAM...).

HTH,

comment:7 Changed 6 years ago by tscrim

FYI - I just did a make distclean && make build (with SAGE_NUM_THREADS=8 and MAKE='make -j8') and I got the same problem.

Changed 6 years ago by charpent

Compilation log on my (charpent's) machine

comment:8 Changed 6 years ago by charpent

Compilation on my machine(s) does not follow the same path as on yours. My compilation never reaches the point where yours goes.

I do not (yet) understand qwhy. Different installed software ?

HTH,

comment:9 Changed 6 years ago by tscrim

I believe it has to do with (lib)lzma being present on the system (FYI, I have a Core i7). In particular, your log has

checking for lzma_version_number in -llzma... yes

whereas my in no. So it probably has to do with dont_lose_libz_libbz2_in_configure.patch and changes in the configure script for R (there was similar fuzz when building R-3.2.3.p1 on my machine as well).

Although I have no idea what to look for on how to fix this, but perhaps this will allow you to reproduce my build error?

comment:10 Changed 6 years ago by tscrim

Were you able to reproduce the build failure?

comment:11 follow-up: Changed 6 years ago by charpent

Eureka ! (maybe...)

Sorry for the delay : my schedule is a little bit busy nowadays...

I found this :

charpent@SAP5057241:~/Temporaire/DemeRde$ diff -ur R-3.2.3/src/extra/xz R-3.2.4/src/extra/xz
diff -ur R-3.2.3/src/extra/xz/Makefile.in R-3.2.4/src/extra/xz/Makefile.in
--- R-3.2.3/src/extra/xz/Makefile.in	2010-11-09 00:05:05.000000000 +0100
+++ R-3.2.4/src/extra/xz/Makefile.in	2015-12-11 00:15:12.000000000 +0100
@@ -58,8 +58,8 @@
 	touch stamp
 
 liblzma.a: $(liblzma_a_OBJECTS)
-	rm -f $@
-	$(AR) cr $@ $(liblzma_a_OBJECTS)
+	$rm -f $@
+	$(AR) -cr $@ $(liblzma_a_OBJECTS)
 	$(RANLIB) $@
 
 mostlyclean: clean

I'll ask around in the R mailing list to know what was intended by this (highly suspicious) change.

Do you want an (interim) patch to revert it ?

comment:12 in reply to: ↑ 11 ; follow-up: Changed 6 years ago by charpent

Replying to myself :

This is indeed an upstream bug. See the r-devel thread starting at this posting : it's ultimately a Makefile typo.

However, this thread says that :

Notice that in 3.3.x, the included xz & al. will disappear.

This might mean that the prerequisites for R may change at 3.3.0 time (which was announced to take place on 2016-04-11 last time I looked...).

Furthermore, r-devel also had this announcement (that I missed) :

The 3.2.4 release had two annoyances which we would rather not have in an "ultra-stable" release, designed to hang around for the duration of the 3.3 series. One was a relatively minor Makefile issue affecting system using R's bundled lzma library. The other, rather more serious, affected printing and formatting of POSIXlt objects, which would unpredictably get the Daylight Savings Time wrong.

A new tarball fixing this issue in 3.2.4 (+ another relevant to Posix time printing) has been released as http://cran.r-project.org/src/base/R-3/R-3.2.4-revised.tar.gz.

So it seems to me that the best "interim" solution is :

  • to get the "revised" tarball from CRAN
  • to document its renaming in spkg-src

and to revise the whole situation at R 3.3.0 release.

Unless I hear contrary opinions before Sun April 3, 2016, I'll do that (and also fix the patches fuzz, which is not incorrect but unaesthetic...).

Your opinions/advice urgently required...

comment:13 in reply to: ↑ 12 ; follow-up: Changed 6 years ago by jdemeyer

Replying to charpent:

  • to document its renaming in spkg-src

Which renaming?

comment:14 in reply to: ↑ 13 Changed 6 years ago by charpent

Replying to jdemeyer:

Replying to charpent:

  • to document its renaming in spkg-src

Which renaming?

R-3.2.4-revised.tar.gz ==> R-3.2.4.tar.gz

comment:15 follow-up: Changed 6 years ago by jdemeyer

-1 to renaming it. I know that 3.2.4-revised is a really silly version number, but that's what upstream chose, so Sage should just use that.

By the way, I wonder why upstream didn't just name it version 3.2.5.

comment:16 in reply to: ↑ 15 Changed 6 years ago by charpent

Replying to jdemeyer:

-1 to renaming it. I know that 3.2.4-revised is a really silly version number, but that's what upstream chose, so Sage should just use that.

Okay (unless I hear other opinions by Sunday). But that's really silly...

By the way, I wonder why upstream didn't just name it version 3.2.5.

De gustibus coloribusque...

comment:17 Changed 6 years ago by tscrim

  • Milestone changed from sage-7.1 to sage-7.2

My 2 cents are with Jeroen, let us try and keep it as close to upstream as we can.

comment:18 follow-up: Changed 6 years ago by charpent

I'm wondering how to do this : upstream has adopted a curious name for this release ; furthermore, it has done it inconsistently :

  • the tarball is named R-3.2.4-revised, but
  • the source tree is named R-revised.

Sage's build system chokes on this ; short of renaming the tarball and/or rebuilding it with a sensible source tree name, I don't see how to get Sage's building system accept this.

We can either :

  • rename/rebuild this tarball consistently via spkg-src, or
  • skip this one-week wonder entierely (last time I looked, R 3.3.0 was scheduled in Apr 11, 2016).

Your thoughts ?

comment:19 in reply to: ↑ 18 ; follow-up: Changed 6 years ago by jdemeyer

Replying to charpent:

Sage's build system chokes on this

Can you be more specific?

comment:20 in reply to: ↑ 19 ; follow-up: Changed 6 years ago by charpent

Replying to jdemeyer:

Replying to charpent:

Sage's build system chokes on this

Can you be more specific?

At the start of the package build log, I see that the script tries to mv to the source tree, but fails to stat it (the script nevertheless continues and fails other steps such as patching and MAKEing it.

comment:21 in reply to: ↑ 20 ; follow-up: Changed 6 years ago by jdemeyer

Replying to charpent:

At the start of the package build log, I see that the script tries to mv to the source tree, but fails to stat it

Full log please.

comment:22 in reply to: ↑ 21 Changed 6 years ago by charpent

Replying to jdemeyer:

Full log please.

This will have to wait : I'm not on my development machine, and have ti reproduce it on a (much slower) work machine...

Changed 6 years ago by charpent

(Unsuccessfull) compilation log of the "revised" package

comment:23 Changed 6 years ago by charpent

  • Description modified (diff)

As requested, log added.

HTH,

Note: See TracTickets for help on using tickets.