Opened 14 years ago
Closed 14 years ago
#3900 closed defect (fixed)
[with patch; positive review] make testing an official pickle jar a part of "make check"
Reported by: | William Stein | Owned by: | Michael Abshoff |
---|---|---|---|
Priority: | blocker | Milestone: | sage-3.2 |
Component: | build | Keywords: | |
Cc: | Merged in: | ||
Authors: | Reviewers: | ||
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Attachments (2)
Change History (14)
comment:1 Changed 14 years ago by
Component: | algebra → build |
---|---|
Owner: | changed from tbd to Michael Abshoff |
Changed 14 years ago by
Attachment: | sage-3900.patch added |
---|
Changed 14 years ago by
Attachment: | pickle_jar.tar.bz2 added |
---|
comment:2 Changed 14 years ago by
Summary: | make testing an official pickle jar a part of "make check" → [with patch; needs review] make testing an official pickle jar a part of "make check" |
---|
comment:3 Changed 14 years ago by
Priority: | critical → blocker |
---|
comment:4 Changed 14 years ago by
Milestone: | sage-3.2.1 → sage-3.2 |
---|
comment:5 Changed 14 years ago by
Milestone: | sage-3.2.1 → sage-3.2 |
---|
comment:6 Changed 14 years ago by
I applied the patch, but got
sage -t devel/sage/sage/structure/sage_object.pyx ********************************************************************** File "/home/jaap/downloads/sage-3.2.rc1/devel/sage/sage/structure/sage_object.pyx", line 750: sage: sage.structure.sage_object.unpickle_all(std) Exception raised: Traceback (most recent call last): File "/home/jaap/downloads/sage-3.2.rc1/local/bin/ncadoctest.py", line 1231, in run_one_test self.run_one_example(test, example, filename, compileflags) File "/home/jaap/downloads/sage-3.2.rc1/local/bin/sagedoctest.py", line 38, in run_one_example OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags) File "/home/jaap/downloads/sage-3.2.rc1/local/bin/ncadoctest.py", line 1172, in run_one_example compileflags, 1) in test.globs File "<doctest __main__.example_18[6]>", line 1, in <module> sage.structure.sage_object.unpickle_all(std)###line 750: sage: sage.structure.sage_object.unpickle_all(std) File "sage_object.pyx", line 764, in sage.structure.sage_object.unpickle_all (sage/structure/sage_object.c:7475) IndexError: list index out of range ********************************************************************** 1 items had failures: 1 of 7 in __main__.example_18 ***Test Failed*** 1 failures. For whitespace errors, see the file /home/jaap/downloads/sage-3.2.rc1/tmp/.doctest_sage_object.pybunzip2: /home/jaap/downloads/sage-3.2.rc1/data/pickle_jar.tar.bz2 is not a bzip2 file. tar: This does not look like a tar archive tar: Error exit delayed from previous errors [3.7 s]
Jaap
comment:7 Changed 14 years ago by
Summary: | [with patch; needs review] make testing an official pickle jar a part of "make check" → [with patch; positive review] make testing an official pickle jar a part of "make check" |
---|
Sorry for the noise.
Now I put in the correct file and all's well
[jaap@paix sage-3.2.rc1]$ ./sage -t devel/sage/sage/structure/sage_object.pyx sage -t devel/sage/sage/structure/sage_object.pyx [6.6 s] ---------------------------------------------------------------------- All tests passed! So positive review.
comment:8 Changed 14 years ago by
We need to deal with -sdist and -bdist copying pickle_jar.tar.bz2. One way to do this would be to check the file into the repo, but that seems like a bad idea.
Thoughts?
Cheers,
Michael
comment:9 follow-up: 10 Changed 14 years ago by
Hi folks,
there does not seem to be any problem with binary distributions, once that file "pickle_jar.tar.bz2" exists. See line 52 of the "sage-bdist" script, where among others the complete subtree under /data is copied over.
As for source distributions, essentially the distribution of the file "pickle_jar.tar.bz2" is not necessary --- it can be (re-)produced from any source distribution by a (very) short sequence of commands that does not change. In that sense it's a "build output". For convenience, one might think about having a make target "make pickle_jar".
But since the doctest introduced by this patch requires the prior existence of this file "pickle_jar.tar.bz2", and since maybe we don't want to produce updated pickle jars for each and every Sage version in say the alpha release cycles, a simplistic spkg could be the solution:
"sage_pickle_jar-X.Y.Z.spkg" containing just the three files "spkg-install", "SPKG.txt", and "pickle_jar.tar.bz2"; and spkg-install just issue a single "cp" command. Then, e.g. Sage version 3.3.2.alpha4 could still contain the spkg "sage_pickle_jar-3.3.1.spkg".
To make the creation of updated versions of this simplistic spkg easier, using the existing scripts machinery, it would be advisable not to use the directory "data/" to store the file "pickle_jar.tar.bz2", but instead the directory "data/sage_pickle_jar/", and storing there all these three files. (So a simple "spkg -pkg ..." does the job of creating the updated spkg.)
Of course the naming could also be "std_pickle_jar" instead, or similar.
I'd volunteer to create and test the needed script "spkg-install" resp. that simplistic spkg, opening another ticket for it, if someone gives me a "Yep. Good idea. Go for it!". Then the usual review process would take its course, and this ticket would depend on that other ticket.
Cheers,
gsw
comment:10 Changed 14 years ago by
Replying to GeorgSWeber:
Hi folks,
there does not seem to be any problem with binary distributions, once that file "pickle_jar.tar.bz2" exists. See line 52 of the "sage-bdist" script, where among others the complete subtree under /data is copied over.
Maybe, but in which spkg does the data directory end up? I can see the content of extcode, but not the data directory itself.
As for source distributions, essentially the distribution of the file "pickle_jar.tar.bz2" is not necessary --- it can be (re-)produced from any source distribution by a (very) short sequence of commands that does not change. In that sense it's a "build output". For convenience, one might think about having a make target "make pickle_jar".
The point is that the *old* pickle jar works.
But since the doctest introduced by this patch requires the prior existence of this file "pickle_jar.tar.bz2", and since maybe we don't want to produce updated pickle jars for each and every Sage version in say the alpha release cycles, a simplistic spkg could be the solution:
"sage_pickle_jar-X.Y.Z.spkg" containing just the three files "spkg-install", "SPKG.txt", and "pickle_jar.tar.bz2"; and spkg-install just issue a single "cp" command. Then, e.g. Sage version 3.3.2.alpha4 could still contain the spkg "sage_pickle_jar-3.3.1.spkg".
To make the creation of updated versions of this simplistic spkg easier, using the existing scripts machinery, it would be advisable not to use the directory "data/" to store the file "pickle_jar.tar.bz2", but instead the directory "data/sage_pickle_jar/", and storing there all these three files. (So a simple "spkg -pkg ..." does the job of creating the updated spkg.)
We don't want another spkg - that idea was rejected. I am not 100% clear why, but that is what lead to this problem in the first place.
Of course the naming could also be "std_pickle_jar" instead, or similar.
I'd volunteer to create and test the needed script "spkg-install" resp. that simplistic spkg, opening another ticket for it, if someone gives me a "Yep. Good idea. Go for it!". Then the usual review process would take its course, and this ticket would depend on that other ticket.
Cheers,
gsw
The easiest solution IMHO would be to check the archive into $SAGE_ROOT/data/extcode/pickle_jar and adjust the path in the patch here.
Cheers,
Michael
comment:11 Changed 14 years ago by
On closer inspection: we don't even have to check in the pickle jar into the ext repo, so I will just put it in there.
Cheers,
Michael
You must place this tarball in SAGE_ROOT/data and *leave* it compressed.