Opened 10 years ago
Closed 8 years ago
#9921 closed enhancement (fixed)
nose testing suite as an optional spkg
Reported by: | jason | Owned by: | tbd |
---|---|---|---|
Priority: | major | Milestone: | sage-5.7 |
Component: | packages: optional | Keywords: | sd41 |
Cc: | drkirkby, jhpalmieri | Merged in: | sage-5.7.beta2 |
Authors: | John Palmieri | Reviewers: | Karl-Dieter Crisman, Martin Raum |
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description (last modified by )
Several projects we depend on use nose to do self-tests. In order to test these packages, we'd have to have nose installed.
This is fairly simple without an spkg:
wget http://pypi.python.org/packages/source/n/nose/nose-1.1.2.tar.gz # or the latest version at http://pypi.python.org/pypi/nose/ tar xzvf nose-1.1.2.tar.gz cd nose-1.1.2 sage -python setup.py install
However, it might make sense to have nose be an optional spkg.
Matplotlib relies on nose to do tests (http://matplotlib.sourceforge.net/devel/coding_guide.html#testing), as well as numpy/scipy (http://projects.scipy.org/numpy/wiki/TestingGuidelines).
New spkg: version 1.1.2.
Apply trac_9921-nose-scripts.patch to the scripts repo.
Attachments (3)
Change History (35)
comment:1 Changed 10 years ago by
comment:2 Changed 10 years ago by
- Description modified (diff)
comment:3 Changed 10 years ago by
IMHO it would be sensible to have nose as a standard package.
I suggested that once, and it got a luke warm reception. Having to install an optional item to test a standard package seems a bad idea to me.
Perhaps if a list of packages which make use of 'nose' could be produced, it might be possible to argue for it to be standard.
Dave
comment:4 follow-up: ↓ 5 Changed 10 years ago by
Yes, I'm thinking (according to guidelines) optional for now, standard after a probationary period.
comment:5 in reply to: ↑ 4 Changed 10 years ago by
Replying to jason:
Yes, I'm thinking (according to guidelines) optional for now, standard after a probationary period.
Yes, according to guidelines it should be optional for a while. But there have been exceptions to that. Since it does not actually link to anything in Sage, and would only be called when running spkg-check, I think one could argue the risk is minimal. In contrast it would allow a number of packages to be checked.
I think any such argument would have to be based on how many packages could benefit from it.
It must be an incredibly low risk package to add.
Dave
comment:6 follow-up: ↓ 7 Changed 10 years ago by
Whether or not it should be standard, I agree that we need to at least have it optional so that it becomes more 'Sage-ic' (totally not the right adjective, but I'm groping here) to test several fundamental python packages from within Sage. Also, it is needed to test some potential other packages, such as Brian (#9675), which is where I originally found out about this.
comment:7 in reply to: ↑ 6 ; follow-up: ↓ 8 Changed 10 years ago by
Replying to kcrisman:
Whether or not it should be standard, I agree that we need to at least have it optional so that it becomes more 'Sage-ic' (totally not the right adjective, but I'm groping here) to test several fundamental python packages from within Sage. Also, it is needed to test some potential other packages, such as Brian (#9675), which is where I originally found out about this.
I installed the package and tried it to test Brian, and it worked perfectly. However, I think some changes should be made in order to fit the structure of Sage packages: all the code should be in a directory named src/, it lacks the .hg folder, the .hgignore and SPKG.txt files, etc. (see the page Producing New Sage Packages to see how it should be).
comment:8 in reply to: ↑ 7 ; follow-ups: ↓ 9 ↓ 12 Changed 10 years ago by
- Type changed from defect to enhancement
Replying to uri:
[...] However, I think some changes should be made in order to fit the structure of Sage packages: all the code should be in a directory named src/, it lacks the .hg folder, the .hgignore and SPKG.txt files, etc. [...]
? Of course we first have to produce an spkg, be it optional or (later) standard.
Or did I miss something?
comment:9 in reply to: ↑ 8 ; follow-ups: ↓ 10 ↓ 13 Changed 10 years ago by
[...] However, I think some changes should be made in order to fit the structure of Sage packages: all the code should be in a directory named src/, it lacks the .hg folder, the .hgignore and SPKG.txt files, etc. [...]
? Of course we first have to produce an spkg, be it optional or (later) standard.
Or did I miss something?
No, I think that uri was perhaps misunderstanding jason's directions to use nose outside of Sage to be a suggestion for how it would become a Sage package. No harm done in reminding us of the official rules, though :)
comment:10 in reply to: ↑ 9 Changed 10 years ago by
Replying to kcrisman:
No, I think that uri was perhaps misunderstanding jason's directions to use nose outside of Sage to be a suggestion for how it would become a Sage package.
Yeah, an spkg just containing spkg-install
which is
wget http://code.google.com/p/python-nose/downloads/detail?name=nose-0.11.3.tar.gz tar xzvf nose-0.11.3.tar.gz cd nose-0.11.3 sage -python setup.py install
:D
comment:11 Changed 10 years ago by
Ok, we should concatenate the lines with &&
to make it more robust...
comment:12 in reply to: ↑ 8 Changed 10 years ago by
Replying to leif:
Replying to uri:
[...] However, I think some changes should be made in order to fit the structure of Sage packages: all the code should be in a directory named src/, it lacks the .hg folder, the .hgignore and SPKG.txt files, etc. [...]
? Of course we first have to produce an spkg, be it optional or (later) standard.
Or did I miss something?
Oh, right, I missunderstood... sorry :)
comment:13 in reply to: ↑ 9 Changed 10 years ago by
Replying to kcrisman:
[...] However, I think some changes should be made in order to fit the structure of Sage packages: all the code should be in a directory named src/, it lacks the .hg folder, the .hgignore and SPKG.txt files, etc. [...]
? Of course we first have to produce an spkg, be it optional or (later) standard.
Or did I miss something?
No, I think that uri was perhaps misunderstanding jason's directions to use nose outside of Sage to be a suggestion for how it would become a Sage package. No harm done in reminding us of the official rules, though :)
Yep, the point of the instructions was to show that even without an spkg, using nose "is fairly simple without an spkg". Of course, you can't download something in the spkg, so those instructions would not work for an spkg. However, it would be a very generic spkg-install file that would basically do the normal standard checks and then run "python setup.py install".
comment:14 Changed 10 years ago by
I tried using nose, and it was very interesting. Tested Brian on two different boxes, and then tested both numpy and scipy on OS X 10.6 - discovered no errors on the first, and a fair number on the second! So I think that there should be no doubt this could go straight to optional if an spkg was made.
I couldn't do
import matplotlib matplotlib.test()
because for some reason our matplotlib doesn't have this method.
Much less importantly, I should also point out that on OS X wget
is not a builtin, though I have an alias curl -O
that seems to accomplish the same purpose. I just downloaded and double-clicked, actually :)
comment:15 Changed 9 years ago by
- Description modified (diff)
Nose is now on Github: https://github.com/nose-devs/nose.
comment:16 Changed 9 years ago by
- Cc jhpalmieri added
comment:17 Changed 9 years ago by
- Status changed from new to needs_review
I've posted an spkg. It builds well enough for me, but I haven't actually used it to test anything except for itself (in spkg-check).
comment:18 Changed 9 years ago by
- Description modified (diff)
comment:19 Changed 9 years ago by
By the way, version 1.1.2 was the most recent version for which I could find a tar ball. Or we could use the version from github; would that be better?
comment:20 Changed 9 years ago by
- Description modified (diff)
comment:21 follow-up: ↓ 22 Changed 9 years ago by
I've now also prepared a version from git. This one fails its own self tests, though ;)
comment:22 in reply to: ↑ 21 Changed 9 years ago by
I've now also prepared a version from git. This one fails its own self tests, though ;)
Interesting!
You can try using it with the brian optional spkg, and I believe numpy and mpl also use this?
comment:23 Changed 9 years ago by
ipython also uses nose.
comment:24 Changed 9 years ago by
I tried running self-tests with numpy, using python -c 'import numpy; numpy.test()'
. The good news: tests ran. The bad news: some tests failed. More bad news: this exited with a return code of zero, so it's hard to tell (within spkg-check, for example), that tests failed. I have the same issue with IPython.
comment:25 Changed 9 years ago by
So is that a problem with nose or a problem with the numpy/ipython test suites? Sounds like nose is ready from this point of view :)
comment:26 Changed 9 years ago by
- Keywords sd41 added
comment:27 Changed 9 years ago by
- Description modified (diff)
- Reviewers set to Karl-Dieter Crisman
This is ridiculous. Nose works fine at testing things on sage.math. It certainly finds various errors and warnings - apparently scipy generates a number as well, I just tried it.
The issue with it not passing its own tests is not so good, and I can confirm this in both cases. On the plus side, it only breaks the Sage installation process in the 1.1.3, so I would say let's go with the 1.1.2 for now.
I do get something weird, hopefully unrelated to nose itself, but instead related to our defaults for matplotlib.
kcrisman@sage:~/sage-5.1.beta1-boxen-x86_64-Linux$ ./sage -c 'import matplotlib; matplotlib.test()' ====================================================================== ERROR: Failure: OSError (No such file /home/kcrisman/sage-5.1.beta1-boxen-x86_64-Linux/import matplotlib; matplotlib.test()) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/kcrisman/sage-5.1.beta1-boxen-x86_64-Linux/local/lib/python2.7/site-packages/nose-1.1.2-py2.7.egg/nose/failure.py", line 39, in runTest raise self.exc_class(self.exc_val) OSError: No such file /home/kcrisman/sage-5.1.beta1-boxen-x86_64-Linux/import matplotlib; matplotlib.test() ---------------------------------------------------------------------- Ran 1 test in 0.001s FAILED (errors=1) sage: import matplotlib sage: matplotlib.test() ====================================================================== ERROR: Failure: ValueError (Unable to load tests from file /home/kcrisman/sage-5.1.beta1-boxen-x86_64-Linux/sage) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/kcrisman/sage-5.1.beta1-boxen-x86_64-Linux/local/lib/python2.7/site-packages/nose-1.1.2-py2.7.egg/nose/loader.py", line 213, in loadTestsFromFile % filename) ValueError: Unable to load tests from file /home/kcrisman/sage-5.1.beta1-boxen-x86_64-Linux/sage ---------------------------------------------------------------------- Ran 1 test in 0.001s FAILED (errors=1) False
On a computer without nose:
sage: matplotlib.test() --------------------------------------------------------------------------- ImportError Traceback (most recent call last) /Users/karl-dietercrisman/Downloads/sage-5.1.beta6/<ipython console> in <module>() /Users/.../sage-5.1.beta6/local/lib/python2.7/site-packages/matplotlib/__init__.pyc in test(verbosity) 986 def test(verbosity=0): 987 """run the matplotlib test suite""" --> 988 import nose 989 import nose.plugins.builtin 990 from testing.noseclasses import KnownFailure ImportError: No module named nose
I think that in matplotlib's lib/init.py
success = nose.run( defaultTest=default_test_modules, config=config, )
we aren't using the right default modules, they aren't imported or something. So it goes back to just looking at .
for the default test module - I get the same thing.
sage: nose.run(defaultTest='.') E ====================================================================== ERROR: Failure: ValueError (Unable to load tests from file /home/kcrisman/sage-5.1.beta1-boxen-x86_64-Linux/sage) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/kcrisman/sage-5.1.beta1-boxen-x86_64-Linux/local/lib/python2.7/site-packages/nose-1.1.2-py2.7.egg/nose/loader.py", line 213, in loadTestsFromFile % filename) ValueError: Unable to load tests from file /home/kcrisman/sage-5.1.beta1-boxen-x86_64-Linux/sage ---------------------------------------------------------------------- Ran 1 test in 0.001s FAILED (errors=1) False
Can you see a reason why this shouldn't have positive review? I just think this must be a problem in how we're dealing with mpl. I'm surprised it does this; we only removed the baseline images, not the testing scripts!
comment:28 Changed 8 years ago by
- Reviewers changed from Karl-Dieter Crisman to Karl-Dieter Crisman, Martin Raum
I have also reviewed this ticket, since I think it is important to have it. The matplotlib error is really a bug (or at least laxness) in matplotlib. If we waited for nosetest to be included until all spkg's pass, we would wait until the cows come home.
If nobody objects, I will set this to positive review by next week.
comment:29 Changed 8 years ago by
- Status changed from needs_review to positive_review
comment:30 Changed 8 years ago by
- Description modified (diff)
comment:31 Changed 8 years ago by
spkg is put on the servers.
comment:32 Changed 8 years ago by
- Merged in set to sage-5.7.beta2
- Resolution set to fixed
- Status changed from positive_review to closed
The current website for nose is: http://code.google.com/p/python-nose/