#13884 closed defect (fixed)
Fix optional internet tests.
Reported by:  Robert Bradshaw  Owned by:  Minh Van Nguyen 

Priority:  major  Milestone:  sage6.2 
Component:  doctest coverage  Keywords:  
Cc:  Merged in:  
Authors:  KarlDieter Crisman  Reviewers:  Ralf Stephan, Volker Braun 
Report Upstream:  N/A  Work issues:  
Branch:  90a4128 (Commits, GitHub, GitLab)  Commit:  
Dependencies:  Stopgaps: 
Description (last modified by )
Untested = broken.
sage t only_optional=internet devel/sage/sage/symbolic/integration/integral.py # 2 doctests failed sage t only_optional=internet devel/sage/sage/combinat/words/paths.py # 1 doctests failed sage t only_optional=internet devel/sage/sage/misc/preparser.py # 1 doctests failed sage t only_optional=internet devel/sage/sage/interfaces/r.py # 1 doctests failed sage t only_optional=internet devel/sage/sage/finance/stock.py # 11 doctests failed sage t only_optional=internet devel/sage/sage/databases/sloane.py # 8 doctests failed
The files sage/databases/sloane.py
and sage/combinat/words/paths.py
are really covered by #10358.
The file sage/interfaces/r.py
is a situation needing clarification at #7771.
This ticket is to fix the minor issues in the integral, preparer, and stock files.
Apply:
Attachments (2)
Change History (26)
comment:1 Changed 10 years ago by
comment:2 Changed 10 years ago by
sage t only_optional=internet devel/sage/sage/interfaces/r.py
Just for clarification, this one is broken, but not because of the internet, and has been for a while. This was originally noticed (and fixed) by Minh while reviewing #7771 in this patch. It wasn't clear how to proceed, as you can see by reading the comments there.
comment:3 Changed 10 years ago by
Haha! And the first one is not really a problem, just a logic issue.
sage t only_optional=internet devel/sage/sage/symbolic/integration/integral.py sage: f.integrate(algorithm="mathematica_free") # optional  requires internet Expected: sqrt(pi)*sqrt(1/2)*fresnels(sqrt(2)*x/sqrt(pi)) + y^z*x Got: doctest:674: DeprecationWarning: Variable of integration should be specified explicitly. See http://trac.sagemath.org/12438 for details. sqrt(pi)*sqrt(1/2)*fresnels(sqrt(2)*x/sqrt(pi)) + y^z*x ********************************************************************** Expected: doctest:...: DeprecationWarning: Variable of integration should be specified explicitly. See http://trac.sagemath.org/12438 for details. 1/2*x^2 Got: 1/2*x^2
What happens is that when the test is done, the deprecation warning, which only occurs once, occurs with the internet test. This is a twocharacter fix. I'll try to see if I can fix any others along with it.
comment:4 Changed 10 years ago by
Next: Considering that William at some point changed the file in this test
sage: sage.misc.preparser.load('http://wstein.org/loadtest.py', globals()) # optional  internet Expected: hi from the net Got: hi from the net 5
to
print "hi from the net" print 2+3
I think we'll be okay just changing this one too.
comment:5 Changed 10 years ago by
Just for reference, in general this file (sloane.py) is almost completely undocumented and #10358 is supposed to update with respect to this. So this is fully known.
comment:6 Changed 10 years ago by
The finance errors seem mostly due to a change in how Google uses startdate
, though at least one of them shouldn't have even worked the first time!
sage: finance.Stock('vmw').google()[:5] # optional  internet Expected: [ 28Nov07 80.57 88.49 80.57 87.69 7496000, 29Nov07 90.91 93.20 89.50 90.85 5497600, 30Nov07 95.39 95.60 89.85 91.37 4750200, 3Dec07 89.87 96.00 88.70 94.97 4401100, 4Dec07 92.26 97.10 92.05 95.08 2896600 ] Got: [ 5Jan12 81.76 82.84 81.52 82.31 2888520, 6Jan12 82.96 84.33 82.47 83.31 1922781, 9Jan12 83.45 85.17 82.94 84.95 1841373, 10Jan12 86.01 86.80 85.10 85.30 1762911, 11Jan12 85.42 88.20 85.30 87.92 2192664 ]
At least this tells us when William added the finance stuff!
comment:7 Changed 10 years ago by
And finally, the combinat error is due to the broken sloane_find
issue. So this ticket should be quite fixable  hope to have a patch up soon.
comment:8 Changed 10 years ago by
Description:  modified (diff) 

comment:9 Changed 10 years ago by
Google now apparently does not return data that is more than a certain number of years old  or at least the interface must have changed?
sage: finance.Stock('F').google('Jan+3,+1978', 'Jul+7,+2008')[:1] [ 20Aug92 0.00 2.26 2.21 2.24 19202479 ] sage: finance.Stock('F').google('Jan+3,+1982', 'Jul+7,+2008')[:1] [ 20Aug92 0.00 2.26 2.21 2.24 19202479 ]
In fact, according to this blog post,
These APIs are now deprecated but have no scheduled shutdown date: ... Finance API ...
And although the data is still there if one uses e.g. a URL like this downloading the CSV only gives you so much.
What should we do, oh many Sage devels who are Google employees? ;) I can fix some stuff but don't want to fix more of this if it's going to be gone eventually anyway.
comment:10 Changed 10 years ago by
Okay, the following patch fixes everything not from #10358, #7771, and the change in the CSV files that the Google Finance API produces. (Note that Apple must have had a 41 stock split at some point as well.) In some cases I could circumvent the latter when the point wasn't testing historical data, but there are still two failures.
sage t only_optional=internet "devel/sage/sage/finance/stock.py" ********************************************************************** sage: finance.Stock('F').google('Jan+3,+1978', 'Jul+7,+2008')[:5] # optional  internet Expected: [ 3Jan78 0.00 1.93 1.89 1.89 1618200, 4Jan78 0.00 1.89 1.87 1.88 2482700, 5Jan78 0.00 1.89 1.84 1.84 2994900, 6Jan78 0.00 1.84 1.82 1.83 3042500, 9Jan78 0.00 1.81 1.79 1.81 3916400 ] Got: [ 20Aug92 0.00 2.26 2.21 2.24 19202479, 21Aug92 0.00 2.27 2.19 2.20 18689616, 24Aug92 0.00 2.17 2.10 2.10 38652825, 25Aug92 0.00 2.19 2.11 2.18 31028049, 26Aug92 0.00 2.21 2.18 2.20 22539411 ] ********************************************************************** sage: sage.finance.stock.Stock("AAPL", 22144).google(startdate='Jan+1,+1990')[:5] #optional  internet Expected: [ 2Jan90 0.00 9.38 8.75 9.31 6542800, 3Jan90 0.00 9.50 9.38 9.38 7428400, 4Jan90 0.00 9.69 9.31 9.41 7911200, 5Jan90 0.00 9.56 9.25 9.44 4404000, 8Jan90 0.00 9.50 9.25 9.50 3627600 ] Got: [ 28Feb97 0.00 1.05 1.02 1.02 17411200, 3Mar97 0.00 1.03 1.00 1.01 18636800, 4Mar97 0.00 1.03 1.00 1.03 14745600, 5Mar97 0.00 1.06 1.03 1.06 13737600, 6Mar97 0.00 1.06 1.03 1.04 16612800 ] **********************************************************************
I don't know if these are fixable.
Changed 10 years ago by
Attachment:  trac_13884.patch added 

comment:11 Changed 10 years ago by
Authors:  → KarlDieter Crisman 

Description:  modified (diff) 
Status:  new → needs_review 
Patchbot, apply trac_13884.patch
I'd appreciate feedback; however, the easiest to me seems to be to review this and then open a new ticket to fix the Google finance interface csv download issue  if that is even possible.
comment:12 Changed 9 years ago by
Status:  needs_review → needs_work 

Okay, in the meantime the Google stuff has changed yet again (???), and we get the following bizarre error:
File "sage/symbolic/integration/integral.py", line 466, in sage.symbolic.integration.integral.integrate Failed example: f.integrate(algorithm="mathematica_free") # optional  internet NameError: name 'f' is not defined **********************************************************************
This one baffles me; even doing f.integrate(x)
with no algorithm or anything still fails. Doing this by hand usually works  I think there is some potential for timeouts. But it consistently fails with this in doctesting. So I'm giving up on this patch for now, unless someone has some other bright ideas.
comment:13 Changed 9 years ago by
Milestone:  sage5.11 → sage5.12 

comment:14 Changed 9 years ago by
Description:  modified (diff) 

Changed 9 years ago by
Attachment:  trac_13884_addon1.patch added 

comment:15 Changed 9 years ago by
Description:  modified (diff) 

I think this solves the problem in the integral file.
comment:16 Changed 9 years ago by
The addon patch is good. I'm not sure why that should make a difference, but it did.
I'm tempted to have us just add the preparser test to the integral one and then open a new ticket for stock.py, which clearly isn't functioning properly in any case (not just a problem of changed values any more). Frederic, do you want to create a patch like that?
comment:19 Changed 9 years ago by
Branch:  → u/chapoton/13884 

Commit:  → 90a4128e8e2eebf6f79511d684b56c1c710464dd 
Status:  needs_work → needs_review 
New commits:
90a4128  Trac #13884  fix most broken internet optional doctests

comment:20 Changed 9 years ago by
Milestone:  sage6.1 → sage6.2 

comment:21 Changed 8 years ago by
What test command do you use? With optional=internet
I get:
sage t src/sage/finance/stock.py # 11 doctests failed sage t src/sage/symbolic/integration/external.py # 1 doctest failed
comment:22 Changed 8 years ago by
Reviewers:  → Ralf Stephan, Volker Braun 

Status:  needs_review → positive_review 
Sounds good to me; somebody interested in finance can take up the stock.py failures.
comment:23 Changed 8 years ago by
Branch:  u/chapoton/13884 → 90a4128e8e2eebf6f79511d684b56c1c710464dd 

Resolution:  → fixed 
Status:  positive_review → closed 
comment:24 Changed 7 years ago by
Commit:  90a4128e8e2eebf6f79511d684b56c1c710464dd 

Description:  modified (diff) 
So this doesn't happen again, reenable these via #13540.