Fix sagenb doctesting

[mpatel]

We have not been doctesting SageNB files as often as we test Sage files.

Please apply

• trac_7650-scripts_doctest_force_lib_v5.patch​ to the scripts repository. This adds two options to sage -t: -sagenb tests all SageNB files and -force_lib tests files as if they live in SAGE_ROOT/devel/sage, i.e., in the Sage library.

• trac_7650-sagenb_doctesting_v6.patch​ and trac_7650-reviewer_v2.patch​ to the sagenb repository. This fixes broken SageNB tests.

and replace SAGE_ROOT/makefile with

• makefile​. This makes make test and make ptest test all SageNB files, too.

Dependencies: Sage 4.3.1.alpha1, #7269.

[mpatel]

Part of the problem is that notebook directories now need to end in .sagenb.

I'm working on a patch for this and whatever other problems I can fix.

[mpatel]

sage -t -verbose "notebook.py"                              
Traceback (most recent call last):
  File "/home/.sage//tmp/.doctest_notebook.py", line 18, in <module>
    from notebook import *
  File "/home/.sage/tmp/notebook.py", line 38, in <module>
    import css          # style
ImportError: No module named css
         [2.5 s]
exit code: 1024
 
----------------------------------------------------------------------
The following tests failed:


        sage -t -verbose "notebook.py"
Total time for all tests: 2.5 seconds

Adding

import sys
sys.path.append('/home/sage/notebook/sagenb/sagenb')
sys.path.append('/home/sage/notebook/sagenb/sagenb/notebook')

to the top of notebook.py allows testing to proceed. I'm not sure about a real solution.

[mpatel]

Add --force_lib option to sage-doctest. Use os.path.join. scripts repo.

[mpatel]

Fix sagenb doctests. sagenb repo.

[mpatel]

Please note: The attachments (or their descriptions) are switched. I apologize for this.

Anyway, with the scripts repository patch, we can do, e.g.,

$ sage -t --force_lib sagenb/

With the sagenb repository patch applied to #7625 + #7483 + #7482 + #7514 + #7648, all tests but 3 pass. The failures are in sagenb/misc/sageinspect.py (cf. #7514). The Selenium tests still pass.

Remarks:

• I hope I did not create false-negatives.
• ​This sage-notebook thread might be relevant to some tests that evaluate cells.
• I "untabified" cell.py, worksheet.py, notebook.py, and twist.py.
• I added 'nodoctest' to simple/twist.py, since at least one test appears to hang indefinitely. I don't know why.
• Feel free to lower the priority.

[mpatel]

Should we add, e.g.,

sage -t --force_lib $SAGE_ROOT/local/lib/python/site-packages/sagenb

to $SAGE_ROOT/makefile? Or sage-test?

[was]

I like this patch a lot. However, a nitpick: get rid of the / before tmp here:

SAGE_TESTDIR = os.path.join(DOT_SAGE, "/tmp") 

Likewise, elsewhere in the patch there are a bunch of os.path.joins combined with explicit* slashes.

Regarding adding sage -t sagenb, etc., to sage-test or whatever: "yes!"

[mpatel]

Updated scripts repo patch. Replaces previous.

[mpatel]

Quit spawned worksheet processes. Replaces previous sagenb repo patch.

[mpatel]

Auto-detect site-packages. scripts repo. Replaces previous.

[mpatel]

Adds -sagenb option. scripts repo. Replaces previous.

[mpatel]

Updated SAGE_ROOT/makefile.

[mpatel]

With trac_7650-scripts_doctest_force_lib_v4.patch​ and an updated top-level makefile​, we can do

• make test
• make ptest
• sage -t -sagenb
• sage -tp -sagenb

etc. Maybe we should unify the test scripts and use ​optparse?

[timdumol]

The new options to sage -t (-force_lib and -sagenb) are not documented in the help message. Please do so.

[was]

I'm declaring a total feature freeze on sage-4.3.

[mpatel]

Also: Be more agressive with os.path.join-ery.

[mpatel]

Document new options. More os.path.join-ery. Replaces previous.

[mpatel]

Rebased vs. #7482's v4. Replaces previous sagenb patch.

[mpatel]

Rebased vs. 4.3.1.alpha0 + #7269. Replaces previous sagenb patch.

[mpatel]

I've attached a rebased sagenb patch. I added """nodoctests""" to the top of sagenb.misc.ipaddr. It's also easy to make its doctest pass, but I haven't done this. Please let me know, if I should.

[mpatel]

Fix tags / user_view asymmetry. Replaces previous sagenb patch.

[mpatel]

V5 makes it so a worksheet reconstructed from the basic representation of another has the same tags and user_view as the original.

[mpatel]

Fix interact doctests for 4.3.1.alpha2 (colors.py). Replaces previous sagenb patch.

[timdumol]

Replying to mpatel:

V5 makes it so a worksheet reconstructed from the basic representation of another has the same tags and user_view as the original.

Shouldn't this be put in a new ticket?

[mpatel]

Yes, but it's already here. :) In the course of fixing many failed doctests, I noticed it and a few other small problems:

• In run_notebook.notebook_twisted: Replaced nb.directory() (not defined) with nb._dir.
• In worksheet: Replaced self.__collaborators (AttributeError) with self.collaborators() in a few places.
• In worksheet.set_filename:

  -        self.__dir = os.path.join(self.notebook().worksheet_directory(), filename)
  +        self.__dir = os.path.join(self.notebook()._dir, filename)
  

• In worksheet.tags:

               d = dict(self.__user_view)
           except AttributeError:
               self.user_view(self.owner())
  -            d = self.__user_view
  +            d = copy.copy(self.__user_view)
           for user, val in d.iteritems():
  -            d[user] = [val]
  +            if not isinstance(val, list):
  +                d[user] = [val]
           return d
  

  This ensures the tests in Worksheet.reconstruct_from_basic pass and that calling tags does not alter the current user view, e.g., turning 1 into [1].

Also:

• Removed argument verbose (obsolete) from Cell.set_introspect_html.

[timdumol]

This bags a positive review from me, except for the few changes I've made in the reviewer patch. Can someone sign them off?

[timdumol]

A few documentation fixes (precision errors and style), and a couple of changes to try-except blocks to make it easier to transition to Python 3

[mpatel]

Fix two doctest failures. Replaces reviewer patch. sagenb repo.

[mpatel]

V2 of the reviewer patch:

• Adds a missing dot (.) in interact.py.
• Moves # output depends on timezone to the input statement in worksheet.py.
• [Pre-existing] most be defined --> must be defined.

[rlm]

I applied the scripts patch and replaced makefile in SAGE_ROOT. Please post a sagenb-0.5.1.spkg.

[rlm]

Merged ​http://boxen.math.washington.edu/home/timdumol/sagenb-0.6.spkg into spkg/standard.