Opened 10 years ago

Closed 8 years ago

Last modified 8 years ago

#7494 closed enhancement (fixed)

remove (or clean up) SAGE_ROOT/examples

Reported by: was Owned by: tbd
Priority: major Milestone: sage-4.7.2
Component: misc Keywords: sd32
Cc: wdj Merged in: sage-4.7.2.alpha4
Authors: William Stein, John Palmieri Reviewers: Karl-Dieter Crisman, Rob Beezer, Leif Leonhardy
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by jhpalmieri)

Did you know there is a directory SAGE_ROOT/examples? Do you care? Because if nobody seriously cares, I'm going to *delete it* from future versions of Sage, since it is still a mess, and the last nontrivial commit was 1.5 years ago (!):

changeset:   158:d18dad210d3b
user:        Mike Hansen <mhansen@gmail.com>
date:        Mon Apr 14 03:08:48 2008 -0700
summary:     Extract sagetex.py and sagetex.sty

I can put the same directory online somewhere, and move the fortran file that is used in one doctest out. I'm just really curious if anybody knows about this directory and cares.

Apply:

  1. trac_7494.patch to the Sage library repo
  2. trac_7494-ref.patch to the Sage library repo
  3. trac_7494-scripts.patch to the scripts repo
  4. trac_7494-root-repo.patch to the Sage root repository

Release Manager:

Delete $SAGE_ROOT/examples/ as part of this ticket.
(This should happen automatically, i.e., the directory should vanish, when sdisting.)

Attachments (4)

trac_7494.patch (5.1 KB) - added by was 8 years ago.
trac_7494-scripts.patch (3.4 KB) - added by jhpalmieri 8 years ago.
scripts repo
trac_7494-root-repo.patch (1.5 KB) - added by jhpalmieri 8 years ago.
root repo
trac_7494-ref.patch (934 bytes) - added by jhpalmieri 8 years ago.
Sage library: fix docbuilding problem

Download all attachments as: .zip

Change History (29)

comment:1 Changed 10 years ago by was

#298 would be invalidated by doing this ticket.

comment:2 follow-up: Changed 9 years ago by kcrisman

  • Cc wdj added
  • Report Upstream set to N/A

I was wondering about that directory too a few months ago. So we don't do anything with it currently? Here are comments.

I think it would be worth linking at least a few of the top-level ones to the wiki, and perhaps put in one of the documentation places, as some of the examples files there are definitely useful for templates.

I bet some of the programming (Pyrex/SageX/Cython) examples might be useful too.

I am cc:ing wdj to see if he thinks all his examples/routines from calculus are now in the main Sage library.

The linalg folder is definitely pointless, as permanents are now in the main library.

The worksheets folder seems pretty pointless in its current state.

The tests directory has some things that should be added as random doctests for those things, though.

The modsym directory stuff likely is already tested in that area, given how important it is to Sage!

The latex_embed is obviously superfluous at this point.

Fortran was has already discussed above.

Finance is *really* pointless.

The Groebner basis thing seems intriguing - perhaps should be incorporated in doctests for that elsewhere?

Is the Ajax thing now very very superfluous, given how much we use it? It's not (that) old.

The GSL folder is quite large compared to the rest and perhaps has some examples which should be combined into one big file which is doctested, if those sorts of tests aren't. It would seem valuable not to lose this many doctests -does it currently pass?

comment:3 in reply to: ↑ 2 Changed 9 years ago by kcrisman

I am cc:ing wdj to see if he thinks all his examples/routines from calculus are now in the main Sage library.

See #7936 for this - we can delete this one, at least.

comment:4 Changed 8 years ago by kini

The last nontrivial commit was 1.5 years before this ticket was created, but now it's been 1.5 years since this ticket was created! Can we get rid of this directory yet?

comment:5 Changed 8 years ago by kcrisman

I have asked about this a lot. In order to give this, it would be really nice to move a few of the examples to other places - wiki, wherever.

  • a few top-level things
  • some of the Cython ones?
  • tests has a few interesting random doctests
  • something fortran is needed somewhere, apparently? According to the comment in was' original report
  • The GSL examples might be worth moing somewhere
  • Groebner useful in doctests?

Anyway, we shouldn't just completely delete in case there are useful tests. But I totally agree on the overall strategy.

comment:6 Changed 8 years ago by was

  • Status changed from new to needs_review

Delete it!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

I wrote almost everything in there, and it should just go.

I give "delete the examples directory" a positive review. To merge this ticket, do

cd SAGE_ROOT
rm -rf examples

comment:7 Changed 8 years ago by was

  • Status changed from needs_review to positive_review

comment:8 Changed 8 years ago by was

For future reference, I've put that directory here:

http://wstein.org/old_sage_examples/

comment:9 Changed 8 years ago by was

  • Status changed from positive_review to needs_work

I just realized that there is one thing that has to be done. Observe:

wstein@ubuntu:~/d/sage$ sage -grep "examples"|grep SAGE_ROOT
finance/stock.py:            sage: finance.Stock('aapl').load_from_file(SAGE_ROOT + '/examples/finance/AAPL-minutely.csv')[:5]
finance/stock.py:            sage: finance.Stock('goog').load_from_file(SAGE_ROOT + '/examples/finance/AAPL-minutely.csv')[:5]
finance/stock.py:            sage: finance.Stock('aapl').load_from_file(SAGE_ROOT + "/examples/finance/AAPL-minutely.csv")
misc/hg.py:hg_examples = HG('%s/data/examples'%SAGE_ROOT,
misc/inline_fortran.py:            sage: s = open(os.environ['SAGE_ROOT'] + '/examples/fortran/FIB1.F').read()
wstein@ubuntu:~/d/sage$ 

So the stuff in examples that are used in doctests need to be moved to the SAGE_ROOT/data/ directory. Thus one needs to attach a patch that does this to this ticket, and that patch will need to be reviewed.

Changed 8 years ago by was

comment:10 Changed 8 years ago by was

The patch trac_7494.patch should change the core sage library so it doesn't depend on anything in the SAGE_ROOT/examples directory for testing. It also does not put anything in the SAGE_ROOT/data directory, since that is not needed.

comment:11 Changed 8 years ago by was

  • Status changed from needs_work to needs_review

comment:12 Changed 8 years ago by kcrisman

  • Authors set to William Stein

So the Groebner, gsl, etc. tests aren't really useful? Ok by me.

Nice - thanks for looking at this "officially". Since there is a Sage Days going on and today is my first day of classes, I think someone will beat me to the review, but thanks for looking at this a bit more carefully. In some sense, you are the only person who can check it, since you're one of the few who really knows why these things were in in the first place :)

comment:13 Changed 8 years ago by was

  • Keywords sd32 added

comment:14 Changed 8 years ago by rbeezer

  • Description modified (diff)
  • Reviewers set to Karl-Dieter Crisman, Rob Beezer
  • Status changed from needs_review to positive_review

Trash it. There's too much old misleading stuff about as is.

I renamed the examples directory, applied the patch, ran tests and got no failures.

I guess we need to alert the release manager somehow to actually delete the directory in the next distribution? I'll put a note above, edit as necessary.

comment:15 Changed 8 years ago by jhpalmieri

  • Authors changed from William Stein to William Stein, John Palmieri
  • Description modified (diff)
  • Status changed from positive_review to needs_work

This also needs a patch to the scripts repo, since sage-make_devel_packages tries to create a new examples spkg and fails with a nonzero exit status if it can't do that. I'm attaching a patch.

comment:16 Changed 8 years ago by jhpalmieri

  • Status changed from needs_work to needs_review

Changed 8 years ago by jhpalmieri

scripts repo

comment:17 Changed 8 years ago by was

  • Status changed from needs_review to positive_review

I'm good with jhpalmieri's cleanup of my patch.

comment:18 Changed 8 years ago by leif

  • Description modified (diff)
  • Status changed from positive_review to needs_work
  • Work issues set to Add a patch for the root repo.

The root repository certainly also has to get patched:

$ grep -win examples spkg/install spkg/standard/deps 
spkg/install:184:EXAMPLES=`$newest examples`
spkg/install:185:export EXAMPLES
spkg/standard/deps:48:     $(INST)/$(EXAMPLES) \
spkg/standard/deps:398:$(INST)/$(EXAMPLES): $(BASE) $(INST)/$(PATCH)
spkg/standard/deps:399: $(INSTALL) "$(SAGE_SPKG) $(EXAMPLES) 2>&1" "tee -a $(SAGE_LOGS)/$(EXAMPLES).log"

comment:19 Changed 8 years ago by jhpalmieri

  • Description modified (diff)
  • Status changed from needs_work to needs_review
  • Work issues Add a patch for the root repo. deleted

I think this searches through all of the relevant files:

$ grep -win examples `hg manifest`

I'm attaching a patch for this. In addition to the two files you found, it also patches libdist_filelist, which looks like it is completely out of date, but it doesn't hurt to patch it anyway.

Changed 8 years ago by jhpalmieri

root repo

comment:20 Changed 8 years ago by leif

  • Reviewers changed from Karl-Dieter Crisman, Rob Beezer to Karl-Dieter Crisman, Rob Beezer, Leif Leonhardy
  • Status changed from needs_review to positive_review

Ok, I've sdisted with a version with the patches applied, built the new Sage distribution from scratch, and all (long) tests pass.

As expected, I got rid of the examples/ directory and the spkg in the new distro. (Note that also the examples spkg in spkg/standard/ of the original distribution got deleted; not sure whether this is intentional. In contrast, the examples/ directory there remained.)

Upgrading from a path without an examples spkg shouldTM IMHO also work; haven't tried that [yet].


Don't know yet whether Jeroen's merger is "prepared" for such an operation, and since Sage 4.7.2.alpha3 is almost out, I don't think I'll attempt to merge this ticket into this release. There'll certainly be an alpha4, or we could try with the next release candidate.

Unfortunately we don't win much regarding the size; the current examples spkg is just 2.0 MB, 3.9 MB uncompressed on my disk (i.e., the size of the examples/ directory), and just 2.6 MB in "real" bytes.

comment:21 Changed 8 years ago by jdemeyer

  • Status changed from positive_review to needs_work

Problems while building the documentation:

dochtml.log:/mnt/usb1/scratch/jdemeyer/merger/sage-4.7.2.alpha4/local/lib/python2.6/site-packages/sage/finance/stock.py:docstring of sage.finance.stock.OHLC:3: (WARNING/2) Block quote ends without a blank line; unexpected unindent.
dochtml.log:/mnt/usb1/scratch/jdemeyer/merger/sage-4.7.2.alpha4/local/lib/python2.6/site-packages/sage/finance/stock.py:docstring of sage.finance.stock.OHLC:8: (ERROR/3) Unexpected indentation.

comment:22 follow-up: Changed 8 years ago by jhpalmieri

  • Description modified (diff)
  • Status changed from needs_work to needs_review

I believe this referee patch should fix the documentation problem.

Changed 8 years ago by jhpalmieri

Sage library: fix docbuilding problem

comment:23 in reply to: ↑ 22 Changed 8 years ago by leif

  • Status changed from needs_review to positive_review

Replying to jhpalmieri:

I believe this referee patch should fix the documentation problem.

Belief is not a technical category.

Nice patch by the way.

comment:24 Changed 8 years ago by jdemeyer

  • Merged in set to sage-4.7.2.alpha4
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:25 Changed 8 years ago by jhpalmieri

See #11907 for a followup.

Note: See TracTickets for help on using tickets.