Opened 8 years ago

Last modified 4 years ago

#11197 needs_info defect

Make spkg docs be built and installed after the full build — at Version 7

Reported by: jason Owned by: tbd
Priority: major Milestone: sage-6.4
Component: build Keywords: sd32
Cc: ddrake Merged in:
Authors: John Palmieri Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by jason)

Following on some concerns brought up about dependencies in building spkg docs, as implemented in #10823, here is a ticket for reimplementing #10823 so that docs are built after the full Sage build. In this way, spkg docs are built when the normal Sage docs are built, and we don't have any worries about having python or sphinx or other dependencies installed.

From Dan's message to sage-devel:

How about, in addition to spkg-install and spkg-check, we have spkg-install-docs? We would move the logic that checks the SAGE_SPKG_INSTALL_DOCS variable to the build/install script, which would run the script if the env var is set. This would also allow the build process to determine if a spkg has documentation that can be built, using something roughly like tar jtf foo.spkg | grep spkg-install-docs

As a data point, doing that for all spkgs in 4.7.alpha4 (except the Sage library, which we handle separately) takes about 70 seconds on a mildly-loaded sagenb.kaist.ac.kr, which is an 8-core Xeon machine.

See http://groups.google.com/group/sage-devel/browse_thread/thread/d46ddccff06d8670 and various tickets associated with #10823

Here are a few updated spkgs:

Change History (8)

comment:1 Changed 8 years ago by jason

  • Description modified (diff)

comment:2 Changed 8 years ago by jason

  • Description modified (diff)

comment:3 Changed 8 years ago by jason

  • Description modified (diff)

comment:4 Changed 8 years ago by jason

  • Description modified (diff)

comment:5 Changed 8 years ago by jason

  • Cc ddrake added

comment:6 follow-up: Changed 8 years ago by jhpalmieri

  • Authors set to John Palmieri
  • Status changed from new to needs_review

Here's a patch for the Sage root repo (the file spkg/install). This should perhaps be considered a first draft, because there are a few issues/questions.

  • Right now, it records a log for the docbuilding in spkg/logs/numpy-1.5.1.p0-docs.log, for example, instead of just appending to the already existing spkg/logs/numpy-1.5.1.p0.log. It's easy enough to switch it around. Opinions?
  • Right now, it stores a placeholder file in spkg/installed if the docs seem to have been built correctly. (Since after building each spkg, we delete the directory where it was built (spkg/build/numpy-...), we can't just rely on 'make' recognizing that it's already been installed, I think.) Is this the right way to do it?

A few things which I think belong on other tickets or which I don't know how to solve:

  • With the approach here, running "sage -i ..." or "sage -f ..." won't install the docs. Should we add something to the sage-spkg script which tries to build the docs if SAGE_SPKG_INSTALL_DOCS is "yes", but fails gracefully if it can't? Or only build the docs in sage-spkg if we can tell that the Sage library has already been installed, or some other criterion like this? This may end up with some code duplication, or separating out the doc-building stuff into a separate script (and then gets us entangled in the base repo vs. scripts repo debate...).
  • I noticed that the most recent cython spkg (the one in 4.7.alpha4) has files for building the docs, at least some of which weren't added to the Mercurial repository. I haven't checked the other spkgs for this sort of issue, but someone should...

Changed 8 years ago by jhpalmieri

root repo

comment:7 in reply to: ↑ 6 Changed 8 years ago by jason

  • Description modified (diff)

Replying to jhpalmieri:

Here's a patch for the Sage root repo (the file spkg/install). This should perhaps be considered a first draft, because there are a few issues/questions. - Right now, it records a log for the docbuilding in spkg/logs/numpy-1.5.1.p0-docs.log, for example, instead of just appending to the already existing spkg/logs/numpy-1.5.1.p0.log. It's easy enough to switch it around. Opinions?

  • Right now, it stores a placeholder file in spkg/installed if the docs seem to have been built correctly. (Since after building each spkg, we delete the directory where it was built (spkg/build/numpy-...), we can't just rely on 'make' recognizing that it's already been installed, I think.) Is this the right way to do it? A few things which I think belong on other tickets or which I don't know how to solve:

To me, it seems okay to think about the docs install as a separate thing from the spkg install, so it makes sense to me to have a separate log file and separate placeholder file. I don't know much about this part of the build process, though, so my +1 doesn't count for much.

  • With the approach here, running "sage -i ..." or "sage -f ..." won't install the docs. Should we add something to the sage-spkg script which tries to build the docs if SAGE_SPKG_INSTALL_DOCS is "yes", but fails gracefully if it can't? Or only build the docs in sage-spkg if we can tell that the Sage library has already been installed, or some other criterion like this? This may end up with some code duplication, or separating out the doc-building stuff into a separate script (and then gets us entangled in the base repo vs. scripts repo debate...).

Hmm...thinking about this one still...

  • I noticed that the most recent cython spkg (the one in 4.7.alpha4) has files for building the docs, at least some of which weren't added to the Mercurial repository. I haven't checked the other spkgs for this sort of issue, but someone should...

Are you talking about the docs directory containing the documentation? That directory is part of the upstream source, so shouldn't be in the spkg repository. However, as of a week or two ago, that docs directory has been merged into the main Cython source tree, so future versions of the spkg will not have that docs directory.

Note: See TracTickets for help on using tickets.