Update the Sage Development Process documentation for new users

[jessicapalencia]

At Sage Days 82, we referred many new users to the Sage Development Process documentation and found many errors or points that need clarification.

[jen]

New commits:

​5a9accd

Fixes to Sage Developer's guide documentation

[mkoeppe]

+    If your system supports multiprocessing and you want to use multiple 
+    processors to build Sage, replace the last line above by::
+
+    [user@localhost sage]$ make -jNUM

The current recommendation seems to be to use

    export MAKE="make -jNUM"
    make

instead. See discussion in #21610.

[dimpase]

Replying to mkoeppe:

The current recommendation seems to be to use

    export MAKE="make -jNUM"
    make

instead. See discussion in #21610.

indeed, as indicated in #21610, one can get away with toplevel make -jNUM, but only after some work, and assuming make's version is at least 4.2.1.

[paulmasson]

Replying to dimpase:

indeed, as indicated in #21610, one can get away with toplevel make -jNUM, but only after some work, and assuming make's version is at least 4.2.1.

I run make -j10 all the time on OS X Sierra (make version 3.81) without doing anything else and have no apparent problems. Does that just mean that the process falls back to a single job for parts of the build, like the documentation?

[paulmasson]

On a minor note, there are a couple really long lines added in this commit. Please break them to keep all lines in a file to about the same length. That makes them easier to read.

[dimpase]

Replying to paulmasson:

Replying to dimpase:

indeed, as indicated in #21610, one can get away with toplevel make -jNUM, but only after some work, and assuming make's version is at least 4.2.1.

I run make -j10 all the time on OS X Sierra (make version 3.81) without doing anything else and have no apparent problems. Does that just mean that the process falls back to a single job for parts of the build, like the documentation?

most probably the latter. Could you try the other way on the documentation, whether there is a speed difference for you?

[paulmasson]

Replying to dimpase:

Replying to paulmasson:

Replying to dimpase:

indeed, as indicated in #21610, one can get away with toplevel make -jNUM, but only after some work, and assuming make's version is at least 4.2.1.

I run make -j10 all the time on OS X Sierra (make version 3.81) without doing anything else and have no apparent problems. Does that just mean that the process falls back to a single job for parts of the build, like the documentation?

most probably the latter. Could you try the other way on the documentation, whether there is a speed difference for you?

With plain make -j10 documentation takes 1284 seconds to build, with export it takes 1155 seconds. Not enough of a difference for me to stop using the simpler command.

[dimpase]

Replying to paulmasson:

Replying to dimpase:

Replying to paulmasson:

Replying to dimpase:

indeed, as indicated in #21610, one can get away with toplevel make -jNUM, but only after some work, and assuming make's version is at least 4.2.1.

I run make -j10 all the time on OS X Sierra (make version 3.81) without doing anything else and have no apparent problems. Does that just mean that the process falls back to a single job for parts of the build, like the documentation?

most probably the latter. Could you try the other way on the documentation, whether there is a speed difference for you?

With plain make -j10 documentation takes 1284 seconds to build, with export it takes 1155 seconds. Not enough of a difference for me to stop using the simpler command.

My bad, documentation build apparently somehow gets hold of '-j10'---you could see this by looking at 'top' output and observing about 10 python processes running.

What would matter is building of Sage packages - each of them would be built with '-j1'.

[jhpalmieri]

On one machine, when I built with MAKE='make -j6', make build took 99 minutes. With MAKE unset, make -j6 build took 107 minutes.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​20b18fd

fixed long lines and changed make -jNUM

[jessicapalencia]

We addressed all the comments above.

[dimpase]

Replying to jhpalmieri:

On one machine, when I built with MAKE='make -j6', make build took 99 minutes. With MAKE unset, make -j6 build took 107 minutes.

I did make distclean followed by make -j8 on an 8-core Linux machine I got, with MAKE set to make -j8

real    41m35.245s
user    259m17.166s
sys     13m55.563s

and with MAKE unset (don't forget to actually run unset MAKE before trying)

real    55m40.123s
user    219m20.552s
sys     13m30.681s

which is almost 30% slowdown (w.r.t. the wall clock).

[dimpase]

isn't

$ export MAKE='make -jNUM'
$ $MAKE

more effective than

$ MAKE='make -jNUM' make

(unless for some reason you really do not want to keep building with -jNUM next time?)

[jhpalmieri]

I think the right place to give full details about how to set the MAKE environment variable is in the ​"Environment variables" section of the Sage installation guide. So the brief comments in the developer's guide don't need to be perfect, for example with respect to

$ MAKE='make -jNUM' make

vs.

$ export MAKE='make -jNUM'
$ make

[dimpase]

OK. Other reviewers, please feel free to write themselves in.

[tscrim]

Two very minor things:

• We are trying to standardize .. NOTE:: instead of .. note::.
• Some of your line breaks make some lines unnaturally short to me.

The more important of the two is the first one.

[dimpase]

when the day comes, someone will make a script to fix the other 500+ occurences...

$ grep -R "note::" src/sage/ | wc -l
513

[tscrim]

travis@apricot:~/sage-build$ grep -R "NOTE::" src/sage/ | wc -l
1636

We don't have to make it worse.

[dimpase]

Well, instead of arguing here, make a ticket to fix this Sage-wise :-) With about 25% for note:: vs 75% for NOTE::, it's a bit premature to make a fuss about it.

[mkoeppe]

Is there a patchbot script that checks on these things?

[dimpase]

IMHO one wants to propose changes like this, which are more of matter of taste than anything else, it's more appropriate to push a commit to fix it, instead of keep setting this to "needs work". Because the latter isn't really the case.

[tscrim]

Instead of sweeping things under the rug, let's make sure things are done right. These are not a "matter of taste" but conventions we have set in Sage that we should adhere to. Here's you're branch, where I noticed some other things that needed fixing too.

---

New commits:

​e1a96f5

Fixing things that need to be fixed before a positive review.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​27ab618	Trac 22178: change two instances of '.. note' to '.. NOTE'.
​ffcc00f	Merge branch 'u/jhpalmieri/22178SageDevel' of git://trac.sagemath.org/sage into u/tscrim/22178SageDevel

[dimpase]

Note NOTE vs note is not a written convention. Anyway.

[tscrim]

See ​http://doc.sagemath.org/html/en/developer/coding_basics.html

[dimpase]

Replying to tscrim:

See ​http://doc.sagemath.org/html/en/developer/coding_basics.html

A NOTE block for tips/tricks (optional).

Does it say it must be all-caps? No. Actually I don't like all-caps, they scream too much at your face...

[mkoeppe]

I know this ticket is already on positive review, but I'd like to suggest to change it to

make V=0

This makes the output much less overwhelming.

[dimpase]

Actually I never heard about V=0. Perhaps it's less common knowledge than you think, and a more detailed explanation ought to be added.

[jhpalmieri]

We already have an installation manual where fine points of make should go. I would suggest documenating make V=0 there, and perhaps putting a pointer to the installation manual in the developer's manual.

[mkoeppe]

I added this feature some time last year in #21539. Looks like I forgot to advertise it in the documentation.

[mkoeppe]

#22227 adds the documentation of make V=0.