id summary reporter owner description type status priority milestone component resolution keywords cc merged author reviewer upstream work_issues branch commit dependencies stopgaps
12016 parallelism in Sage: just use value of 'MAKE' jhpalmieri GeorgSWeber "The various parallel aspects of Sage should be controlled by setting the `-j` (possible also `-l`) flags in `MAKE` or `MAKEFLAGS`. That is, if `MAKE='make -j16'`, then
- running `make` will build spkg's in parallel, using 16 processes (this was done in #11959). This is standard `make` behaviour, but we need to patch `spkg/standard/deps` to ensure that `make` recognizes that we are doing a recursive make.
- running `make ptestlong` or `sage -tp 0 ` will doctest in parallel using 16 threads. If the `-j` flag in `MAKE` is not set, then determine the number of threads as before: `min(8, cpu_count())`.
- running `./sage -b` will build the Sage library using 16 threads. If the `-j` flag in `MAKE` is not set, then use only 1 thread.
'''Testing this ticket''': you can set the environment variable `SAGE_NUM_CORES` to the number of cores you want to pretend to have. For example, running
{{{
SAGE_NUM_CORES=24 make ptestlong
}}}
should run 8 threads (see `sage-num-threads.py`; this is undocumented because the only purpose I see is for testing this ticket).
'''Notes''':
With the patches applied, building spkgs in parallel works well, except for race conditions in:
* python (#12096)
* singular (#12137)
* zlib (#12138)
* mpir (#12139)
* atlas on Solaris (#12312)
and a ""jobserver unavailable"" warning in:
* ntl
* singular (#12137)
* rubiks
'''Apply''':
1. [attachment:12016-root.patch] to the `SAGE_ROOT` repository.
1. [attachment:12016-base.patch] to `spkg/base`.
1. [attachment:12016-scripts.patch] and [attachment:trac_12016-scripts-ref.patch] to the `SCRIPTS` repository.
1. [attachment:12016-sage.patch] to the Sage library.
See also: #6495 to implement the same behavior for doc building." enhancement closed critical sage-4.8 build fixed jdemeyer leif sage-4.8.alpha5 John Palmieri, Jeroen Demeyer John Palmieri, Jeroen Demeyer N/A sage-4.8.alpha4