Opened 4 months ago

Closed 2 months ago

#34593 closed defect (fixed)

Document and manage temporary directories

Reported by: jhpalmieri Owned by:
Priority: critical Milestone: sage-9.8
Component: misc Keywords:
Cc: Merged in:
Authors: John Palmieri Reviewers: Matthias Koeppe
Report Upstream: N/A Work issues:
Branch: c729c7b (Commits, GitHub, GitLab) Commit: c729c7be2d426cd2ca54d47ba2e823eb25469efa
Dependencies: Stopgaps:

Status badges

Description (last modified by mkoeppe)

This is a followup to #33213. Some users prefer (or need) to control where temporary files are created, and we should document this:

Possible follow-up:

Change History (16)

comment:1 Changed 4 months ago by jhpalmieri

Of course I don't know where we should document this, since it's a Python feature which may be used in various components of Sage. I don't know the natural home for it.

comment:2 Changed 4 months ago by egourgoulhon

Regarding the ticket title, IMHO the objective should be more "manage" than "document" since 3D plots are badly broken in Sage 9.7 for (at least) Ubuntu 22.04 users with the default settings, as revealed by https://ask.sagemath.org/question/64192/temporary-html-files-location-in-sage-97/

comment:3 Changed 4 months ago by egourgoulhon

Priority: majorcritical

comment:4 Changed 4 months ago by jhpalmieri

Should Sage itself set the environment variable TMPDIR? (If the user hasn't set it, I mean.)

Last edited 4 months ago by jhpalmieri (previous) (diff)

comment:5 in reply to:  4 ; Changed 4 months ago by egourgoulhon

Replying to John Palmieri:

Should Sage itself set the environment variable TMPDIR? (If the user hasn't set it, I mean.)

This could a solution, for instance defining TMPDIR to be a subdirectory of DOT_SAGE. But there may be some undesirable effects, e.g. for Sage running on servers or SageMathCell. Also would it be desirable/doable to clean that directory when the Sage session ends?

comment:6 Changed 4 months ago by jhpalmieri

Description: modified (diff)

comment:7 in reply to:  5 Changed 4 months ago by dimpase

Replying to Eric Gourgoulhon:

Replying to John Palmieri:

Should Sage itself set the environment variable TMPDIR? (If the user hasn't set it, I mean.)

This could a solution, for instance defining TMPDIR to be a subdirectory of DOT_SAGE. But there may be some undesirable effects, e.g. for Sage running on servers or SageMathCell. Also would it be desirable/doable to clean that directory when the Sage session ends?

I beleive that the cleanup is automatic, using Python's atexit callback. https://docs.python.org/3/library/atexit.html

(there is of course always a chance of a segfault-like abnormal exit, which might leave rubbish after itself:

The functions registered via this module are not called when the program is killed
by a signal not handled by Python, when a Python fatal internal error is detected,
or when os._exit() is called.

This is what we have in src/misc/temporary_file.py:

import tempfile

import atexit

# Until tmp_dir() and tmp_filename() are removed, we use this directory
# as the parent for all temporary files & directories created by them.
# This lets us clean up after those two functions when sage exits normally
# using an atexit hook
TMP_DIR_FILENAME_BASE=tempfile.TemporaryDirectory()
atexit.register(lambda: TMP_DIR_FILENAME_BASE.cleanup())

comment:8 Changed 4 months ago by dimpase

In an environment such as SageMathCell, I think one user runs many Sage instances, so one cannot clean anything on startup without risking to affect other instances.

comment:9 Changed 3 months ago by mkoeppe

Let's move the documentation of runtime environment variables, starting at "Sage uses the following environment variables when it runs:" https://doc.sagemath.org/html/en/installation/source.html to the "Launching section" and add TMPDIR there (perhaps pointer to Python runtime environment variables)

comment:12 Changed 2 months ago by jhpalmieri

Branch: u/jhpalmieri/document-TMPDIR

comment:13 Changed 2 months ago by jhpalmieri

Authors: John Palmieri
Commit: c729c7be2d426cd2ca54d47ba2e823eb25469efa
Status: newneeds_review

Here is an addition to the documentation.


New commits:

c729c7btrac 34593: document the variable TMPDIR.

comment:14 Changed 2 months ago by mkoeppe

Description: modified (diff)
Reviewers: Matthias Koeppe
Status: needs_reviewpositive_review

comment:15 Changed 2 months ago by jhpalmieri

Thank you!

comment:16 Changed 2 months ago by vbraun

Branch: u/jhpalmieri/document-TMPDIRc729c7be2d426cd2ca54d47ba2e823eb25469efa
Resolution: fixed
Status: positive_reviewclosed
Note: See TracTickets for help on using tickets.