Opened 11 years ago

Closed 10 years ago

#11932 closed enhancement (fixed)

Allow running Sage with a temporary DOT_SAGE directory

Reported by: Jeroen Demeyer Owned by: Leif Leonhardy
Priority: major Milestone: sage-5.1
Component: scripts Keywords:
Cc: John Palmieri Merged in: sage-5.1.beta0
Authors: John Palmieri Reviewers: Jeroen Demeyer
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: #11926 Stopgaps:

Status badges

Description (last modified by Jeroen Demeyer)

Version which deals with matplotlib issue: apply

Attachments (5)

trac_11932-scripts.patch (920 bytes) - added by John Palmieri 11 years ago.
scripts repo
trac_11932-doc.patch (882 bytes) - added by John Palmieri 11 years ago.
documentation patch (Sage library)
trac_11932-root.patch (1.5 KB) - added by John Palmieri 10 years ago.
root repo
trac_11932-root.v2.patch (2.4 KB) - added by John Palmieri 10 years ago.
trac_11932-sage.v2.patch (726 bytes) - added by John Palmieri 10 years ago.

Download all attachments as: .zip

Change History (37)

comment:1 Changed 11 years ago by Jeroen Demeyer

Cc: John Palmieri added

I copied some patches from #11926 (and make some small changes to John's patch).

comment:2 Changed 11 years ago by Jeroen Demeyer

Dependencies: #11926

comment:3 Changed 11 years ago by Leif Leonhardy

I'd rename the function (as it creates a temporary directory), and not use a global variable for the "return value".

The function could either take the variable name as its parameter (such that sage_make_tmp_dir DIR assigns the directory name to DIR), or just "echo" the result (as mktemp already does), such that one writes DIR=sage_make_tmp_dir or whatever.

comment:4 in reply to:  3 Changed 11 years ago by Leif Leonhardy

Replying to leif:

or just "echo" the result (as mktemp already does), such that one writes DIR=sage_make_tmp_dir or whatever.

DIR=`sage_make_tmp_dir` of course.

comment:5 Changed 11 years ago by John Palmieri

Description: modified (diff)
Status: newneeds_review

Here's a version in which the function echoes the path of the temp dir.

This works best with the patch from #11924: without applying the patch there, the parent directory of the temporary .sage directory gets polluted with pickle cache files.

comment:6 Changed 11 years ago by Jeroen Demeyer

Milestone: sage-4.7.3

Milestone sage-4.7.3 deleted

comment:7 Changed 11 years ago by John Palmieri

Description: modified (diff)
Milestone: sage-5.0

Rebased to #11073. Documentation added.

comment:8 Changed 11 years ago by Jeroen Demeyer

Authors: John Palmieri
Reviewers: Jeroen Demeyer
Status: needs_reviewneeds_work

The following seems to be portable:

mktemp -d ${TMPDIR:-/tmp}/dotsageXXXXXX

so I would use this instead of sage_mktempdir.

sage-sage needs to be replaced by sage.

Finally, why not simply run

./sage --norc </dev/null >/dev/null 2>/dev/null

in spkg-install?

comment:9 Changed 11 years ago by Jeroen Demeyer

I meant "in spkg/install".

comment:10 Changed 11 years ago by John Palmieri

Description: modified (diff)
Status: needs_workneeds_review

Fixed. The patch to spkg/install is no longer valid; instead, we need to patch the sage-starts script.

comment:11 Changed 11 years ago by John Palmieri

Status: needs_reviewneeds_work

This is not working, and I'm probably making some bash error in the sage script. When I run

sage -c 'print "hello"'

it works as it should: no banner, just "hello", then a new shell prompt. When I run

sage --norc -c 'print "hello"'

it starts up Sage, prints a banner, prints "hello", and then gives the Sage prompt. As a consequence, the sage-starts script is not working right. What am I doing wrong?

comment:12 Changed 11 years ago by John Palmieri

Status: needs_workneeds_review

Never mind, I've found various work-arounds. The best seems to be to replace sage "@$" with source sage "@$".

comment:13 Changed 11 years ago by John Palmieri

No, that doesn't work: it doesn't clean up the temporary directory afterward. Using "$SAGE_ROOT/sage" "$@" seems to work, but is that a good solution?

comment:14 in reply to:  13 Changed 11 years ago by Jeroen Demeyer

Replying to jhpalmieri:

Using "$SAGE_ROOT/sage" "$@" seems to work, but is that a good solution?

I don't see how calling sage could be different from calling $SAGE_ROOT/sage, are you sure the former doesn't work (I haven't tried).

comment:15 Changed 11 years ago by John Palmieri

Calling sage at this point in the script will call the script spkg/bin/sage, because that directory is now in the path, so it is conceivably different from calling $SAGE_ROOT/sage. In practice, it does behave differently, as I mentioned above: using $SAGE_ROOT/sage in the script, the command sage --norc -c 'print "hello"' works as expected, while using just sage, the command prints the Sage banner, prints "hello", and then you end up at a Sage prompt, running Sage interactively.

comment:16 Changed 11 years ago by Jeroen Demeyer

Status: needs_reviewneeds_work

I confirm your observation but I still don't understand what's going on. I'm investigating...

comment:17 Changed 11 years ago by Jeroen Demeyer

Okay, found it. "sage" refers to the shell function

sage() {
    sage_setup
    sage-ipython "$@" -i
}

We should probably rename this function to run_sage or so...

In the mean time, replace $SAGE_ROOT/sage "$@" by

command sage "$@"

Help of command:

command: command [-pVv] command [arg ...]
    Execute a simple command or display information about commands.

    Runs COMMAND with ARGS suppressing  shell function lookup, or display
    information about the specified COMMANDs.  Can be used to invoke commands
    on disk when a function with the same name exists.

    Options:
      -p        use a default value for PATH that is guaranteed to find all of
        the standard utilities
      -v        print a description of COMMAND similar to the `type' builtin
      -V        print a more verbose description of each COMMAND

    Exit Status:
    Returns exit status of COMMAND, or failure if COMMAND is not found.

comment:18 Changed 11 years ago by John Palmieri

Status: needs_workneeds_review

Ah, good, thank you. I agree, having a function named "sage" is probably a bad idea, but changing it should be dealt with elsewhere. (It would require a number of changes in the sage script, and a correspondingly large number of tests to make sure everything works.)

comment:19 Changed 11 years ago by Jeroen Demeyer

On an entirely different topic: I don't like the --norc option name. In the light of #12647, I would expect that option to suppress the reading of the sagerc script (and/or the init.sage script). I would only support --nodotsage.

comment:20 Changed 11 years ago by John Palmieri

Here are new patches which just use --nodotsage. (I also considered --tmpdotsage, but it doesn't matter too much, as long as it's documented.)

Changed 11 years ago by John Palmieri

Attachment: trac_11932-scripts.patch added

scripts repo

Changed 11 years ago by John Palmieri

Attachment: trac_11932-doc.patch added

documentation patch (Sage library)

comment:21 Changed 11 years ago by Leif Leonhardy

It would be nice to also have --nosagerc, and let --nodotsage by default not bypass [$DOT_SAGE/]sagerc (if it exists; of course no directory should be created if it doesn't).

comment:22 Changed 11 years ago by John Palmieri

Perhaps on another ticket...

comment:23 Changed 10 years ago by Jeroen Demeyer

Unfortunately, this doesn't quite work properly since DOT_SAGE is changed after reading sage-env. The following variables still refer to $HOME/.sage:

jdemeyer@arcanis:/usr/local/src/sage-5.0.beta13$ ./sage --nodotsage --sh -c env | fgrep "$HOME/.sage"
MPLCONFIGDIR=/home/jdemeyer/.sage//matplotlib-1.1.0
SAGE_STARTUP_FILE=/home/jdemeyer/.sage//init.sage
SAGE_TESTDIR=/home/jdemeyer/.sage//tmp

I suggest moving the --nodotsage check before the sourcing of sage-env.

comment:24 Changed 10 years ago by Jeroen Demeyer

Status: needs_reviewneeds_work

comment:25 Changed 10 years ago by John Palmieri

Status: needs_workneeds_review

Okay, here's a new root patch.

comment:26 Changed 10 years ago by Jeroen Demeyer

Just one more issue: now that you don't source sage-env, the command sage looks for sage in the system PATH. I suggest to replace

command sage "$@"

by

command "$0" "$@"

If you make this change, you can give this ticket positive review.

Furthermore, I would like to point out that $HOME/.sage/matplotlib-1.1.0 is still created by sage-env when building Sage. But that shouldn't stop this ticket.

Changed 10 years ago by John Palmieri

Attachment: trac_11932-root.patch added

root repo

Changed 10 years ago by John Palmieri

Attachment: trac_11932-root.v2.patch added

comment:27 Changed 10 years ago by John Palmieri

Description: modified (diff)

Here are two versions. If we want the easy root, the first version just makes the change

  • spkg/bin/sage

    diff --git a/spkg/bin/sage b/spkg/bin/sage
    a b cd "$SAGE_ROOT" 
    238238if [ "$1" = '--nodotsage' ]; then
    239239    export DOT_SAGE=`mktemp -d ${TMPDIR:-/tmp}/dotsageXXXXXX`
    240240    shift
    241     command sage "$@"
     241    command "$0" "$@"
    242242    status=$?
    243243    rm -rf "$DOT_SAGE"
    244244    exit $status

The second version deals (I think) with the matplotlib issue, creating the config directory in sage.misc.misc, where other subdirectories of DOT_SAGE are created. It's not a very significant difference: the file trac_11932-sage.v2.patch and these changes to the root patch:

  • spkg/bin/sage-env

    diff --git a/spkg/bin/sage-env b/spkg/bin/sage-env
    a b fi 
    281281
    282282if [ "$DOT_SAGE" = "" ]; then
    283283    # It is *not* an error if this directory does not exist, it will
    284     # be created in spkg/bin/sage.  This also works if $HOME/.sage is a
    285     # symbolic link to a non-existing directory.
     284    # be created in spkg/bin/sage or devel/sage/sage/misc/misc.py.
     285    # This also works if $HOME/.sage is a symbolic link to a
     286    # non-existing directory.
    286287    DOT_SAGE=`resolvelinks "$HOME/.sage"`
    287288
    288289    # In theory, DOT_SAGE is not required to have a trailing slash.
    if [ -f "$MPLINITFILE" ]; then 
    340341        MPLCONFIGDIR="$DOT_SAGE/matplotlib"
    341342    fi
    342343    export MPLCONFIGDIR
    343     # Create the directory, because matplotlib doesn't seem to be happy otherwi
    344     mkdir -p "$MPLCONFIGDIR"
     344    # The directory is created when Sage starts (see sage.misc.misc).
    345345fi
    346346
    347347# Add some directories to $LD_LIBRARY_PATH:

If you have time to look at the second version, please consider it. If you would rather deal with just the first version here, we can move the other version to a new ticket.

comment:28 Changed 10 years ago by Jeroen Demeyer

Milestone: sage-5.0sage-5.1

For the Sage patch, what about:

try:
    # Create the matplotlib config directory
    sage_makedirs(os.environ["MPLCONFIGDIR"])
except KeyError:
    pass

I don't see a reason to create a directory when the environment variable doesn't exist.

comment:29 Changed 10 years ago by John Palmieri

Since we set MPLCONFIGDIR in sage-env, there is no reason for it not to be set (unless someone does something risky and unsupported like try to import the Sage library using the system's Python). But I made the change anyway.

Changed 10 years ago by John Palmieri

Attachment: trac_11932-sage.v2.patch added

comment:30 Changed 10 years ago by Jeroen Demeyer

Description: modified (diff)

Looks good and works well.

I'm currently running

sage --nodotsage -tp 2 devel/sage/sage

comment:31 Changed 10 years ago by Jeroen Demeyer

Status: needs_reviewpositive_review
All tests passed!

comment:32 Changed 10 years ago by Jeroen Demeyer

Merged in: sage-5.1.beta0
Resolution: fixed
Status: positive_reviewclosed
Note: See TracTickets for help on using tickets.