sage <script> does not run sage-cleaner

[vbraun]

When you are running a long computation, you are probably putting your sage program in a script.sage file and then run sage script.sage in a screen'ed terminal. In that case, sage-cleaner is not run. So if you use @parallel, each fork creates a new subdirectory $DOT_SAGE/temp/hostname/pid that is not being cleaned up. This can easily reach a limit of the file system, for example ext2/ext3 are limited to 2¹⁵ subdirectories per directory.

A typical error message then looks like

[Errno 31] Too many links: '/home/vbraun/.sage//temp/hostname/26457/'

Apply trac_11818_run_sage_setup.patch​ to the sage-scripts repository ($SAGE_LOCAL/bin)

[vbraun]

Steps to reproduce:

[vbraun@volker-desktop ~]$ ls ~/.sage/temp/volker_desktop.stp.dias.ie/ | wc
      0       0       0
[vbraun@volker-desktop ~]$ cat > script.sage <<EOF
@parallel(verbose=True)
def f(i):
    return i

list(f(range(0,100)))
EOF
[vbraun@volker-desktop ~]$ sage script.sage
[vbraun@volker-desktop ~]$ ls ~/.sage/temp/volker_desktop.stp.dias.ie/ | wc
    101     101     505

[jhpalmieri]

Is a change like this good enough?

sage-sage

@@ -1021 +1021 @@
        exit $?
    fi
    cd "$SAGE_LOCAL/bin/"
+   SAGE_BANNER="no" sage_setup
    sage-run "$CUR" "$@"
    exit $?
 fi

[vbraun]

Yes, that is it!

[vbraun]

John's patch

[vbraun]

I am giving this positive review :-)

[leif]

I'll see if I can include this patch bomb into Sage 4.7.2.alpha3...

[leif]

P.S.:

Some doctest in sage/parallel/* analoguous to what Volker did wouldn't be bad.

[vbraun]

But during parallel doctests there should be a large and fluctuating number of directories in the temp dir. So I don't see an easy way of testing this...

[leif]

Replying to leif:

P.S.:

Some doctest in sage/parallel/* analoguous to what Volker did wouldn't be bad.

... which perhaps isn't that easy.

[jhpalmieri]

I had an idea how to doctest this, but it passes doctests regardless of whether the patch here has been applied. I'll post it here in case you have any suggestions.

[jhpalmieri]

for Sage library, but currently not working properly

[vbraun]

When you are doctesting this, the sage-cleaner was already started by the doctesting process. In a single-threaded doctest you could kill the sage-cleaner and check that it is being restarted, but with parallel testing that would just be a race condition.

[jhpalmieri]

Replying to vbraun:

When you are doctesting this, the sage-cleaner was already started by the doctesting process.

Right, I was just coming to the same conclusion, although I'm not sure where sage-cleaner is actually started when you run "sage -t FILE".

In any case, it looks difficult, or at least annoying, to test this from within Sage's doctesting framework.