Opened 8 months ago

Closed 7 months ago

#29281 closed defect (fixed)

saving 3d plots to disk seems to be completely broken in sage-9.0

Reported by: was Owned by:
Priority: major Milestone: sage-9.1
Component: graphics Keywords:
Cc: Merged in:
Authors: John Palmieri Reviewers: William Stein
Report Upstream: N/A Work issues:
Branch: 1f17926 (Commits) Commit: 1f17926cce998f457af5097e2dd3937c3efdd5dd
Dependencies: Stopgaps:

Description

  1. Type sphere().save('a.png')
  2. Get a stacktrace that looks very much like a Python3 porting mistake:
    ~$ sage
    ┌────────────────────────────────────────────────────────────────────┐
    │ SageMath version 9.0, Release Date: 2020-01-01                     │
    │ Create a "Sage Worksheet" file for the notebook interface.         │
    │ Enhanced for CoCalc.                                               │
    │ Using Python 3.7.3. Type "help()" for help.                        │
    └────────────────────────────────────────────────────────────────────┘
    sage: sphere().save('a.png')
    ---------------------------------------------------------------------------
    ValueError                                Traceback (most recent call last)
    <ipython-input-1-fbbed3d16b26> in <module>()
    ----> 1 sphere().save('a.png')
    
    /ext/sage/sage-9.0/local/lib/python3.7/site-packages/sage/plot/plot3d/base.pyx in sage.plot.plot3d.base.Graphics3d.save (build/cythonized/sage/plot/plot3d/base.c:21661)()
       1635         elif ext in ['.bmp', '.png', '.gif', '.ppm', '.tiff', '.tif',
       1636                      '.jpg', '.jpeg']:
    -> 1637             self.save_image(filename, **kwds)
       1638         elif filename.endswith('.spt.zip'):
       1639             scene = self._rich_repr_jmol(**kwds)
    
    /ext/sage/sage-9.0/local/lib/python3.7/site-packages/sage/plot/plot3d/base.pyx in sage.plot.plot3d.base.Graphics3d.save_image (build/cythonized/sage/plot/plot3d/base.c:21219)()
       1564             raise ValueError('unknown image file type: {0}'.format(ext))
       1565         if ext == '.png':
    -> 1566             self._save_image_png(filename, **kwds)
       1567         else:
       1568             png = tmp_filename(ext='.png')
    
    /ext/sage/sage-9.0/local/lib/python3.7/site-packages/sage/plot/plot3d/base.pyx in sage.plot.plot3d.base.Graphics3d._save_image_png (build/cythonized/sage/plot/plot3d/base.c:20856)()
       1526             render.png.save_as(filename)
       1527         elif viewer == 'jmol':
    -> 1528             scene = self._rich_repr_jmol(**opts)
       1529             scene.preview_png.save_as(filename)
       1530         else:
    
    /ext/sage/sage-9.0/local/lib/python3.7/site-packages/sage/plot/plot3d/base.pyx in sage.plot.plot3d.base.Graphics3d._rich_repr_jmol (build/cythonized/sage/plot/plot3d/base.c:7224)()
        259         from sage.interfaces.jmoldata import JmolData
        260         jdata = JmolData()
    --> 261         if not jdata.is_jvm_available():
        262             # We can only use JMol to generate preview if a jvm is installed
        263             from sage.repl.rich_output.output_graphics import OutputImagePng
    
    /ext/sage/sage-9.0/local/lib/python3.7/site-packages/sage/interfaces/jmoldata.py in is_jvm_available(self)
         68             return False
         69 
    ---> 70         java_version_number = int(re.sub(r'.*version "(0\.|1\.)?(\d*)[\s\S]*', r'\2', version))
         71         return java_version_number >= 7
         72 
    
    ValueError: invalid literal for int() with base 10: 'Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/home/user/tmp/ -Xms64m\n11'
    sage: 
    

This breaks SageTex?, probably animation, etc...

See https://github.com/sagemathinc/cocalc/issues/4433

Also, there is another unrelated old ticket about plotting and 3d graphics: https://trac.sagemath.org/ticket/21686

Change History (20)

comment:1 Changed 8 months ago by chapoton

This works fine outside C*Calc. Did you try ?

comment:2 Changed 8 months ago by jhpalmieri

Works for me, too. What does "Enhanced for CoCalc?" mean in the Sage banner? Could some of the enhancements be involved?

comment:3 Changed 8 months ago by chapoton

I think I have seen Harald say something about JAVA options in C*calc somewhere.

comment:4 Changed 8 months ago by was

Thanks for testing. The error message in the stacktrace above is in some code to determine the java version; maybe our java runtime environment on cocalc behaves slightly differently than whatever you are testing on. In the stacktrace, it is trying to turn 'Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/home/user/tmp/ -Xms64m\n11' into an int. Maybe it needs to take only the 11 in the last line? No clue... I'll see if Harald has an idea, since he setup the Java environment.

--

I fixed a possibly similar issue here

https://github.com/sagemathinc/cocalc-docker/issues/69#issuecomment-594981326

where it turned out the permissions on the usr/local/sage/local/share/jmol directory were strangely wrong for normal user usage.

Whatever is happening with sage-9.0 might involve some similar permissions issue or something else (no clue).

comment:5 Changed 8 months ago by jhpalmieri

What does "java -version" say on your machine?

comment:6 Changed 8 months ago by jhpalmieri

This is what I see in Sage:

sage: from sage.cpython.string import bytes_to_str
sage: import subprocess
sage: import re
sage: version = bytes_to_str(subprocess.check_output(['java', '-version'], stderr=subprocess.STDOUT))
sage: version
'java version "1.8.0_172"\nJava(TM) SE Runtime Environment (build 1.8.0_172-b11)\nJava HotSpot(TM) 64-Bit Server VM (build 25.172-b11, mixed mode)\n'
sage: print(version)
java version "1.8.0_172"
Java(TM) SE Runtime Environment (build 1.8.0_172-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.172-b11, mixed mode)

sage: int(re.sub(r'.*version "(0\.|1\.)?(\d*)[\s\S]*', r'\2', version))
8

It looks like something in this chain of commands is not working as expected.

comment:7 Changed 8 months ago by was

On cocalc (as anybody can reproduce in any project):

~/cocalc/src/smc-webapp$ java -version
Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/home/user/tmp/ -Xms64m
openjdk version "11.0.6" 2020-01-14
OpenJDK Runtime Environment (build 11.0.6+10-post-Ubuntu-1ubuntu118.04.1)
OpenJDK 64-Bit Server VM (build 11.0.6+10-post-Ubuntu-1ubuntu118.04.1, mixed mode, sharing)
~/cocalc/src/smc-webapp$ sage-9.0
┌────────────────────────────────────────────────────────────────────┐
│ SageMath version 9.0, Release Date: 2020-01-01                     │
│ Create a "Sage Worksheet" file for the notebook interface.         │
│ Enhanced for CoCalc.                                               │
│ Using Python 3.7.3. Type "help()" for help.                        │
└────────────────────────────────────────────────────────────────────┘
sage: from sage.cpython.string import bytes_to_str
sage: import subprocess
sage: import re
sage: version = bytes_to_str(subprocess.check_output(['java', '-version'], stderr=subprocess.STDOUT))
....: 
sage: version
'Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/home/user/tmp/ -Xms64m\nopenjdk version "11.0.6" 2020-01-14\nOpenJDK Runtime Environment (build 11.0.6+10-post-Ubuntu-1ubuntu118.04.1)\nOpenJDK 64-Bit Server VM (build 11.0.6+10-post-Ubuntu-1ubuntu118.04.1, mixed mode, sharing)\n'
sage: print(version)
Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/home/user/tmp/ -Xms64m
openjdk version "11.0.6" 2020-01-14
OpenJDK Runtime Environment (build 11.0.6+10-post-Ubuntu-1ubuntu118.04.1)
OpenJDK 64-Bit Server VM (build 11.0.6+10-post-Ubuntu-1ubuntu118.04.1, mixed mode, sharing)

sage: int(re.sub(r'.*version "(0\.|1\.)?(\d*)[\s\S]*', r'\2', version))
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-7-67eb0f74af21> in <module>()
----> 1 int(re.sub(r'.*version "(0\.|1\.)?(\d*)[\s\S]*', r'\2', version))

ValueError: invalid literal for int() with base 10: 'Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/home/user/tmp/ -Xms64m\n11'

Maybe that regular expression isn't quite good enough...

Last edited 8 months ago by was (previous) (diff)

comment:8 Changed 8 months ago by was

Look:

wstein@kucalc:~/cocalc-docker$ export _JAVA_OPTIONS="-Xms64m"
wstein@kucalc:~/cocalc-docker$ java -version
Picked up _JAVA_OPTIONS: -Xms64m
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (build 1.8.0_242-8u242-b08-0ubuntu3~16.04-b08)
OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)

So anybody who happens to have the env variable _JAVA_OPTIONS set will find that saving 3d graphics to disk is broken in sage-9.0....

comment:9 Changed 8 months ago by jhpalmieri

I don't know why it's using re.sub instead of re.search. Can you test this?

  • src/sage/interfaces/jmoldata.py

    diff --git a/src/sage/interfaces/jmoldata.py b/src/sage/interfaces/jmoldata.py
    index 8f291fb954..86b21d9722 100644
    a b class JmolData(SageObject): 
    6767        except (subprocess.CalledProcessError, OSError):
    6868            return False
    6969
    70         java_version_number = int(re.sub(r'.*version "(0\.|1\.)?(\d*)[\s\S]*', r'\2', version))
     70        m = re.search(r'.*version "(0\.|1\.)?(\d*)[\s\S]*', version)
     71        java_version_number = int(m.group(2))
    7172        return java_version_number >= 7
    7273
    7374    def export_image(self,

comment:10 Changed 8 months ago by jhpalmieri

Or a smaller change:

  • src/sage/interfaces/jmoldata.py

    diff --git a/src/sage/interfaces/jmoldata.py b/src/sage/interfaces/jmoldata.py
    index 8f291fb954..40984bdc33 100644
    a b class JmolData(SageObject): 
    6767        except (subprocess.CalledProcessError, OSError):
    6868            return False
    6969
    70         java_version_number = int(re.sub(r'.*version "(0\.|1\.)?(\d*)[\s\S]*', r'\2', version))
     70        java_version_number = int(re.sub(r'.*version "(0\.|1\.)?(\d*)[\s\S]*', r'\2', version, flags=re.S))
    7171        return java_version_number >= 7
    7272
    7373    def export_image(self,

comment:11 Changed 8 months ago by was

Yes, your latter suggestion using flags works for me:

~/cocalc/src/smc-webapp$ sage
┌────────────────────────────────────────────────────────────────────┐
│ SageMath version 9.0, Release Date: 2020-01-01                     │
│ Create a "Sage Worksheet" file for the notebook interface.         │
│ Enhanced for CoCalc.                                               │
│ Using Python 3.7.3. Type "help()" for help.                        │
└────────────────────────────────────────────────────────────────────┘
sage: sage: from sage.cpython.string import bytes_to_str
....: sage: import subprocess
....: sage: import re
....: sage: version = bytes_to_str(subprocess.check_output(['java', '-version'], stderr=subprocess.STDOUT))
....: 
sage: int(re.sub(r'.*version "(0\.|1\.)?(\d*)[\s\S]*', r'\2', version, flags=re.S))
11
sage: print(version)
Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/home/user/tmp/ -Xms64m
openjdk version "11.0.6" 2020-01-14
OpenJDK Runtime Environment (build 11.0.6+10-post-Ubuntu-1ubuntu118.04.1)
OpenJDK 64-Bit Server VM (build 11.0.6+10-post-Ubuntu-1ubuntu118.04.1, mixed mode, sharing)

Last edited 8 months ago by was (previous) (diff)

comment:12 Changed 8 months ago by jhpalmieri

  • Branch set to u/jhpalmieri/jmol-regexp

comment:13 Changed 8 months ago by jhpalmieri

  • Authors set to John Palmieri
  • Commit set to 1f17926cce998f457af5097e2dd3937c3efdd5dd
  • Status changed from new to needs_review

The flag re.S means that . matches that "dot matches all", including a newline. So in case "java -version" prints lines before version "...", those lines are swallowed up by the . in .*version, which is what we want.


New commits:

1f17926trac 29281: modify regular expression when testing java veraion
Last edited 8 months ago by jhpalmieri (previous) (diff)

comment:14 Changed 8 months ago by was

Positive review since it works for me. Thanks!

comment:15 Changed 8 months ago by was

  • Status changed from needs_review to positive_review

comment:16 Changed 8 months ago by vbraun

Please fill out the reviewer name...

comment:17 Changed 8 months ago by vbraun

  • Status changed from positive_review to needs_work

comment:18 Changed 8 months ago by was

  • Reviewers set to William Stein
  • Status changed from needs_work to positive_review

Done. Sorry for forgetting that!

comment:20 Changed 7 months ago by vbraun

  • Branch changed from u/jhpalmieri/jmol-regexp to 1f17926cce998f457af5097e2dd3937c3efdd5dd
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.