Opened 2 years ago

Last modified 2 years ago

#28444 closed defect

Fix backwards incompatibility of unpickling in Python 3 — at Initial Version

Reported by: SimonKing Owned by:
Priority: blocker Milestone: sage-8.9
Component: python3 Keywords: unpickling UnicodeError backwards compatibility
Cc: Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description

The following happens in Sage with Python-3, when trying to unpickle the attached file State.sobj:

sage: class unpickle_old_mtx:
....:     def __call__(self, *args, **kwds):
....:         return None
....:     
sage: register_unpickle_override('pGroupCohomology.mtx', 'MTX_unpickle_class', unpickle_old_mtx)
sage: X = load('/home/king/Projekte/coho/tests/State.sobj')
---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
<ipython-input-3-74e7e91f9969> in <module>()
----> 1 X = load('/home/king/Projekte/coho/tests/State.sobj')

/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.load (build/cythonized/sage/misc/persist.c:2824)()
    149 
    150     ## Load file by absolute filename
--> 151     with open(filename, 'rb') as fobj:
    152         X = loads(fobj.read(), compress=compress)
    153     try:

/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.load (build/cythonized/sage/misc/persist.c:2774)()
    150     ## Load file by absolute filename
    151     with open(filename, 'rb') as fobj:
--> 152         X = loads(fobj.read(), compress=compress)
    153     try:
    154         X._default_filename = os.path.abspath(filename)

/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.loads (build/cythonized/sage/misc/persist.c:7270)()
    967 
    968     unpickler = SageUnpickler(io.BytesIO(s))
--> 969     return unpickler.load()
    970 
    971 

UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 0: ordinal not in range(128)

Doing the same in Sage-with-Python-2, one gets

sage: class unpickle_old_mtx:
....:     def __call__(self, *args, **kwds):
....:         return None
....:     
sage: register_unpickle_override('pGroupCohomology.mtx', 'MTX_unpickle_class', unpickle_old_mtx)
sage: X = load('/home/king/Projekte/coho/tests/State.sobj')
sage: X
([[1,
   [(2, 1),
    [None,
     [None,
      '/home/king/SPKG/database/8gp3/sgp/8gp3sg1_1',
      '/home/king/SPKG/database/8gp3/sgp/8gp3sg1_2']]]],
  [2,
   [(4, 2),
    [None,
     [None,
      '/home/king/SPKG/database/8gp3/sgp/8gp3sg2_1',
      '/home/king/SPKG/database/8gp3/sgp/8gp3sg2_2']]]],
  [3,
   [(4, 2),
    [None,
     [None,
      '/home/king/SPKG/database/8gp3/sgp/8gp3sg3_1',
      '/home/king/SPKG/database/8gp3/sgp/8gp3sg3_2']]]]],
 [2, 1, 1],
 1,
 1,
 '8gp3/',
 ['b_1_0*b_1_1'],
 '8gp3/dat/',
 [((4, 2), '/home/king/.sage/pGroupCohomology/db/4gp2/H4gp2'),
  ((2, 1), '/home/king/.sage/pGroupCohomology/db/2gp1/H2gp1')],
 '8gp3/dat/',
 [],
 '8gp3/sgp/',
 2,
 [2, 3],
 [2, 2],
 2,
 2,
 ['b_1_0*b_1_1'],
 [[2, 'c_2_2', None, 1, 0],
  [1, 'b_1_0', None, 0, 0],
  [1, 'b_1_1', None, 0, 0]],
 [[0, [['1', ['1']]]],
  [1, [['b_1_0', ['b_1_0']], ['b_1_1', ['b_1_1']]]],
  [2,
   [['c_2_2', ['c_2_2']],
    ['b_1_0', ['b_1_0^2']],
    ['b_1_1', ['b_1_0*b_1_1', 'b_1_1^2']]]]],
 {},
 0.24,
 True,
 'Monomials',
 2,
 None,
 '',
 0,
 '8gp3',
 '8gp3/dat/R8gp3.sobj',
 1,
 ['0'],
 ['b_1_0 + b_1_1'],
 None,
 [('_max_module_deg', 0),
  ('sgpDickson', [((4, 2), [[1, 'c_1_1', None]])]),
  ('useFactorization', True),
  ('useElimination', False),
  ('GroupDescr', 'Dihedral group of order 8'),
  ('_key', ((8, 3), '/home/king/SPKG/database/8gp3/dat/State')),
  ('auto', 4),
  ('Restriction_2', ['c_1_0*c_1_1+c_1_0^2', 'c_1_1', '0']),
  ('GroupName', 'D8'),
  ('DicksonExp', 3),
  ('_parameters_do_exist', True),
  ('KeepBases', None),
  ('completeGroebner', True),
  ('_SymondsTestdata', ['c_2_2', 'b_1_1', 'b_1_0']),
  ('_parameters_for_criterion', ['c_2_2', 'b_1_1', 'b_1_0']),
  ('root', '/home/king/SPKG/database/'),
  ('Restriction_3', ['c_1_0*c_1_1+c_1_0^2', '0', 'c_1_1']),
  ('_method', 'Symonds')],
 [(('dependent_parameters',), [['c_2_2', 'b_1_1', 'b_1_0'], 2])])

I believe that a backwards incompatible change of pickling is a blocker for Python-3 support.

Change History (1)

Changed 2 years ago by SimonKing

File that cannot be unpickled in Python-3

Note: See TracTickets for help on using tickets.