Opened 3 years ago
Last modified 3 years ago
#28414 closed defect
p_group_cohomology does not build on Python3 — at Version 113
| Reported by: | dimpase | Owned by: | |
|---|---|---|---|
| Priority: | major | Milestone: | sage-9.0 |
| Component: | packages: optional | Keywords: | py3 |
| Cc: | SimonKing | Merged in: | |
| Authors: | Simon King | Reviewers: | |
| Report Upstream: | N/A | Work issues: | |
| Branch: | u/SimonKing/p_group_cohomology_in_py_3 (Commits, GitHub, GitLab) | Commit: | 463806c6cebb3120e20df0f19a216dd529553741 |
| Dependencies: | #28444 | Stopgaps: |
Status badges
Description (last modified by )
The aim of this ticket is to provide a new version of the optional p_group_cohomology spkg so that it builds and passes tests both with python-2 and python-3.
Hopefully the version number 3.3 (two times three) is appropriate.
Change History (114)
comment:1 Changed 3 years ago by
comment:2 Changed 3 years ago by
Simon, I have made some progress in converting this to Python 3. Let me know and I can email you some diff files.
comment:3 Changed 3 years ago by
Hi John, in fact Python-3-compatibility was one of the things that I considered for the next package version. So, if you already have a diff, I'd appreciate to see it!
comment:4 follow-ups: ↓ 5 ↓ 36 Changed 3 years ago by
I just sent email to your Jena address. I can get the package to build, but it doesn't work yet. An illustration of one problem:
sage: load('/Users/palmieri/.sage/pGroupCohomology/db/8gp3/H8gp3.sobj')
/Users/palmieri/Desktop/Sage_stuff/sage_builds/PYTHON3/sage-8.9.beta8/src/bin/sage-ipython:1: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
#!/usr/bin/env sage-python23
WARNING: Files on disk have been moved or are not writeable.
> Will try to recover later.
/Users/palmieri/Desktop/Sage_stuff/sage_builds/PYTHON3/sage-8.9.beta8/local/lib/python3.7/site-packages/sage/repl/rich_output/display_manager.py:623: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
has_rich_repr = isinstance(obj, SageObject) and hasattr(obj, '_rich_repr_')
8gp3: Files on disk have been moved - trying to get things right
---------------------------------------------------------------------------
UnicodeDecodeError Traceback (most recent call last)
<ipython-input-5-9bd7fbf5ad54> in <module>()
----> 1 load('/Users/palmieri/.sage/pGroupCohomology/db/8gp3/H8gp3.sobj')
/Users/palmieri/Desktop/Sage_stuff/sage_builds/PYTHON3/sage-8.9.beta8/local/lib/python3.7/site-packages/IPython/core/displayhook.py in __call__(self, result)
244 self.start_displayhook()
245 self.write_output_prompt()
--> 246 format_dict, md_dict = self.compute_format_data(result)
247 self.update_user_ns(result)
248 self.fill_exec_result(result)
/Users/palmieri/Desktop/Sage_stuff/sage_builds/PYTHON3/sage-8.9.beta8/local/lib/python3.7/site-packages/IPython/core/displayhook.py in compute_format_data(self, result)
148
149 """
--> 150 return self.shell.display_formatter.format(result)
151
152 # This can be set to True by the write_output_prompt method in a subclass
/Users/palmieri/Desktop/Sage_stuff/sage_builds/PYTHON3/sage-8.9.beta8/local/lib/python3.7/site-packages/sage/repl/display/formatter.py in format(self, obj, include, exclude)
198 """
199 # First, use Sage rich output if there is any
--> 200 sage_format, sage_metadata = self.dm.displayhook(obj)
201 assert PLAIN_TEXT in sage_format, 'plain text is always present'
202 if not set(sage_format.keys()).issubset(self.default_mime()):
/Users/palmieri/Desktop/Sage_stuff/sage_builds/PYTHON3/sage-8.9.beta8/local/lib/python3.7/site-packages/sage/repl/rich_output/display_manager.py in displayhook(self, obj)
809 return
810 self._backend.set_underscore_variable(obj)
--> 811 plain_text, rich_output = self._rich_output_formatter(obj, dict())
812 return self._backend.displayhook(plain_text, rich_output)
813
/Users/palmieri/Desktop/Sage_stuff/sage_builds/PYTHON3/sage-8.9.beta8/local/lib/python3.7/site-packages/sage/repl/rich_output/display_manager.py in _rich_output_formatter(self, obj, rich_repr_kwds)
621 rich_output = None
622 plain_text = None
--> 623 has_rich_repr = isinstance(obj, SageObject) and hasattr(obj, '_rich_repr_')
624 if has_rich_repr:
625 rich_output = self._call_rich_repr(obj, rich_repr_kwds)
pGroupCohomology/cohomology.pyx in pGroupCohomology.cohomology.COHO.__getattr__()
/Users/palmieri/Desktop/Sage_stuff/sage_builds/PYTHON3/sage-8.9.beta8/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.load (build/cythonized/sage/misc/persist.c:2824)()
149
150 ## Load file by absolute filename
--> 151 with open(filename, 'rb') as fobj:
152 X = loads(fobj.read(), compress=compress)
153 try:
/Users/palmieri/Desktop/Sage_stuff/sage_builds/PYTHON3/sage-8.9.beta8/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.load (build/cythonized/sage/misc/persist.c:2774)()
150 ## Load file by absolute filename
151 with open(filename, 'rb') as fobj:
--> 152 X = loads(fobj.read(), compress=compress)
153 try:
154 X._default_filename = os.path.abspath(filename)
/Users/palmieri/Desktop/Sage_stuff/sage_builds/PYTHON3/sage-8.9.beta8/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.loads (build/cythonized/sage/misc/persist.c:7270)()
967
968 unpickler = SageUnpickler(io.BytesIO(s))
--> 969 return unpickler.load()
970
971
UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 0: ordinal not in range(128)
comment:5 in reply to: ↑ 4 Changed 3 years ago by
Replying to jhpalmieri:
I just sent email to your Jena address. I can get the package to build, but it doesn't work yet.
Thank you very much, I received the patch.
An illustration of one problem:
sage: load('/Users/palmieri/.sage/pGroupCohomology/db/8gp3/H8gp3.sobj') /Users/palmieri/Desktop/Sage_stuff/sage_builds/PYTHON3/sage-8.9.beta8/src/bin/sage-ipython:1: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead ... UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 0: ordinal not in range(128)
That probably is harmless. I.e., I should use "warning", and apparently in some unpickling method I use "str" where it should be "bytes".
comment:6 Changed 3 years ago by
Yes, and I would like to track down the str vs. bytes issue. It would be nice if I could get basic functionality like CohomologyRing(8,3) to work.
comment:7 follow-up: ↓ 8 Changed 3 years ago by
Changing logger.warn (or coho_logger.warn) to X.warning fixes the deprecation message. I don't know about the str/bytes issue.
comment:8 in reply to: ↑ 7 Changed 3 years ago by
Replying to jhpalmieri:
Changing
logger.warn(orcoho_logger.warn) toX.warningfixes the deprecation message. I don't know about the str/bytes issue.
I did change .warn( into .warning( everywhere (so, no need to include it in a diff).
Concerning str vs. byte: There is only one occurrence of cdef str, and this gets assigned to the output of <some singular object>.name(). Is that output really a string in python-3? If not, it should be changed to cdef bytes. However, so far the error in unpickling stuff looks mysterious to me, as the traceback doesn't mention where the problem actually occurs in my code.
comment:9 follow-up: ↓ 10 Changed 3 years ago by
Could the problem be in the way the .sobj files were produced? What if you regenerate them using Python 3?
comment:10 in reply to: ↑ 9 ; follow-up: ↓ 13 Changed 3 years ago by
Replying to jhpalmieri:
Could the problem be in the way the .sobj files were produced? What if you regenerate them using Python 3?
You mean Python 3 is severely backwards incompatible and cannot deal with old pickles? That would be a show stopper for Sage, I think.
Note that at some point in the past I inserted code to regenerate very old pickles and translate them automatically to a new format. However, I would certainly not like to regenerate several gigabytes of data in the web repository.
Note that the problem occurs together with a warning:
WARNING: Files on disk have been moved or are not writeable. > Will try to recover later. ... 8gp3: Files on disk have been moved - trying to get things right
Maybe that indicates where the problem comes from. So, let's recall how storing the modular cohomology ring of a prime power group is solved.
Very early in the package development I found that is is terribly slow to pack all data of a ring in a single file. It simply takes so long and requires so much memory (at least in some cases) that it is not feasible to automatically store intermediate results of a lengthy computation (which evidently is a reasonable thing to do).
My solution was to not store a ring in a single file but distribute the data in a folder. So, the actual .sobj pickle file only contains basic information but is not stand-alone. In particular, the p_group_cohomology spkg needs to know where the data folder is located, the .sobj file has to be in the data folder and also it needs write permission for otherwise it couldn't do a continuation of the cohomology ring's computation. What is stored in the web repository is a compressed tar file of the data folder (i.e., when the web repository is accessed, then the tar file is downloaded, unpacked, and then the cohomology ring is loaded from there).
By default, the data folder's location is composed from the location of the current workspace (which are global variables) and from the identifier of the group (which is stored in the .sobj file).
The warning mentioned above occurs when the data folder's default location cannot be accessed. And in that case, I do some trickery --- and perhaps that trickery doesn't work in Python 3? Namely:
- When an object is obtained from reading a pickle file, then Python (at least Python 2) stores the original file location as an attribute of the object. However, that attribute is only available after successful unpickling.
- Therefore, when initially the location of the data folder cannot be determined, an unitialised cohomology ring is returned.
- As soon as the uninitialised cohomology ring is touched (I think constructing its repr is enough), initialisation is triggered. Namely, recall that the location of the .sobj file is stored as an attribute of the uninitialised ring and that the .sobj file is supposed to be located in the data folder -- voilà, you can now determine the data folder and finish initialisation of the cohomology ring.
If the .sobj file's location is not stored as an attribute in Python 3 then I have a problem.
comment:11 follow-up: ↓ 12 Changed 3 years ago by
Question: How to test whether something has a type that qualifies it as a filename? Is isinstance(self._property_dict['_need_new_root'],basestring) ok in Python-3?
comment:12 in reply to: ↑ 11 ; follow-up: ↓ 31 Changed 3 years ago by
Replying to SimonKing:
Question: How to test whether something has a type that qualifies it as a filename? Is
isinstance(self._property_dict['_need_new_root'],basestring)ok in Python-3?
Here is what I found in the "internetz" concerning how to test if something is a string in a way that works in python 2 and 3:
import six
if isinstance(obj, six.string_types):
print('obj is a string!')
So, I think I should fix this.
comment:13 in reply to: ↑ 10 ; follow-up: ↓ 88 Changed 3 years ago by
Replying to SimonKing:
If the .sobj file's location is not stored as an attribute in Python 3 then I have a problem.
Hooray:
# in Sage-with-python-2 sage: P = CombinatorialFreeModule(ZZ,'x') sage: save(P,'../../pickle')
# BOTH with python-2 and python-3
sage: B = load('../../pickle.sobj')
sage: B._default_filename
'/home/king/Sage/pickle.sobj'
So, the old trick should still work.
comment:14 Changed 3 years ago by
Currently, I get the following behaviour:
sage: from pGroupCohomology import CohomologyRing sage: CohomologyRing.set_workspace(tmp_dir()) sage: H = CohomologyRing(8,3,from_scratch=True,options='debug') We compute this cohomology ring from scratch Group data are rooted at '/home/king/.sage/temp/klap/26693/dir_f6_f2ei3/' Computing basic setup for Small Group number 1 of order 2 Computing basic setup for Small Group number 2 of order 4 Computing basic setup for Small Group number 3 of order 8 --------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-3-d1c5c11acd7a> in <module>() ----> 1 H = CohomologyRing(Integer(8),Integer(3),from_scratch=True,options='debug') /home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/factory.py in __call__(self, *args, **kwds) 1707 if q < 128: 1708 extras['websource'] = False -> 1709 OUT = self._check_compatibility(CacheKey, self._get_p_group_from_cache_or_db(GStem, KEY, **extras) or self._get_p_group_from_scratch(KEY, q, GStem, GroupName, **extras)) 1710 return OUT 1711 /home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/factory.py in _get_p_group_from_scratch(self, KEY, q, GStem, GroupName, **kwds) 1246 OUT = COHO(gap(KEY[0]), **extras) 1247 else: -> 1248 OUT = COHO(KEY[0],KEY[1], **extras) 1249 _gap_reset_random_seed() 1250 try: pGroupCohomology/cohomology.pyx in pGroupCohomology.cohomology.COHO.__init__() pGroupCohomology/resolution.pyx in pGroupCohomology.resolution.RESL.__init__() TypeError: expected bytes, str found
and
sage: H = CohomologyRing(8,3,options='debug')
Local data found at /home/king/Sage/git/py3/local/share/pGroupCohomology/8gp3/H8gp3.sobj
Creating symbolic links from /home/king/.sage/temp/klap/26693/dir_f6_f2ei3/8gp3 to /home/king/Sage/git/py3/local/share/pGroupCohomology/8gp3
Group data are rooted at '/home/king/.sage/temp/klap/26693/dir_f6_f2ei3/'
The state descriptor of the to-be-unpickled ring is expected to be provided at '/home/king/.sage/temp/klap/26693/dir_f6_f2ei3/8gp3/dat/State.sobj'
---------------------------------------------------------------------------
UnicodeDecodeError Traceback (most recent call last)
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/factory.py in _get_p_group_from_cache_or_db(self, GStem, KEY, **kwds)
1154 coho_options['@use_this_root@'] = root_workspace
-> 1155 OUT = load(os.path.join(root_workspace,file_name)) # realpath here?
1156 if '@use_this_root@' in coho_options:
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.load (build/cythonized/sage/misc/persist.c:2824)()
150 ## Load file by absolute filename
--> 151 with open(filename, 'rb') as fobj:
152 X = loads(fobj.read(), compress=compress)
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.load (build/cythonized/sage/misc/persist.c:2774)()
151 with open(filename, 'rb') as fobj:
--> 152 X = loads(fobj.read(), compress=compress)
153 try:
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.loads (build/cythonized/sage/misc/persist.c:7270)()
968 unpickler = SageUnpickler(io.BytesIO(s))
--> 969 return unpickler.load()
970
pGroupCohomology/cohomology.pyx in pGroupCohomology.cohomology.COHO_unpickle()
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.load (build/cythonized/sage/misc/persist.c:2824)()
150 ## Load file by absolute filename
--> 151 with open(filename, 'rb') as fobj:
152 X = loads(fobj.read(), compress=compress)
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.load (build/cythonized/sage/misc/persist.c:2774)()
151 with open(filename, 'rb') as fobj:
--> 152 X = loads(fobj.read(), compress=compress)
153 try:
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.loads (build/cythonized/sage/misc/persist.c:7270)()
968 unpickler = SageUnpickler(io.BytesIO(s))
--> 969 return unpickler.load()
970
UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 0: ordinal not in range(128)
During handling of the above exception, another exception occurred:
OSError Traceback (most recent call last)
<ipython-input-4-927128be23f4> in <module>()
----> 1 H = CohomologyRing(Integer(8),Integer(3),options='debug')
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/factory.py in __call__(self, *args, **kwds)
1707 if q < 128:
1708 extras['websource'] = False
-> 1709 OUT = self._check_compatibility(CacheKey, self._get_p_group_from_cache_or_db(GStem, KEY, **extras) or self._get_p_group_from_scratch(KEY, q, GStem, GroupName, **extras))
1710 return OUT
1711
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/factory.py in _get_p_group_from_cache_or_db(self, GStem, KEY, **kwds)
1159 if '@use_this_root@' in coho_options:
1160 del coho_options['@use_this_root@']
-> 1161 raise IOError("Saved data at %s are not readable: %s"%(os.path.join(root_local_sources,file_name), msg))
1162 ## 4. Search web repository
1163 elif kwds.get('websource')!=False and (not from_scratch):
OSError: Saved data at /home/king/Sage/git/py3/local/share/pGroupCohomology/8gp3/H8gp3.sobj are not readable: 'ascii' codec can't decode byte 0x80 in position 0: ordinal not in range(128)
So, that's something I can now try to debug.
comment:15 follow-up: ↓ 16 Changed 3 years ago by
in case you need them, sage has its own compatibility functions "bytes_to_str" and "str_to_bytes", to be found in sage.cpython.string and working in both py2 and py3
comment:16 in reply to: ↑ 15 Changed 3 years ago by
Replying to chapoton:
in case you need them, sage has its own compatibility functions "bytes_to_str" and "str_to_bytes", to be found in sage.cpython.string and working in both py2 and py3
Thank you! I don't know yet whether or not I'll need them, but certainly it is good to know.
comment:17 Changed 3 years ago by
I found that this fails in RESL.__init__:
tmprstem = os.path.join(res_folder,rstem)
tmpgstem = os.path.join(gps_folder,gstem)
self.Data = newResolWithGroupLoaded(tmprstem,tmpgstem,1)
Apparently tmprstem or tmpgstem are strings, but the C function newResolWithGroupLoaded expects char * --- is there no automatic conversion from py3-string to char*?
comment:18 follow-up: ↓ 19 Changed 3 years ago by
Maybe try
self.Data = newResolWithGroupLoaded(str_to_bytes(tmprstem),str_to_bytes(tmpgstem),1)
..with no guarantee..
comment:19 in reply to: ↑ 18 Changed 3 years ago by
Replying to chapoton:
Maybe try
self.Data = newResolWithGroupLoaded(str_to_bytes(tmprstem),str_to_bytes(tmpgstem),1)..with no guarantee..
That is likely to work. Another potential solution would be to generally work with bytes. I.e., let self.gstem be a bytes and then do os.path.join(self.res_folder, b'A'+self.gstem+b'.sobj')) etc.
What would be the preferred way?
comment:20 follow-up: ↓ 21 Changed 3 years ago by
beware that you cannot concatenate bytes and str, so everything must become bytes. Not the way to go in my opinion.. moreover, not sure that os.* can handle bytes
EDIT:
indeed:
sage: os.path.join('home', b'jojo')
...
TypeError: Can't mix strings and bytes in path components
comment:21 in reply to: ↑ 20 Changed 3 years ago by
Replying to chapoton:
beware that you cannot concatenate bytes and str, so everything must become bytes. Not the way to go in my opinion.. moreover, not sure that os.* can handle bytes
EDIT:
indeed:
sage: os.path.join('home', b'jojo') ... TypeError: Can't mix strings and bytes in path components
This only shows that it cannot mix both types. In fact, if both are bytes then things work:
sage: os.path.join(b'home', b'foo') b'home/foo'
Probably it would be a long and tedious work to change everything to bytes. In principle I would be willing to do that work, but if there is a different solution involving less work and only a slight overhead, I'd prefer that.
Currently, using str_to_bytes when passing a string to a C function does seem to work.
comment:22 Changed 3 years ago by
WTF? I just see that I have the following argument types of COCH.__init__:
def __init__(self, PARENT, int n, char *Nick, L, ydeg=None, rdeg=None, is_polyrep=False)
I suppose Nick shouldn't be char* but (if anything) str. Or better not type it at all.
comment:23 Changed 3 years ago by
I think I should also fix the following compiler warning (there are several similar):
pGroupCohomology/cohomology.c: In function ‘__pyx_pf_16pGroupCohomology_10cohomology_12explore_one_parameter’:
pGroupCohomology/cohomology.c:17678:35: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
for (__pyx_t_7 = 0; __pyx_t_7 < __pyx_t_16; __pyx_t_7+=1) {
comment:24 Changed 3 years ago by
I guesss getting the following error is a progress, although it seems a bit strange to me:
sage: H = CohomologyRing(8,3,from_scratch=True,options='debug')
We compute this cohomology ring from scratch
Group data are rooted at '/home/king/.sage/temp/klap/31253/dir_88iattmw/'
Computing basic setup for Small Group number 1 of order 2
Computing basic setup for Small Group number 2 of order 4
Computing basic setup for Small Group number 3 of order 8
> export action matrices
Initialising maximal p-elementary abelian subgroups
Inserting SmallGroup(2,1) as a subgroup
We compute this cohomology ring from scratch
Group data are rooted at '/home/king/.sage/temp/klap/31253/dir_88iattmw/'
> export action matrices
Unable to save basic ring setup
Traceback (most recent call last):
File "sage/misc/persist.pyx", line 240, in sage.misc.persist.save (build/cythonized/sage/misc/persist.c:3627)
obj.save(filename=filename, compress=compress, **kwargs)
AttributeError: 'tuple' object has no attribute 'save'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "sage/misc/persist.pyx", line 240, in sage.misc.persist.save (build/cythonized/sage/misc/persist.c:3636)
obj.save(filename=filename, compress=compress, **kwargs)
File "sage/structure/sage_object.pyx", line 424, in sage.structure.sage_object.SageObject.save (build/cythonized/sage/structure/sage_object.c:3328)
filename = _base_save(self, filename, compress=compress)
File "sage/misc/persist.pyx", line 179, in sage.misc.persist._base_save (build/cythonized/sage/misc/persist.c:3318)
with open(filename, 'wb') as fobj:
File "sage/misc/persist.pyx", line 180, in sage.misc.persist._base_save (build/cythonized/sage/misc/persist.c:3253)
fobj.write(_base_dumps(obj, compress=compress))
File "sage/misc/persist.pyx", line 257, in sage.misc.persist._base_dumps (build/cythonized/sage/misc/persist.c:3898)
gherkin = SagePickler.dumps(obj)
File "sage/misc/persist.pyx", line 836, in sage.misc.persist.SagePickler.dumps (build/cythonized/sage/misc/persist.c:6613)
pickler.dump(obj)
File "pGroupCohomology/cohomology.pyx", line 3464, in pGroupCohomology.cohomology.COHO.__reduce__
File "/home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/auxiliaries.py", line 85, in safe_save
save (obj, path)
File "sage/misc/persist.pyx", line 242, in sage.misc.persist.save (build/cythonized/sage/misc/persist.c:3694)
_base_save(obj, filename, compress=compress)
File "sage/misc/persist.pyx", line 179, in sage.misc.persist._base_save (build/cythonized/sage/misc/persist.c:3318)
with open(filename, 'wb') as fobj:
File "sage/misc/persist.pyx", line 180, in sage.misc.persist._base_save (build/cythonized/sage/misc/persist.c:3253)
fobj.write(_base_dumps(obj, compress=compress))
File "sage/misc/persist.pyx", line 257, in sage.misc.persist._base_dumps (build/cythonized/sage/misc/persist.c:3898)
gherkin = SagePickler.dumps(obj)
File "sage/misc/persist.pyx", line 836, in sage.misc.persist.SagePickler.dumps (build/cythonized/sage/misc/persist.c:6613)
pickler.dump(obj)
TypeError: can't pickle dict_items objects
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "sage/misc/persist.pyx", line 240, in sage.misc.persist.save (build/cythonized/sage/misc/persist.c:3627)
obj.save(filename=filename, compress=compress, **kwargs)
AttributeError: 'tuple' object has no attribute 'save'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/factory.py", line 1254, in _get_p_group_from_scratch
safe_save(OUT, OUT.autosave_name())
File "/home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/auxiliaries.py", line 85, in safe_save
save (obj, path)
File "sage/misc/persist.pyx", line 242, in sage.misc.persist.save (build/cythonized/sage/misc/persist.c:3694)
_base_save(obj, filename, compress=compress)
File "sage/misc/persist.pyx", line 179, in sage.misc.persist._base_save (build/cythonized/sage/misc/persist.c:3318)
with open(filename, 'wb') as fobj:
File "sage/misc/persist.pyx", line 180, in sage.misc.persist._base_save (build/cythonized/sage/misc/persist.c:3253)
fobj.write(_base_dumps(obj, compress=compress))
File "sage/misc/persist.pyx", line 257, in sage.misc.persist._base_dumps (build/cythonized/sage/misc/persist.c:3898)
gherkin = SagePickler.dumps(obj)
File "sage/misc/persist.pyx", line 836, in sage.misc.persist.SagePickler.dumps (build/cythonized/sage/misc/persist.c:6613)
pickler.dump(obj)
File "pGroupCohomology/cohomology.pyx", line 3464, in pGroupCohomology.cohomology.COHO.__reduce__
File "/home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/auxiliaries.py", line 85, in safe_save
save (obj, path)
File "sage/misc/persist.pyx", line 242, in sage.misc.persist.save (build/cythonized/sage/misc/persist.c:3694)
_base_save(obj, filename, compress=compress)
File "sage/misc/persist.pyx", line 179, in sage.misc.persist._base_save (build/cythonized/sage/misc/persist.c:3318)
with open(filename, 'wb') as fobj:
File "sage/misc/persist.pyx", line 180, in sage.misc.persist._base_save (build/cythonized/sage/misc/persist.c:3253)
fobj.write(_base_dumps(obj, compress=compress))
File "sage/misc/persist.pyx", line 257, in sage.misc.persist._base_dumps (build/cythonized/sage/misc/persist.c:3898)
gherkin = SagePickler.dumps(obj)
File "sage/misc/persist.pyx", line 836, in sage.misc.persist.SagePickler.dumps (build/cythonized/sage/misc/persist.c:6613)
pickler.dump(obj)
TypeError: can't pickle dict_items objects
Computing next term
> rk P_02 = 1
We have to choose 1 new generator in degree 1
> There is 1 Duflot regular generator in degree 1
Summary: 0 relations and 1 generators in degree 1
Ring approximation computed out to degree 1!
Storing approximation data
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.save (build/cythonized/sage/misc/persist.c:3627)()
239 try:
--> 240 obj.save(filename=filename, compress=compress, **kwargs)
241 except (AttributeError, RuntimeError, TypeError):
AttributeError: 'pGroupCohomology.resolution.RESL' object has no attribute 'save'
During handling of the above exception, another exception occurred:
AttributeError Traceback (most recent call last)
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.save (build/cythonized/sage/misc/persist.c:3627)()
239 try:
--> 240 obj.save(filename=filename, compress=compress, **kwargs)
241 except (AttributeError, RuntimeError, TypeError):
AttributeError: 'dict_items' object has no attribute 'save'
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call last)
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.save (build/cythonized/sage/misc/persist.c:3636)()
239 try:
--> 240 obj.save(filename=filename, compress=compress, **kwargs)
241 except (AttributeError, RuntimeError, TypeError):
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/structure/sage_object.pyx in sage.structure.sage_object.SageObject.save (build/cythonized/sage/structure/sage_object.c:3328)()
423
--> 424 filename = _base_save(self, filename, compress=compress)
425
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist._base_save (build/cythonized/sage/misc/persist.c:3318)()
178
--> 179 with open(filename, 'wb') as fobj:
180 fobj.write(_base_dumps(obj, compress=compress))
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist._base_save (build/cythonized/sage/misc/persist.c:3253)()
179 with open(filename, 'wb') as fobj:
--> 180 fobj.write(_base_dumps(obj, compress=compress))
181
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist._base_dumps (build/cythonized/sage/misc/persist.c:3898)()
256
--> 257 gherkin = SagePickler.dumps(obj)
258
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.SagePickler.dumps (build/cythonized/sage/misc/persist.c:6613)()
835 pickler = cls(buf, **kwargs)
--> 836 pickler.dump(obj)
837 return buf.getvalue()
pGroupCohomology/cohomology.pyx in pGroupCohomology.cohomology.COHO.__reduce__()
pGroupCohomology/cohomology.pyx in pGroupCohomology.cohomology.COHO.__getstate__()
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/auxiliaries.py in safe_save(obj, path)
84 os.unlink(path)
---> 85 save (obj, path)
86
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.save (build/cythonized/sage/misc/persist.c:3694)()
241 except (AttributeError, RuntimeError, TypeError):
--> 242 _base_save(obj, filename, compress=compress)
243 else:
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist._base_save (build/cythonized/sage/misc/persist.c:3318)()
178
--> 179 with open(filename, 'wb') as fobj:
180 fobj.write(_base_dumps(obj, compress=compress))
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist._base_save (build/cythonized/sage/misc/persist.c:3253)()
179 with open(filename, 'wb') as fobj:
--> 180 fobj.write(_base_dumps(obj, compress=compress))
181
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist._base_dumps (build/cythonized/sage/misc/persist.c:3898)()
256
--> 257 gherkin = SagePickler.dumps(obj)
258
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.SagePickler.dumps (build/cythonized/sage/misc/persist.c:6613)()
835 pickler = cls(buf, **kwargs)
--> 836 pickler.dump(obj)
837 return buf.getvalue()
pGroupCohomology/resolution.pyx in pGroupCohomology.resolution.RESL.__reduce__()
pGroupCohomology/resolution.pyx in pGroupCohomology.resolution.RESL.exportLifts()
pGroupCohomology/resolution.pyx in pGroupCohomology.resolution.LIFTcontainer.export()
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/auxiliaries.py in safe_save(obj, path)
84 os.unlink(path)
---> 85 save (obj, path)
86
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.save (build/cythonized/sage/misc/persist.c:3694)()
241 except (AttributeError, RuntimeError, TypeError):
--> 242 _base_save(obj, filename, compress=compress)
243 else:
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist._base_save (build/cythonized/sage/misc/persist.c:3318)()
178
--> 179 with open(filename, 'wb') as fobj:
180 fobj.write(_base_dumps(obj, compress=compress))
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist._base_save (build/cythonized/sage/misc/persist.c:3253)()
179 with open(filename, 'wb') as fobj:
--> 180 fobj.write(_base_dumps(obj, compress=compress))
181
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist._base_dumps (build/cythonized/sage/misc/persist.c:3898)()
256
--> 257 gherkin = SagePickler.dumps(obj)
258
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.SagePickler.dumps (build/cythonized/sage/misc/persist.c:6613)()
835 pickler = cls(buf, **kwargs)
--> 836 pickler.dump(obj)
837 return buf.getvalue()
TypeError: can't pickle dict_items objects
During handling of the above exception, another exception occurred:
AttributeError Traceback (most recent call last)
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.save (build/cythonized/sage/misc/persist.c:3627)()
239 try:
--> 240 obj.save(filename=filename, compress=compress, **kwargs)
241 except (AttributeError, RuntimeError, TypeError):
AttributeError: 'pGroupCohomology.resolution.RESL' object has no attribute 'save'
During handling of the above exception, another exception occurred:
AttributeError Traceback (most recent call last)
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.save (build/cythonized/sage/misc/persist.c:3627)()
239 try:
--> 240 obj.save(filename=filename, compress=compress, **kwargs)
241 except (AttributeError, RuntimeError, TypeError):
AttributeError: 'dict_items' object has no attribute 'save'
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call last)
<ipython-input-3-d1c5c11acd7a> in <module>()
----> 1 H = CohomologyRing(Integer(8),Integer(3),from_scratch=True,options='debug')
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/factory.py in __call__(self, *args, **kwds)
1707 if q < 128:
1708 extras['websource'] = False
-> 1709 OUT = self._check_compatibility(CacheKey, self._get_p_group_from_cache_or_db(GStem, KEY, **extras) or self._get_p_group_from_scratch(KEY, q, GStem, GroupName, **extras))
1710 return OUT
1711
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/factory.py in _get_p_group_from_scratch(self, KEY, q, GStem, GroupName, **kwds)
1246 OUT = COHO(gap(KEY[0]), **extras)
1247 else:
-> 1248 OUT = COHO(KEY[0],KEY[1], **extras)
1249 _gap_reset_random_seed()
1250 try:
pGroupCohomology/cohomology.pyx in pGroupCohomology.cohomology.COHO.__init__()
pGroupCohomology/cohomology.pyx in pGroupCohomology.cohomology.COHO.InitSubgroups()
pGroupCohomology/cohomology.pyx in pGroupCohomology.cohomology.COHO.InsertSubgroup()
pGroupCohomology/cohomology.pyx in pGroupCohomology.cohomology.COHO.make()
pGroupCohomology/cohomology.pyx in pGroupCohomology.cohomology.COHO.next()
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/auxiliaries.py in safe_save(obj, path)
83 if os.path.islink(path):
84 os.unlink(path)
---> 85 save (obj, path)
86
87 ###################################
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.save (build/cythonized/sage/misc/persist.c:3694)()
240 obj.save(filename=filename, compress=compress, **kwargs)
241 except (AttributeError, RuntimeError, TypeError):
--> 242 _base_save(obj, filename, compress=compress)
243 else:
244 # Saving an object to an image file.
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist._base_save (build/cythonized/sage/misc/persist.c:3318)()
177 filename = _normalize_filename(filename)
178
--> 179 with open(filename, 'wb') as fobj:
180 fobj.write(_base_dumps(obj, compress=compress))
181
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist._base_save (build/cythonized/sage/misc/persist.c:3253)()
178
179 with open(filename, 'wb') as fobj:
--> 180 fobj.write(_base_dumps(obj, compress=compress))
181
182 return filename
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist._base_dumps (build/cythonized/sage/misc/persist.c:3898)()
255 """
256
--> 257 gherkin = SagePickler.dumps(obj)
258
259 if compress:
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.SagePickler.dumps (build/cythonized/sage/misc/persist.c:6613)()
834 buf = io.BytesIO()
835 pickler = cls(buf, **kwargs)
--> 836 pickler.dump(obj)
837 return buf.getvalue()
838
pGroupCohomology/cohomology.pyx in pGroupCohomology.cohomology.COHO.__reduce__()
pGroupCohomology/cohomology.pyx in pGroupCohomology.cohomology.COHO.__getstate__()
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/auxiliaries.py in safe_save(obj, path)
83 if os.path.islink(path):
84 os.unlink(path)
---> 85 save (obj, path)
86
87 ###################################
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.save (build/cythonized/sage/misc/persist.c:3694)()
240 obj.save(filename=filename, compress=compress, **kwargs)
241 except (AttributeError, RuntimeError, TypeError):
--> 242 _base_save(obj, filename, compress=compress)
243 else:
244 # Saving an object to an image file.
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist._base_save (build/cythonized/sage/misc/persist.c:3318)()
177 filename = _normalize_filename(filename)
178
--> 179 with open(filename, 'wb') as fobj:
180 fobj.write(_base_dumps(obj, compress=compress))
181
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist._base_save (build/cythonized/sage/misc/persist.c:3253)()
178
179 with open(filename, 'wb') as fobj:
--> 180 fobj.write(_base_dumps(obj, compress=compress))
181
182 return filename
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist._base_dumps (build/cythonized/sage/misc/persist.c:3898)()
255 """
256
--> 257 gherkin = SagePickler.dumps(obj)
258
259 if compress:
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.SagePickler.dumps (build/cythonized/sage/misc/persist.c:6613)()
834 buf = io.BytesIO()
835 pickler = cls(buf, **kwargs)
--> 836 pickler.dump(obj)
837 return buf.getvalue()
838
pGroupCohomology/resolution.pyx in pGroupCohomology.resolution.RESL.__reduce__()
pGroupCohomology/resolution.pyx in pGroupCohomology.resolution.RESL.exportLifts()
pGroupCohomology/resolution.pyx in pGroupCohomology.resolution.LIFTcontainer.export()
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/auxiliaries.py in safe_save(obj, path)
83 if os.path.islink(path):
84 os.unlink(path)
---> 85 save (obj, path)
86
87 ###################################
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.save (build/cythonized/sage/misc/persist.c:3694)()
240 obj.save(filename=filename, compress=compress, **kwargs)
241 except (AttributeError, RuntimeError, TypeError):
--> 242 _base_save(obj, filename, compress=compress)
243 else:
244 # Saving an object to an image file.
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist._base_save (build/cythonized/sage/misc/persist.c:3318)()
177 filename = _normalize_filename(filename)
178
--> 179 with open(filename, 'wb') as fobj:
180 fobj.write(_base_dumps(obj, compress=compress))
181
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist._base_save (build/cythonized/sage/misc/persist.c:3253)()
178
179 with open(filename, 'wb') as fobj:
--> 180 fobj.write(_base_dumps(obj, compress=compress))
181
182 return filename
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist._base_dumps (build/cythonized/sage/misc/persist.c:3898)()
255 """
256
--> 257 gherkin = SagePickler.dumps(obj)
258
259 if compress:
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.SagePickler.dumps (build/cythonized/sage/misc/persist.c:6613)()
834 buf = io.BytesIO()
835 pickler = cls(buf, **kwargs)
--> 836 pickler.dump(obj)
837 return buf.getvalue()
838
TypeError: can't pickle dict_items objects
What are dict_items, and why can't they be pickled?
comment:25 follow-up: ↓ 26 Changed 3 years ago by
If d is a dict, then d.items() is no longer a list, but a view (some special kind of iterable).
You may need to wrap with list(d.items()) for pickling.
comment:26 in reply to: ↑ 25 ; follow-ups: ↓ 28 ↓ 32 Changed 3 years ago by
Replying to chapoton:
If d is a dict, then d.items() is no longer a list, but a view (some special kind of iterable).
You may need to wrap with list(d.items()) for pickling.
Wouldn't that considerably slow things down, at least in Python-2? After all, d.items() is a list, and list(...) applied to a list creates a (shallow) copy of that list.
comment:27 follow-up: ↓ 29 Changed 3 years ago by
There is a considerable progress now (after putting list around .items()):
sage: from pGroupCohomology import CohomologyRing sage: CohomologyRing.set_workspace(tmp_dir()) sage: H = CohomologyRing(8,3,from_scratch=True,options='debug') We compute this cohomology ring from scratch Group data are rooted at '/home/king/.sage/temp/klap/2449/dir_dxt9xtpq/' Computing basic setup for Small Group number 1 of order 2 Computing basic setup for Small Group number 2 of order 4 Computing basic setup for Small Group number 3 of order 8 > export action matrices Initialising maximal p-elementary abelian subgroups Inserting SmallGroup(2,1) as a subgroup We compute this cohomology ring from scratch Group data are rooted at '/home/king/.sage/temp/klap/2449/dir_dxt9xtpq/' > export action matrices Computing next term > rk P_02 = 1 We have to choose 1 new generator in degree 1 > There is 1 Duflot regular generator in degree 1 Summary: 0 relations and 1 generators in degree 1 Ring approximation computed out to degree 1! Storing approximation data Determine degree 2 standard monomials We got 1 standard monomials Monomial 0 > Candidate: c_1_0^2 > Express monomial as a Cochain Compute c_1_0*c_1_0 Compose chain maps R_2 -> R_1 -> R_0 Lift with Urbild Groebner basis in degree 1 load Urbild Groebner basis Decomposable cochain found There is no new generator in degree 2 Summary: 0 relations and 0 generators in degree 2 Ring approximation computed out to degree 2! Storing approximation data Inserting SmallGroup(4,2) as a subgroup We compute this cohomology ring from scratch Group data are rooted at '/home/king/.sage/temp/klap/2449/dir_dxt9xtpq/' > export action matrices --------------------------------------------------------------------------- KeyError Traceback (most recent call last) <ipython-input-3-d1c5c11acd7a> in <module>() ----> 1 H = CohomologyRing(Integer(8),Integer(3),from_scratch=True,options='debug') /home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/factory.py in __call__(self, *args, **kwds) 1707 if q < 128: 1708 extras['websource'] = False -> 1709 OUT = self._check_compatibility(CacheKey, self._get_p_group_from_cache_or_db(GStem, KEY, **extras) or self._get_p_group_from_scratch(KEY, q, GStem, GroupName, **extras)) 1710 return OUT 1711 /home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/factory.py in _get_p_group_from_scratch(self, KEY, q, GStem, GroupName, **kwds) 1246 OUT = COHO(gap(KEY[0]), **extras) 1247 else: -> 1248 OUT = COHO(KEY[0],KEY[1], **extras) 1249 _gap_reset_random_seed() 1250 try: pGroupCohomology/cohomology.pyx in pGroupCohomology.cohomology.COHO.__init__() pGroupCohomology/cohomology.pyx in pGroupCohomology.cohomology.COHO.InitSubgroups() pGroupCohomology/cohomology.pyx in pGroupCohomology.cohomology.COHO.InsertSubgroup() /home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/factory.py in __call__(self, *args, **kwds) 1707 if q < 128: 1708 extras['websource'] = False -> 1709 OUT = self._check_compatibility(CacheKey, self._get_p_group_from_cache_or_db(GStem, KEY, **extras) or self._get_p_group_from_scratch(KEY, q, GStem, GroupName, **extras)) 1710 return OUT 1711 /home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/factory.py in _get_p_group_from_scratch(self, KEY, q, GStem, GroupName, **kwds) 1246 OUT = COHO(gap(KEY[0]), **extras) 1247 else: -> 1248 OUT = COHO(KEY[0],KEY[1], **extras) 1249 _gap_reset_random_seed() 1250 try: pGroupCohomology/cohomology.pyx in pGroupCohomology.cohomology.COHO.__init__() KeyError: 'autoliftElAb'
The good news is that for the first time cohomology rings were computed in py-3 (namely cohomology rings of elementary abelian groups).
comment:28 in reply to: ↑ 26 Changed 3 years ago by
Replying to SimonKing:
Replying to chapoton:
If d is a dict, then d.items() is no longer a list, but a view (some special kind of iterable).
You may need to wrap with list(d.items()) for pickling.
Wouldn't that considerably slow things down, at least in Python-2? After all,
d.items()is a list, andlist(...)applied to a list creates a (shallow) copy of that list.
You could use a try ... except block to deal with the Python 2 case quickly. Sage will soon (I hope) default to Python 3, so I don't think this is worth it, though.
comment:29 in reply to: ↑ 27 Changed 3 years ago by
Replying to SimonKing:
There is a considerable progress now (after putting
listaround.items()): ...KeyError: 'autoliftElAb'
Apparently I forgot to create a shallow copy in some places in which I temporarily stored the global options -- it doesn't work to recreate a dict from its items in Py-3, and that's why I got the key error above:
sage: D = {1:2}
sage: I = D.items()
sage: D.clear()
sage: D.update(I)
sage: D
{}
In Py-2, one would get
sage: D
{1: 2}
comment:30 Changed 3 years ago by
Hooray!
I am now in a position to completely compute CohomologyRing(8,3) from scratch. Unfortunately reading old data doesn't work yet, but certainly it is a progress :-)
comment:31 in reply to: ↑ 12 ; follow-up: ↓ 33 Changed 3 years ago by
Replying to SimonKing:
import six
Please don't do this in Cython files. There is almost never a reason to use six in Cython since Cython handles most of the Python 2/3 incompatibilities for you.
comment:32 in reply to: ↑ 26 ; follow-up: ↓ 34 Changed 3 years ago by
comment:33 in reply to: ↑ 31 ; follow-up: ↓ 35 Changed 3 years ago by
Replying to jdemeyer:
Replying to SimonKing:
import sixPlease don't do this in Cython files. There is almost never a reason to use
sixin Cython since Cython handles most of the Python 2/3 incompatibilities for you.
I am not dealing with incompatibilities in that case. I import six in order to be able to safely test for string types. How to, if I don't use six?
comment:34 in reply to: ↑ 32 Changed 3 years ago by
Replying to jdemeyer:
Use
list(d.iteritems())in Cython.
Would this also work fast if Cython doesn't know that d is a dict? Should I do
list(<dict>(self._D).iteritems()), for instance?
comment:35 in reply to: ↑ 33 Changed 3 years ago by
Replying to SimonKing:
I import six in order to be able to safely test for string types. How to, if I don't use six?
First of all, you should check whether you really need to test for "string types". Can you use duck typing instead? If for some reason, duck typing is hard, then it depends what you mean precisely with "string types".
Since you were mentioning filenames: isinstance(s, (bytes, unicode)) are valid types for filenames both in Python 2 and Python 3. Note that Python 3 also has https://docs.python.org/3/library/pathlib.html, a custom class to deal specifically with filenames.
comment:36 in reply to: ↑ 4 Changed 3 years ago by
Replying to jhpalmieri:
I can get the package to build, but it doesn't work yet. An illustration of one problem:
sage: load('/Users/palmieri/.sage/pGroupCohomology/db/8gp3/H8gp3.sobj') ... UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 0: ordinal not in range(128)
I tried to track down that problem, and currently it seems to me that it is due to a very bad backwards incompatibility in Python. You can reproduce it as follows:
- Download attachment:State.sobj
- Install the optional meataxe spkg (this may involve
sage -b, so that some optional module in the Sage library is built) - Define some unpickle helper:
sage: from sage.matrix.matrix_gfpn_dense import mtx_unpickle sage: class unpickle_old_mtx: ....: def __call__(self, *args, **kwds): ....: return mtx_unpickle(*args, **kwds) ....: sage: register_unpickle_override('pGroupCohomology.mtx', 'MTX_unpickle_class', unpickle_old_mtx)
Now, in Sage-with-Py-2, loading the file works:
sage: X = load('/home/king/Projekte/coho/tests/State.sobj')
sage: X
([[1,
[(2, 1),
[
[1 0 0 0 0 0 0 0]
[0 0 0 1 1 1 1 1], [[1 0 0 0 0 0 0 0], '/home/king/SPKG/database/8gp3/sgp/8gp3sg1_1', '/home/king/SPKG/database/8gp3/sgp/8gp3sg1_2']
]]],
[2,
[(4, 2),
[
[1 0 0 0 0 0 0 0]
[0 0 0 1 1 1 1 1]
[0 1 0 0 0 0 0 0]
[0 0 0 0 0 1 0 1], [[1 0 0 0 0 0 0 0], '/home/king/SPKG/database/8gp3/sgp/8gp3sg2_1', '/home/king/SPKG/database/8gp3/sgp/8gp3sg2_2']
]]],
[3,
[(4, 2),
[
[1 0 0 0 0 0 0 0]
[0 0 0 1 1 1 1 1]
[0 0 1 0 0 0 0 0]
[0 0 0 0 0 0 1 1], [[1 0 0 0 0 0 0 0], '/home/king/SPKG/database/8gp3/sgp/8gp3sg3_1', '/home/king/SPKG/database/8gp3/sgp/8gp3sg3_2']
]]]],
[2, 1, 1],
1,
1,
'8gp3/',
['b_1_0*b_1_1'],
'8gp3/dat/',
[((4, 2), '/home/king/.sage/pGroupCohomology/db/4gp2/H4gp2'),
((2, 1), '/home/king/.sage/pGroupCohomology/db/2gp1/H2gp1')],
'8gp3/dat/',
[],
'8gp3/sgp/',
2,
[2, 3],
[2, 2],
2,
2,
['b_1_0*b_1_1'],
[[2, 'c_2_2', [0 0 1], 1, 0],
[1, 'b_1_0', [1 0], 0, 0],
[1, 'b_1_1', [0 1], 0, 0]],
[[0, [['1', ['1']]]],
[1, [['b_1_0', ['b_1_0']], ['b_1_1', ['b_1_1']]]],
[2,
[['c_2_2', ['c_2_2']],
['b_1_0', ['b_1_0^2']],
['b_1_1', ['b_1_0*b_1_1', 'b_1_1^2']]]]],
{},
0.24,
True,
'Monomials',
2,
None,
'',
0,
'8gp3',
'8gp3/dat/R8gp3.sobj',
1,
['0'],
['b_1_0 + b_1_1'],
None,
[('_max_module_deg', 0),
('sgpDickson', [((4, 2), [[1, 'c_1_1', [0 1]]])]),
('useFactorization', True),
('useElimination', False),
('GroupDescr', 'Dihedral group of order 8'),
('_key', ((8, 3), '/home/king/SPKG/database/8gp3/dat/State')),
('auto', 4),
('Restriction_2', ['c_1_0*c_1_1+c_1_0^2', 'c_1_1', '0']),
('GroupName', 'D8'),
('DicksonExp', 3),
('_parameters_do_exist', True),
('KeepBases', None),
('completeGroebner', True),
('_SymondsTestdata', ['c_2_2', 'b_1_1', 'b_1_0']),
('_parameters_for_criterion', ['c_2_2', 'b_1_1', 'b_1_0']),
('root', '/home/king/SPKG/database/'),
('Restriction_3', ['c_1_0*c_1_1+c_1_0^2', '0', 'c_1_1']),
('_method', 'Symonds')],
[(('dependent_parameters',), [['c_2_2', 'b_1_1', 'b_1_0'], 2])])
However, Sage-with-Py-3 yields the error found by John:
sage: X = load('/home/king/Projekte/coho/tests/State.sobj')
---------------------------------------------------------------------------
UnicodeDecodeError Traceback (most recent call last)
<ipython-input-4-74e7e91f9969> in <module>()
----> 1 X = load('/home/king/Projekte/coho/tests/State.sobj')
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.load (build/cythonized/sage/misc/persist.c:2824)()
149
150 ## Load file by absolute filename
--> 151 with open(filename, 'rb') as fobj:
152 X = loads(fobj.read(), compress=compress)
153 try:
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.load (build/cythonized/sage/misc/persist.c:2774)()
150 ## Load file by absolute filename
151 with open(filename, 'rb') as fobj:
--> 152 X = loads(fobj.read(), compress=compress)
153 try:
154 X._default_filename = os.path.abspath(filename)
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.loads (build/cythonized/sage/misc/persist.c:7270)()
967
968 unpickler = SageUnpickler(io.BytesIO(s))
--> 969 return unpickler.load()
970
971
UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 0: ordinal not in range(128)
And the point is: Python-3 even is unable to read the pickle file as a string! In Python-2, reading the file is no problem at all.
king@klap:~$ ~/Sage/git/py3/sage -python
Python 3.7.3 (default, Aug 27 2019, 23:22:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> f = open('/home/king/Projekte/coho/tests/State.sobj')
>>> f.read()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/king/Sage/git/py3/local/lib/python3.7/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9c in position 1: invalid start byte
>>>
king@klap:~$ ~/Sage/git/sage/sage --python
Python 2.7.15 (default, Jul 26 2019, 11:49:43)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> f = open('/home/king/Projekte/coho/tests/State.sobj')
>>> f.read()
'x\x9c\x95U\x8bZ\x1bE\x14&!\x91\x92J\x8b\x82V\xbc\x94x\xedXl\x96\x04\xaah\x95B\x0b\x05]\xc1\x16\x18\xad\x97\xe9:\xd9\x1dv\xb7d3{v7\nU,^\xd8B\xbd\xd6\xfb\xfd\xae\xcf\xd0\xa7\xf0E|\t\xcfl\xb2\x08\xf5\xab_I\xbe\x9d\xefd\xe6?\xe7\xfc\xe7?g6kY\xc2 \x83O\x96\xe8\x19\x06\xed\xc4\x0c\xb9-J\x81[\xb7\xc3\x92[\x8f\x84-\x82\x82\xc7\x97\x84\x91\xfe\x80\x1c\xcdT\xd6\xe7 \xef\xa0QF\xe3\xa6\x18:\x18\xec!\xc4\xf4\xa7\x02\xd9\xf0OJGz\xb2&\xed\x95\x92\x17-\x17f\x16\xce\x1a\x8d\xba\xef\x9aK5a\x985\x1e\x86\x05\xe8\x94PX\x85\xbd\xf4\x80a\x84|Q\x18\x8b2H1\x98\xd80\xe0\xe6\x8d\xb0J\xf4,~\xf7\xd0\xecZ\xffF4\x07]\x0c\xf6\x11\xa7\xa0v3\xb8\x9bY\xdb\xc4\xcd\xfdt@\xc3lB[B?m\xfe\xb4>\xa5Y<\xe2U\x1e\nm\xc4\xf6\x87\xb4\xd0\xf6\x13#\xb4\xcbF\x19\xbaw\x03\xaf\xc0-\x02?\x0cn\xc5\x9c\x0cz\x88*x\x18\x0b\xeeuZ\x12\xdc\x16\xc3\xed\x0c\x0e\xb4X\xe5\x90Un\xad\x7f,\xaf\xd8\xde\xc1\xa0\xef?l\xef\xbc\xf1\xf4\x15d{\xd7n\xe0\x15\xb8\xbb\xc9\xf6\x1e\xa2\xb738\xb8\xc5\xb6?e[\x8c\xe1^\x06\xf7\xedd[lWl\xefg\xf0\x00qvAo\x08\xe9=\xb8\x1bx\x05\x1e\x12M~\x87\x9a\x9adD:?$5\x1e\xa6\xf9\xc4\x8f\xc1a\xba\xb7j\x94\x8d\xc1\xc3j-\xc3\x00\xa7\x9d\xc9\t\xc6\xc7\xd3G\x88\xd3\xeb\xa0\xf6G\xa8\xb6\x8d@I\r\xaev\xed\x08jVU\x1b\xb6\xfd\x8a6\xad\xd6\x18JN\xde\xc1\x81\xd5n\xd0\xb5b\xfbemZ\xad1\x0c\x8a\xed,Z\xb6\xaaQMG\x99\xa42W\x941\x84\xc6\x10\x16;\xbc\xb5}45\x1e\x15\xceQ\xe5\xf1\xd8\xce\x1aG8\x83\xc7\xf1\x1e>\x81\xf2\xd0\xbci\xa8\x8e\x1eK\x07\xa8\x9df\x8aj\x80\x9eD\xbb\r\xc3>\x85\x97\x95\xe6\x13w\x18MA\xd9\xd6\x94\x1d\xd7\xdb\x12\xd0\xd8\x16\xa8\x0c\xe3\xdb@c\nt"\x01!\xea$\xe6\x9c z\x1b\x83I\x06\xa7\x08v\x82\xc1\x14\xae\\p<\x9dN\xde\nO#\xe6\x19\xe2\x8c2\xd0\xd3\xac\xcf\xaa\xd3\x19\xe2\x8c3\x98M\x93<\xc7\x93\x06\x9fN\xae\xcb\x19\xf4\x99#\xce1\x06\xf3i9\x0b\xca\x87\x924\xc4\xf3\x0c^\xa0\x1d\x89}\xae\x02g\xd5\xe1\x8b$\x8d\xf5\x12\x83\x97\xc9N\x85^i\x82\xcb\x08f\xc9,\xad\xc2\xb9\xa9\xe3\x7f]=\xf3\xf7\xfa\xc1\xab\x1b\xb4sF\xd6\xa5\xe7\xf2Z\x08F\xaa\xf5\xab\xb3\x14\xcb\xa49\xd5*\xe0\xb4g\xab}s\xca*\x85\xb2z^\xd5W\xa5\x99A\x94\xdf\xa4]I\xbe\xe2@\xb1\x99\xd1\xe2\xb3\x0c\x04\xa1\xfb\r\x8f/\x1b\x9e\xb4\x1a\xf8\n\xb3\x84\r\x8b*\xc1 &\xb0cph\x01\x87`\x02\xdf]\xa1\xac\x83\xcb\xe0\xbcs\x84\xc1\x12\x83Z\xa2\xbf\x99D\xf2\xae\xd5\xbf.x\x0c\x12\x1f\x9fv7Bq\x8a\x9b\x91\x0c\xdc\x0b<r1\x08l\xc4\x10\xd0}\xb8?Ys=\xb7\xde\xdc\r7c\x88h!\x99\xd2\t\x11\x9a\x014h\xdf\x84\xeb\x08+\xe0\xb5\xa2\xad\xb6\x8br\xb1(\x03K\x04\xc5\x11x-\x86\xd7i\xceX\x12+\xb0\xac\xe8\x8e \xdd\x95t6/\xc4\xf0\x06=\xf4\xff\xd7W\xe94\x1f\xf1H\xc4\xf0f\x0c\xab4\xc7\x1b\x91\x84\xb7\xf4\\\x0c\x17i\xd7\x9c\x08\xa3\xc05\x153l\xed\x1a\x83\xb7\t\xed1\x93v%%\x0f\x98\xad\xc6\xbe\x93j\xf0.J\x86\xb1\xde\xa3\x9dI\t\xb3\xdc\x13\xb0N\xb3\x13#\x10\xc7p\x89\x16Z\x12N.\xfb\xb0\xa1\xb7\xc7\xb0I{\r\x9f\x07\x08\x8bD\x10\x1a\x964\xc4\xb2\x1bFp\x19\xd5y\x9fv\xeaB\xf8\'\x90l\x08\x1f\xcc\xc6\xf0!\xed6\xa5\xe7\xd7\x10\x8b\xd1E\xb5.\x02\xf8\x08\x81\x1f\xd3nc~\xc5\x93u+\\@\xc6\xaaB\xf8\x84\xc1\x15\x9cLg\xdc\x19E>\x9f\xd2\xbe\xedi\xd4_\x91\x19\xb8h+\xcd?c\xf0\xf9\xbf\xd0/h.\x902\x82/i\xdfu\xa5\x83\xafb\xf8z\xa7<C\xf0\r\x83o\xaf#\xcfw\xa8J\xaa\xd0\xf7\x98\xe3\x07\xdaa \x15GZ\xf0#\xedhQ\x87\x9fb\xf8\x19\xef\xc7/\xb4\xd7\x12\xbe\xa8[\xa2\x1em#\r\xbf\xae\xc3o\x0c~\'\xce\x15=\x8b1\xfe\xe0\x11\xfcY\xfa\x07\x14\x1f\x95\xac'
comment:37 follow-up: ↓ 38 Changed 3 years ago by
This works fine in python3:
sage: with open('State.sobj', 'rb') as f:
....: f.read()
comment:38 in reply to: ↑ 37 Changed 3 years ago by
Replying to chapoton:
This works fine in python3:
sage: with open('State.sobj', 'rb') as f: ....: f.read()
Do I understand that it works because b means "binary"? But of course the real question is why Python 3 cannot unpickle from that file.
I tried to dig a bit deeper into Sage's load function (I guess the error is hidden there), and indeed load() uses with open(filename, 'rb') as fobj:. So, the unpickling problem is something else:
sage: from sage.matrix.matrix_gfpn_dense import mtx_unpickle
sage: filename = '/home/king/Projekte/coho/tests/State.sobj'
sage: compress = True
sage: class unpickle_old_mtx:
....: def __call__(self, *args, **kwds):
....: return mtx_unpickle(*args, **kwds)
....:
sage: register_unpickle_override('pGroupCohomology.mtx', 'MTX_unpickle_class', unpickle_old_mtx)
# The following is copied from Sage's load() function.
sage: ## Load file by absolute filename
....: with open(filename, 'rb') as fobj:
....: X = loads(fobj.read(), compress=compress)
....: try:
....: X._default_filename = os.path.abspath(filename)
....: except AttributeError:
....: pass
....:
---------------------------------------------------------------------------
UnicodeDecodeError Traceback (most recent call last)
<ipython-input-9-654643a51ef6> in <module>()
1 ## Load file by absolute filename
2 with open(filename, 'rb') as fobj:
----> 3 X = loads(fobj.read(), compress=compress)
4 try:
5 X._default_filename = os.path.abspath(filename)
/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.loads (build/cythonized/sage/misc/persist.c:7270)()
967
968 unpickler = SageUnpickler(io.BytesIO(s))
--> 969 return unpickler.load()
970
971
UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 0: ordinal not in range(128)
So, now we see that the error occurs in SageUnpickler(io.BytesIO(x)).load().
comment:39 follow-up: ↓ 40 Changed 3 years ago by
I find it very difficult to track down. But the fact that one does see a mentioning of pGroupCohomology.mtx when one forgets to register_unpickle_override(...) seems to indicate that the error occurs when an mtx matrix is unpickled.
So, I am now trying to find if this is really the case or not.
comment:40 in reply to: ↑ 39 ; follow-up: ↓ 41 Changed 3 years ago by
Replying to SimonKing:
I find it very difficult to track down. But the fact that one does see a mentioning of
pGroupCohomology.mtxwhen one forgets toregister_unpickle_override(...)seems to indicate that the error occurs when an mtx matrix is unpickled.So, I am now trying to find if this is really the case or not.
It isn't. I inserted a print statement at the beginning of mtx_unpickle, but it isn't shown before the error triggered by load('/home/king/Projekte/coho/tests/State.sobj') is raised.
I'll open a ticket for that problem.
comment:41 in reply to: ↑ 40 Changed 3 years ago by
comment:42 Changed 3 years ago by
- Dependencies set to #28444
comment:43 Changed 3 years ago by
It seems that #28444 fixes the problem for p_group_cohomology on Python-3. With the current unpublished code for the package, I get
$ ./sage ┌────────────────────────────────────────────────────────────────────┐ │ SageMath version 8.9.beta9, Release Date: 2019-09-02 │ │ Using Python 3.7.3. Type "help()" for help. │ └────────────────────────────────────────────────────────────────────┘ ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ Warning: this is a prerelease version, and it may be unstable. ┃ ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛ sage: from pGroupCohomology import CohomologyRing sage: CohomologyRing.set_workspace(tmp_dir()) sage: H = CohomologyRing(8,3,options='debug') Local data found at /home/king/Sage/git/py3/local/share/pGroupCohomology/8gp3/H8gp3.sobj Creating symbolic links from /home/king/.sage/temp/klap/7894/dir_05fi7c3s/8gp3 to /home/king/Sage/git/py3/local/share/pGroupCohomology/8gp3 Get original root and GStem /home/king/SPKG/database 8gp3 now, StateFile is /home/king/.sage/temp/klap/7894/dir_05fi7c3s/8gp3/dat/State.sobj Try to get (8, 3) from cache Group data are rooted at '/home/king/.sage/temp/klap/7894/dir_05fi7c3s/' The state descriptor of the to-be-unpickled ring is expected to be provided at '/home/king/.sage/temp/klap/7894/dir_05fi7c3s/8gp3/dat/State.sobj' Setting state > export action matrices Differential reloaded > rk P_02 = 3 Differential reloaded > rk P_03 = 4 Import monomials Checking compatibility of SmallGroups library and stored cohomology ring sage: print(H) Cohomology ring of Dihedral group of order 8 with coefficients in GF(2) Computation complete Minimal list of generators: [c_2_2: 2-Cocycle in H^*(D8; GF(2)), b_1_0: 1-Cocycle in H^*(D8; GF(2)), b_1_1: 1-Cocycle in H^*(D8; GF(2))] Minimal list of algebraic relations: [b_1_0*b_1_1]
Hooray...
comment:44 Changed 3 years ago by
Strangely, while I can compute some cohomology rings, the following computation fails in py3:
sage: from pGroupCohomology import CohomologyRing sage: CohomologyRing.doctest_setup() sage: H = CohomologyRing(32,4) sage: H.make()
Reason: Automatic saving does not work, and apparently it is because H.__getstate__() returns Singular elements, which cannot be pickled. It seems I did take care of it, as H.__getstate__() in py2 only returns the class sage.interfaces.singular.SingularElement and not the instance, but for some reasons it doesn't in py3.
comment:45 Changed 3 years ago by
Question: Has the mechanism of tab completion changed between py2 and py3? I notice that tab completion on a cohomology ring works in py2, but doesn't work (no error, but no completion either) in py3.
comment:46 Changed 3 years ago by
tab completion is done by ipython, which should be upgraded (they've switched to Python 3 only, so we're using an old version, which in turn might lag in its Python 3 support).
I suggest not to worry about it now.
comment:47 Changed 3 years ago by
Aha! The culprit for pickle problem is pGroupCohomology.cohomology.pickle_gap_data, which inspite of its name also takes care of singular elements. In Py2:
sage: self = CohomologyRing(32,4)
sage: self.make()
sage: G = self._decorator_cache['_parameter_restrictions',frozenset({'c_2_2'}),('radical', True)]
sage: G
[[(polynomial ring, over a field, global ordering
// coefficients: ZZ/2
// number of vars : 2
// block 1 : ordering M
// : names c_1_0 c_1_1
// : weights 1 1
// : weights -1 0
// block 2 : ordering C,
<repr(<sage.interfaces.singular.SingularElement at 0x7ff4a0cca690>) failed: AttributeError: 'NoneType' object has no attribute 'group'>,
<repr(<sage.interfaces.singular.SingularElement at 0x7ff4a0cca500>) failed: AttributeError: 'NoneType' object has no attribute 'group'>)],
2]
sage: pickle_gap_data(G)
[[(<class 'sage.interfaces.singular.SingularElement'>,
<class 'sage.interfaces.singular.SingularElement'>,
<class 'sage.interfaces.singular.SingularElement'>)],
2]
In Py3:
sage: pickle_gap_data(G) [[(polynomial ring, over a field, global ordering // coefficients: ZZ/2 // number of vars : 2 // block 1 : ordering M // : names c_1_0 c_1_1 // : weights 1 1 // : weights -1 0 // block 2 : ordering C, <repr(<sage.interfaces.singular.SingularElement at 0x7f6475b86a20>) failed: AttributeError: 'NoneType' object has no attribute 'group'>, <repr(<sage.interfaces.singular.SingularElement at 0x7f6475b86dc8>) failed: AttributeError: 'NoneType' object has no attribute 'group'>)], 2]
comment:48 Changed 3 years ago by
Ouch. A nasty thing.
In pickle_gap_data, I test if the input is an iterable class, and if it is and if the class is a builtin, then the pickle_gap_data traverses the items of the input and creates from the individual results a modified copy of the input. In code:
try: I = iter(G) except: return G if isinstance(G,dict): return dict((pickle_gap_data(k), pickle_gap_data(v)) for k,v in G.items()) if getattr(type(G), '__module__', None) == '__builtin__': return type(G)(pickle_gap_data(X) for X in I) return G
The problem is that in Py3 we get
sage: getattr(list, '__module__', None) 'builtins'
Question: How to test in a py2/py3 invariant way whether a type is a builtin?
That asked, are there more iterable builtins than list, tuple, str/bytes and dict? I guess it is safer to just explicitly test for list and tuple. EDIT Like this:
if isinstance(G, dict): return dict((pickle_gap_data(k), pickle_gap_data(v)) for k,v in G.items()) if isinstance(G, (list, tuple)): return type(G)(pickle_gap_data(X) for X in G) return G
comment:49 follow-up: ↓ 50 Changed 3 years ago by
An iterable is an object with an __iter__ method or/and __getitem__ method defined.
So perhaps you can test for their presense. (These methods may be broken, but OK...)
comment:50 in reply to: ↑ 49 Changed 3 years ago by
Replying to dimpase:
An iterable is an object with an
__iter__method or/and__getitem__method defined. So perhaps you can test for their presense. (These methods may be broken, but OK...)
You misunderstood. The problem was not how to test if something is iterable but how to test if that iterable is a builtin. But, as I said in my previous post, probably this isn't the actual problem anyway. What I want is to deal with strings, lists, tuples and dicts. Period. So, I could be more explicit (and I already tested that the error from comment:44 can be fixed in that way).
comment:51 Changed 3 years ago by
In Python-2, one implemented c.__nonzero__() and then bool(c) would work. It seems that in Python-3, something else needs to be done. Is it perhaps the case that I need to implement __bool__?
What should I do to make stuff work both in py-2 and py-3? Implement both __bool__ and __nonzero__? Or would it be enough to replace __nonzero__ with __bool__?
comment:52 Changed 3 years ago by
comment:53 follow-up: ↓ 54 Changed 3 years ago by
There is a problem with the logging module: In Python-2, I get
sage: from pGroupCohomology import CohomologyRing
sage: CohomologyRing.doctest_setup()
sage: P = CohomologyRing(9,2,options='debug')
We compute this cohomology ring from scratch
Group data are rooted at '/home/king/.sage/temp/klap/19448/dir_0MUJxK/'
Computing basic setup for Small Group number 1 of order 3
Computing basic setup for Small Group number 2 of order 9
Resolution of GF(3)[9gp2]:
> export action matrices
H^*(SmallGroup(9,2); GF(3)):
Initialising maximal p-elementary abelian subgroups
whereas in Python-3 I get
sage: from pGroupCohomology import CohomologyRing sage: CohomologyRing.doctest_setup() sage: P = CohomologyRing(9,2,options='debug') We compute this cohomology ring from scratch Group data are rooted at '/home/king/.sage/temp/klap/18649/dir_9bz1_9bh/' Computing basic setup for Small Group number 1 of order 3 Computing basic setup for Small Group number 2 of order 9 > export action matrices Initialising maximal p-elementary abelian subgroups
So, the indentation is gone, and (whats worse) the logging doesn't indicate what object the log comes from (which I found very useful for debugging, as in the computation of a single cohomology ring often many other rings are involved).
Can someone please give me a pointer to a page explaining the differences between py-2 and py-3 in the logging module?
comment:54 in reply to: ↑ 53 Changed 3 years ago by
Replying to SimonKing:
Can someone please give me a pointer to a page explaining the differences between py-2 and py-3 in the logging module?
By looking at the code of the logging module in Python-2 and Python-3, I see where the problem comes from. In the following, self denotes an instance of pGroupCohomology.auxiliary.CohoFormatter, which sub-classes loggin.Formatter.
In Python-2, logging.Formatter.format(self, record) uses self._fmt to format the output. In Python-3, self._fmt is not used. Instead, self._style._fmt is consulted, where self._style is one of the styles in logging._STYLES.
The point is: So far, my custom self.format modifies self._fmt and then calls the default .format(). However, in Python-3, when calling the default .format(), the attribute self._fmt plays no role, but this is where I currently put the format into.
Simple way out: Do not call logging.Formatter.format in CohoFormatter.format.
But I wonder: Is there a recommended way to make the logging format change from call to call?
By this, I mean: Make the output of CohoFormatter.format(self, record) depend on record.args[0], in the sense that the output makes visible whether record.args[0] is the same object as in the previous call to CohoFormatter.format.
Or, more radically: What kind of logging approach would you recommend, what do you find most easy to comprehend/parse as a human (I am not interested in automatic parsing of the logging messages)?
I somehow like the current approach, which is: Assume that certain objects (say, ZZ and RR) are involved in a computation, and each object may issue a logging message. Then, stream of messages is arranged in blocks of messages coming from a single object, each block being prepended by an identifier of the object. I.e., if ZZ gives two messages, then RR gives three messages and then again ZZ gives a message, the output would be
Integer Ring:
message 1
message 2
Rational Field:
message 3
message 4
message 5
Integer Ring:
message 6
- Do you like this thematic arrangement of messages (which is provided by
coho_loggerin Python 2, but fails in Python-3)? - If yes: Can you recommend a clean and easy way to achieve it both in Python 2 and Python 3? Ideally, one should still be able to include time information in the logging, which currently is not supported.
- If no: What logging format would you recommend?
comment:55 follow-up: ↓ 56 Changed 3 years ago by
Instead of setting self._fmt, can you instead run self.basicConfig(format=FORMAT)? (https://docs.python.org/3/library/logging.html#logging.basicConfig) There are also ways of having several formatters (https://docs.python.org/3.7/library/logging.config.html#user-defined-objects), so maybe you can make use of that somehow.
comment:56 in reply to: ↑ 55 Changed 3 years ago by
Replying to jhpalmieri:
Instead of setting
self._fmt, can you instead runself.basicConfig(format=FORMAT)? (https://docs.python.org/3/library/logging.html#logging.basicConfig) There are also ways of having several formatters (https://docs.python.org/3.7/library/logging.config.html#user-defined-objects), so maybe you can make use of that somehow.
From these links, I get the impression that each call to the logger results in the creation of a new formatter. Is that really the case?
comment:57 follow-up: ↓ 58 Changed 3 years ago by
I don't know, I only looked at the links, too. It's worth trying.
(My impression from the links was that you could create several formatters and then switch from one to another on the fly.)
comment:58 in reply to: ↑ 57 Changed 3 years ago by
Replying to jhpalmieri:
I don't know, I only looked at the links, too. It's worth trying.
(My impression from the links was that you could create several formatters and then switch from one to another on the fly.)
Sure. But I am unsure how a change of formatter could solve the problem without impact on performance.
If logging is in "warn" mode, then there are of course still silent calls to coho_logger.debug. I want that (1) these calls are cheap (i.e., they shouldn't involve creation of Python objects) and (2) the message blocks should not take into account silent messages, i.e., a block shouldn't be closed if a silent message comes from a different object.
Of course, it should be that what I do with the formatter should rather be done with a handler, or maybe a filter. So, I will try to understand what exactly what handlers and filters do.
And, as in point 3. of comment:54: Would you recommend a different formatting of log messages?
comment:59 Changed 3 years ago by
Currently I am reading logging HOWTO and logging Cookbook, and at the moment it seems to me that I want to use logging.LoggingAdapter, because it allows to add contextual information (which is what I'd like to use, unless people tell me a nicer way log format).
comment:60 Changed 3 years ago by
I think I now know what to do: The customisation should be done in a Formatter, i.e., the existing solution was a good approach. Only the format() method should be different: Before calling super(CohoFormatter, self).format(record), I should add some attribute to the record that is then used in the formatter's format string.
comment:61 Changed 3 years ago by
I think I solved the problem. I can now optionally add time information to the log, such as here:
sage: from pGroupCohomology import CohomologyRing
sage: CohomologyRing.doctest_setup()
sage: CohomologyRing.global_options('debug')
sage: CohomologyRing.global_options('time')
sage: H = CohomologyRing(720,763,prime=3)
2019-09-18 20:54:20,277: Try to compute a Sylow 3-subgroup
2019-09-18 20:54:20,283: Try to find the SmallGroups address of the Sylow subgroup
2019-09-18 20:54:20,288: We compute this cohomology ring from scratch
2019-09-18 20:54:20,289: Group data are rooted at '/home/king/.sage/temp/klap/962/dir_tnhvqeq7/'
2019-09-18 20:54:20,293: Computing basic setup for Small Group number 1 of order 3
2019-09-18 20:54:20,405: Computing basic setup for Small Group number 2 of order 9
Resolution of GF(3)[9gp2]:
2019-09-18 20:54:20,466: > export action matrices
H^*(SmallGroup(9,2); GF(3)):
2019-09-18 20:54:20,517: Initialising maximal p-elementary abelian subgroups
2019-09-18 20:54:20,555: Computing intermediate subgroup
2019-09-18 20:54:20,567: Try to find the SmallGroups address of the intermediate subgroup
2019-09-18 20:54:20,603: Computing group order
2019-09-18 20:54:20,604: The group is of order 72
2019-09-18 20:54:20,609: Computing intermediate subgroup
Resolution of GF(3)[SmallGroup(9,2)]:
2019-09-18 20:54:20,625: Computing next term
2019-09-18 20:54:20,644: > rk P_02 = 3
H^*(SmallGroup(9,2); GF(3)):
2019-09-18 20:54:20,647: We have to choose 2 new generators in degree 1
2019-09-18 20:54:20,648: > There are 2 nilpotent generators in degree 1
...
However, what I'd find even nicer if the sum of the cputimes of Sage and Singular since start of logging was shown, not just the walltime, as above.
Note that the indentation has changed, which means that I will have to change various doc tests, but that will be doable.
comment:62 Changed 3 years ago by
Sigh.
After solving the log problem, I find that meanwhile the code ONLY works in Python-3, not Python-2.
First problem:
class FOO(object):
def __bool__(self):
return something
__nonzero__ = __bool__
works in Py3, but not in Py2, although it was recommended in some stackoverflow discussion. In Py2 it results in NameError: name '__bool__' is not defined. How to fix it without code duplication?
Second problem:
/home/king/Sage/git/sage/local/lib/python2.7/site-packages/pGroupCohomology/factory.py in <module>()
47 import re,os
48
---> 49 import urllib.request, urllib.error
50 import tarfile
51 import logging
ImportError: No module named request
How to deal with urllib in a py2/py3-compatible way?
comment:63 follow-up: ↓ 64 Changed 3 years ago by
For the urllib issue: http://python3porting.com/noconv.html#import-errors suggests code like
try:
from urllib.request import ...
except ImportError:
from urllib import ...
I can't reproduce the first problem. With Python 2, if I do
sage: class FOO(object):
....: def __bool__(self):
....: return something
....: __nonzero__ = __bool__
....:
sage: a = FOO()
sage: a.__nonzero__()
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-3-74dab0bd3f82> in <module>()
----> 1 a.__nonzero__()
<ipython-input-1-ec93abb3d539> in __bool__(self)
1 class FOO(object):
2 def __bool__(self):
----> 3 return something
4 __nonzero__ = __bool__
5
NameError: global name 'something' is not defined
so a.__nonzero__() is certainly calling __bool__. If I change the return line to something sensible, it works.
comment:64 in reply to: ↑ 63 Changed 3 years ago by
Replying to jhpalmieri:
For the
urllibissue: http://python3porting.com/noconv.html#import-errors suggests code liketry: from urllib.request import ... except ImportError: from urllib import ...
OK, thank you.
I can't reproduce the first problem. With Python 2, if I do
The "something" was of course figuratively. Anyway, the problem seems to be that I am using a cdef class here:. In Sage-with-py2:
sage: cython('''
....: class Foo:
....: def __bool__(self):
....: return True
....: __nonzero__ = __bool__
....: ''')
sage: cython('''
....: cdef class Foo:
....: def __bool__(self):
....: return True
....: __nonzero__ = __bool__
....: ''')
ERROR:root:An unexpected error occurred while tokenizing input
The following traceback may be corrupted or invalid
The error message is: ('EOF in multi-line string', (1, 0))
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-2-91e48d391924> in <module>()
4 return True
5 __nonzero__ = __bool__
----> 6 ''')
/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/misc/lazy_import.pyx in sage.misc.lazy_import.LazyImport.__call__ (build/cythonized/sage/misc/lazy_import.c:3697)()
352 True
353 """
--> 354 return self.get_object()(*args, **kwds)
355
356 def __repr__(self):
/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/misc/cython.pyc in cython_compile(code, **kwds)
649 with open(tmpfile,'w') as f:
650 f.write(code)
--> 651 return cython_import_all(tmpfile, get_globals(), **kwds)
/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/misc/cython.pyc in cython_import_all(filename, globals, **kwds)
539 code
540 """
--> 541 m = cython_import(filename, **kwds)
542 for k, x in iteritems(m.__dict__):
543 if k[0] != '_':
/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/misc/cython.pyc in cython_import(filename, **kwds)
519 try:
520 sys.path.append(build_dir)
--> 521 return builtins.__import__(name)
522 finally:
523 sys.path = oldpath
/home/king/.sage/temp/klap/23274/spyx/_home_king__sage_temp_klap_23274_tmp_FgSymA_pyx/_home_king__sage_temp_klap_23274_tmp_FgSymA_pyx_0.pyx in init _home_king__sage_temp_klap_23274_tmp_FgSymA_pyx_0()
3 def __bool__(self):
4 return True
----> 5 __nonzero__ = __bool__
NameError: name '__bool__' is not defined
whereas in Sage-with-py3 I get
sage: cython('''
....: class Foo:
....: def __bool__(self):
....: return True
....: __nonzero__ = __bool__
....: ''')
sage: cython('''
....: cdef class Foo:
....: def __bool__(self):
....: return True
....: __nonzero__ = __bool__
....: ''')
So, what to do?
comment:65 follow-up: ↓ 66 Changed 3 years ago by
I don't know Cython, but replacing __nonzero__ = __bool__ with
def __nonzero__(self):
return self.__bool__()
seems to work.
comment:66 in reply to: ↑ 65 Changed 3 years ago by
Replying to jhpalmieri:
I don't know Cython, but replacing
__nonzero__ = __bool__withdef __nonzero__(self): return self.__bool__()seems to work.
Sure. that's what I basically meant with "code duplication". It is an additional call, thus, if the boolean value of a cochain is tested very often then one will see a slow-down. But I don't know if it actually is tested very often.
Anyway, with this code cuplication, things seem to work. Hooray!
At the moment, it seems that most of the remaining work will be to change the expected output for those tests that use logging.
comment:67 follow-up: ↓ 68 Changed 3 years ago by
My hope is that we can drop Python 2 support pretty soon, or at least make sure that Python 3 works well, even if it as the expense of performance with Python 2. Does __nonzero__ do anything with Python 3? If so, maybe you could do something like
from six import PY2
if PY2:
def __nonzero__(self):
...
to avoid the extra call when using Python 3.
comment:68 in reply to: ↑ 67 ; follow-up: ↓ 81 Changed 3 years ago by
Replying to jhpalmieri:
My hope is that we can drop Python 2 support pretty soon, or at least make sure that Python 3 works well, even if it as the expense of performance with Python 2. Does
__nonzero__do anything with Python 3? If so, maybe you could do something likefrom six import PY2 if PY2: def __nonzero__(self): ...to avoid the extra call when using Python 3.
I don't think that __nonzero__ does anything in Python 3. But now as you mention it: It is possible (and is actually used in some places in the cythonised parts of the Sage library and also in the current not-yet-published code of the cohomology package) to make the code depend on the language level. I.e., I could have definitions for __nonzero__ and for __bool__, but only one of them would be compiled.
comment:69 Changed 3 years ago by
One detail: According to unit_test_64, there is no significant speed difference between the py2 and py3 versions. In any case, the cputime obtained as sum of the cputime() and Singular's timer is only half of the walltime. Do I recall correctly that the time used to save data on disk is not measured by cputime()? Is there an easy way to determine that time, too?
comment:70 Changed 3 years ago by
I don't think it makes much sense measuring time to save to disk, as it varies wildly between different OS's, different hardware, different configuration of buffering, etc etc.
Often saving is completely asyncronous from the computation due to buffering and separate threads run to do buffer flushing, etc.
comment:71 Changed 3 years ago by
Now I am totally baffled. Is it not the case that comparison is supposed to be implemented by __richcmp__? Apparently __richcmp__ isn't called anymore.
The following corresponds to the code in pGroupCohomology.dickson, that did work in the past:
sage: cython("""
....: from sage.structure.richcmp import richcmp, op_LT, op_NE, op_GT
....: from sage.all import GF
....: class DICKSON:
....: def __init__(self,p):
....: self.K = GF(p)
....: def __richcmp__(self, other, op):
....: print("Comparison called")
....: return richcmp(self.K, other.K, op)
....: """)
Now, rich comparison fails both in py2 and py3:
sage: D = DICKSON(5) sage: F = loads(dumps(D)) sage: D == F False sage: F.K == D.K True
What has changed, how to fix it?
comment:72 follow-up: ↓ 73 Changed 3 years ago by
Aha! I see in sage.misc.unknown that a decorator @richcmp_method is used. So, I guess I need to use it, too.
comment:73 in reply to: ↑ 72 Changed 3 years ago by
Replying to SimonKing:
Aha! I see in sage.misc.unknown that a decorator
@richcmp_methodis used. So, I guess I need to use it, too.
Hooray:
sage: cython("""
....: from sage.structure.richcmp import richcmp, richcmp_method
....: from sage.all import GF
....: @richcmp_method
....: class DICKSON(object):
....: def __init__(self,p):
....: self.K = GF(p)
....: def __richcmp__(self, other, op):
....: print("Comparison called")
....: return richcmp(self.K, other.K, op)
....: """)
sage: D = DICKSON(5)
sage: F = loads(dumps(D))
sage: D == F
Comparison called
True
comment:74 follow-ups: ↓ 75 ↓ 76 Changed 3 years ago by
At the moment, tests in dickson, auxiliaries, resolution and factory pass both in Py2 and Py3. In isomorphism_test, there are currently two failing tests, and it seems to me that they reveal a Py3-bug in cartesian_product_iterator:
File "/home/king/Projekte/coho/coho-devel/src/pGroupCohomology/isomorphism_test.py", line 353, in pGroupCohomology.isomorphism_test.IsomorphismTest.explore_isomorphisms
Failed example:
T.explore_isomorphisms()
Exception raised:
Traceback (most recent call last):
File "sage/misc/cachefunc.pyx", line 1944, in sage.misc.cachefunc.CachedMethodCaller.__call__ (build/cythonized/sage/misc/cachefunc.c:10139)
return cache[k]
KeyError: (((3, 5, 4, 6, 7, 1, 2), 0, True), ())
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/mrange.py", line 139, in _xmrange_iter
curr_elt = [next(i) for i in curr_iters[:-1]] + [None]
File "/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/misc/mrange.py", line 139, in <listcomp>
curr_elt = [next(i) for i in curr_iters[:-1]] + [None]
StopIteration
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 681, in _run
self.compile_and_execute(example, compiler, test.globs)
File "/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1123, in compile_and_execute
exec(compiled, globs)
File "<doctest pGroupCohomology.isomorphism_test.IsomorphismTest.explore_isomorphisms[10]>", line 1, in <module>
T.explore_isomorphisms()
File "/home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/isomorphism_test.py", line 402, in explore_isomorphisms
Cands = self.candidates_of_gens(tuple([p[4] for p in Sizes]),len(self.rigid_generators))
File "sage/misc/cachefunc.pyx", line 1949, in sage.misc.cachefunc.CachedMethodCaller.__call__ (build/cythonized/sage/misc/cachefunc.c:10273)
w = self._instance_call(*args, **kwds)
File "sage/misc/cachefunc.pyx", line 1825, in sage.misc.cachefunc.CachedMethodCaller._instance_call (build/cythonized/sage/misc/cachefunc.c:9758)
return self.f(self._instance, *args, **kwds)
File "/home/king/Sage/git/py3/local/lib/python3.7/site-packages/pGroupCohomology/isomorphism_test.py", line 644, in candidates_of_gens
for Ims1,Ims2 in cartesian_product_iterator([FirstHalf, SecondHalf]):
RuntimeError: generator raised StopIteration
I will investigate which values of FirstHalf and SecondHalf trigger it.
comment:75 in reply to: ↑ 74 Changed 3 years ago by
comment:76 in reply to: ↑ 74 Changed 3 years ago by
Replying to SimonKing:
At the moment, tests in
dickson,auxiliaries,resolutionandfactorypass both in Py2 and Py3.
When working around the cartesian_product_iterator bug, the tests in isomorphism_test pass both in Py2 and Py3. Yay!
comment:77 Changed 3 years ago by
Two errors in Py3 concern __div__, namely
Running doctests with ID 2019-09-21-07-26-57-20b48344.
Using --optional=build,dochtml,gap_packages,libsemigroups,meataxe,memlimit,mpir,python2,sage
Doctesting 1 file.
sage -t --long --warn-long 47.4 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/cochain.pyx
**********************************************************************
File "/home/king/Projekte/coho/coho-devel/src/pGroupCohomology/cochain.pyx", line 3131, in pGroupCohomology.cochain.MODCOCH.__div__
Failed example:
print(H.1/3) #indirect doctest
Expected:
(a_6_0)/(3): 6-Cocycle in H^*(SmallGroup(400,206); GF(5))
defined by
2*c_2_1*c_2_2*a_1_0*a_1_1
Got:
(a_6_0)*(2): 6-Cocycle in H^*(SmallGroup(400,206); GF(5))
defined by
2*c_2_1*c_2_2*a_1_0*a_1_1
**********************************************************************
File "/home/king/Projekte/coho/coho-devel/src/pGroupCohomology/cochain.pyx", line 3139, in pGroupCohomology.cochain.MODCOCH.__div__
Failed example:
H.2/H.1
Expected:
Traceback (most recent call last):
...
TypeError: Can not divide by <class 'pGroupCohomology.cochain.MODCOCH'>
Got:
<BLANKLINE>
Traceback (most recent call last):
File "/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 681, in _run
self.compile_and_execute(example, compiler, test.globs)
File "/home/king/Sage/git/py3/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1123, in compile_and_execute
exec(compiled, globs)
File "<doctest pGroupCohomology.cochain.MODCOCH.__div__[6]>", line 1, in <module>
H.gen(2)/H.gen(1)
File "sage/structure/element.pyx", line 1718, in sage.structure.element.Element.__truediv__ (build/cythonized/sage/structure/element.c:12780)
return (<Element>left)._div_(right)
File "sage/structure/element.pyx", line 2654, in sage.structure.element.RingElement._div_ (build/cythonized/sage/structure/element.c:18184)
frac = self._parent.fraction_field()
TypeError: 'NoneType' object is not callable
**********************************************************************
1 item had failures:
2 of 8 in pGroupCohomology.cochain.MODCOCH.__div__
[1490 tests, 2 failures, 465.54 s]
----------------------------------------------------------------------
sage -t --long --warn-long 47.4 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/cochain.pyx # 2 doctests failed
----------------------------------------------------------------------
Total time for all tests: 470.6 seconds
cpu time: 249.3 seconds
cumulative wall time: 465.5 seconds
The first error seems to indicate that H.1/3 is automatically translated to H.1*2, which does make sense modulo 5, but which I did not implement like this. And the second error is nothing that I implemented.
In fact I overrode __div__ for COCH and MODCOCH. Is it perhaps the case that overriding __div__ won't work in Py3?
comment:78 follow-up: ↓ 79 Changed 3 years ago by
In py3, there is __truediv__ and __floordiv__
comment:79 in reply to: ↑ 78 ; follow-up: ↓ 80 Changed 3 years ago by
Replying to chapoton:
In py3, there is
__truediv__and__floordiv__
I see. Do I understand correctly that in order to make my code Py2 and Py3 compatible, I should implement __truediv__, __floordiv__ and __div__?
On the other hand, I reckon that Sage has default implementations of the double underscore methods, that I should rather not override.
So, given a RingElement, what methods do I need to implement for providing division by an element of the base field, and what do I need to implement for making clear that division of two ring elements doesn't work?
I guess I should implement a fraction_field method for cohomology rings. Of course, since cohomology rings can have zero divisors, a fraction field will not always exist. So, should in that situation fraction_field return NotImplemented, or raise a ValueError, or what else?
Do I need to provide a single-underscore _div_ method, in addition to the fraction_field?
How to implement division by a base field element (I guess _div_ is for division of two ring elements)?
comment:80 in reply to: ↑ 79 Changed 3 years ago by
Replying to SimonKing:
Replying to chapoton:
In py3, there is
__truediv__and__floordiv__I see. Do I understand correctly that in order to make my code Py2 and Py3 compatible, I should implement
__truediv__,__floordiv__and__div__?On the other hand, I reckon that Sage has default implementations of the double underscore methods, that I should rather not override.
I think I'll proceed as follows:
- Implement a
fraction_fieldmethod for cohomology rings raising a TypeError (note thatZZ.quo(6).fraction_field()raises a TypeError, too) - Do NOT implement
_div_-- the attempt to divide two elements of a cohomology ring will result in the TypeError raised by thefraction_field()method - Division of a cohomology element by an element of the base field will automatically work, by multiplying the inverse of the field element with the cohomology element, using
_lmul_/_rmul_. I should add an example/test in the documentation of the_lmul_/_rmul_methods
comment:81 in reply to: ↑ 68 Changed 3 years ago by
Replying to SimonKing:
I don't think that
__nonzero__does anything in Python 3. But now as you mention it: It is possible (and is actually used in some places in the cythonised parts of the Sage library and also in the current not-yet-published code of the cohomology package) to make the code depend on the language level. I.e., I could have definitions for__nonzero__and for__bool__, but only one of them would be compiled.
This did work, to some extent. But not really reliably. Therefore, to be on the safe side, I will implement both methods in both python versions, since the code duplication is only small.
comment:82 Changed 3 years ago by
I am currently dealing with different but mathematically equivalent behaviour: Apparently, previously Singular did not always do tail reduction, i.e., some rings in the database do not have a fully reduced representation of the relation ideal. For example, here are relations for the cohomology of SmallGroup(64,14):
sage: H.rels() # not tail reduced, result obtained with py2 ['a_1_0^2', 'a_1_0*a_1_1', 'a_1_1^3', 'a_2_1*a_1_0', 'a_2_1^2+a_2_1*a_1_1^2', 'a_1_1*a_3_3+a_2_1^2', 'a_1_0*a_3_3+a_2_1^2', 'a_2_1*a_3_3', 'a_3_3^2']
versus
sage: H.rels() # tail reduced, obtained with Py3 ['a_1_0^2', 'a_1_0*a_1_1', 'a_1_1^3', 'a_2_1*a_1_0', 'a_2_1^2+a_2_1*a_1_1^2', 'a_1_0*a_3_3+a_2_1*a_1_1^2', 'a_1_1*a_3_3+a_2_1*a_1_1^2', 'a_2_1*a_3_3', 'a_3_3^2']
As you can see, the relation a_2_1^2+a_2_1*a_1_1^2 could be used to reduce the tail of a_1_1*a_3_3+a_2_1^2. Also, the sorting of the relations is different.
The question is how to deal with it. I guess I could proceed as follows:
sage: if (2, 8) < sys.version_info: ....: expected_rels = ['a_1_0^2', 'a_1_0*a_1_1', 'a_1_1^3', 'a_2_1*a_1_0', ....: 'a_2_1^2+a_2_1*a_1_1^2', 'a_1_0*a_3_3+a_2_1*a_1_1^2', ....: 'a_1_1*a_3_3+a_2_1*a_1_1^2', 'a_2_1*a_3_3', 'a_3_3^2'] ....: else: ....: expected_rels = ['a_1_0^2', 'a_1_0*a_1_1', 'a_1_1^3', 'a_2_1*a_1_0', ....: 'a_2_1^2+a_2_1*a_1_1^2', 'a_1_1*a_3_3+a_2_1^2', ....: 'a_1_0*a_3_3+a_2_1^2', 'a_2_1*a_3_3', 'a_3_3^2'] ....: sage: H.rels() == expected_rels True
comment:83 Changed 3 years ago by
Sigh. I am suffering from "interesting" debugging sessions. At the moment, it seems to me that some errors in the py2-version only occur, when I first install p_group_cohomology in the py2-install of Sage, then install p_group_cohomology in the py3-install of Sage, and only then do tests in the py2-install.
comment:84 follow-up: ↓ 85 Changed 3 years ago by
I had a bad experience with doing py2/3 things on the same account, the problems was apparently in the content of ~/.sage/.
comment:85 in reply to: ↑ 84 ; follow-up: ↓ 86 Changed 3 years ago by
Replying to dimpase:
I had a bad experience with doing py2/3 things on the same account, the problems was apparently in the content of
~/.sage/.
You mean by testing the p_group_cohomology package, or something else?
Indeed I had the same impression concerning ~/.sage/. Fortunately most of the trouble could be solved by putting str() around some data (because I got unicode where a string was expected - so, perhaps a rogue py3-pickle got misinterpreted in py2).
comment:86 in reply to: ↑ 85 Changed 3 years ago by
Replying to SimonKing:
Replying to dimpase:
I had a bad experience with doing py2/3 things on the same account, the problems was apparently in the content of
~/.sage/.You mean by testing the p_group_cohomology package, or something else?
Indeed I had the same impression concerning
~/.sage/. Fortunately most of the trouble could be solved by puttingstr()around some data (because I got unicode where a string was expected - so, perhaps a rogue py3-pickle got misinterpreted in py2).
I was getting different output from a test, IIRC it was releated to unicode, and the problem went away after I removed ~/.sage/. I thought it might have been related to ipython, but who knowns.
comment:87 Changed 3 years ago by
I tried to solve it by doing an explicit conversion to str whenever there is data that is expected of this type. And instead of if isinstance(bla, str), I now do if isinstance(bla, (str, unicode)), where I define unicode=str in py3.
So far, this seems to work. And after all, most users will have only one Sage installation anyway.
comment:88 in reply to: ↑ 13 Changed 3 years ago by
Replying to SimonKing:
Replying to SimonKing:
If the .sobj file's location is not stored as an attribute in Python 3 then I have a problem.
Hooray:
# in Sage-with-python-2 sage: P = CombinatorialFreeModule(ZZ,'x') sage: save(P,'../../pickle')# BOTH with python-2 and python-3 sage: B = load('../../pickle.sobj') sage: B._default_filename '/home/king/Sage/pickle.sobj'So, the old trick should still work.
I wonder if I shouldn't improve the trick.
Currently, if the initialisation of an unpickled object can not be finished, it is the __getattr__ method that would trigger the completion of initialisation based on the value of self._default_filename that is set by Sage's loads() after unpickling the object and before returning it.
But what if I turn _default_filename into a property? I could make it so that its setter triggers the completion of initialisation, to the effect that the object returned by loads() will be valid. The trickery would be in the property setter, not in __getattr__.
comment:89 Changed 3 years ago by
I am puzzled about the appearance of unicode. For example, consider these two lines in my code:
root = str(kwds.get('root', COHO.workspace)) coho_logger.debug("Group data are rooted at %r", None, root)
The logging appears in the following session (in Sage-with-py2, where I first installed the package in Sage-with-py3 and then in Sage-with-py2):
sage: from pGroupCohomology import CohomologyRing
sage: CohomologyRing.doctest_setup()
sage: CohomologyRing.global_options('debug')
sage: from pGroupCohomology.cohomology import COHO
sage: _ = COHO()
<module>:
Group data are rooted at u'/home/king/.sage/temp/klap/6618/dir_T5Im2o/'
So, apparently the variable root in the code snipped above is a unicode, although I explicitly converted it to a str. Moreover:
sage: COHO.workspace '/home/king/.sage/temp/klap/6618/dir_T5Im2o/' sage: type(COHO.workspace) <type 'str'>
In other words: There is an attribute COHO.workspace of type str, and when I convert it to a str then a unicode results. WTF???
comment:90 follow-up: ↓ 91 Changed 3 years ago by
I don't see a full picture here, as it's unclear which sessions are in py2 and which are in py3.
It might be that iPython installs some kind of display hook that sets str to convert to unicode. There is code in src/sage/repl/rich_output that might be doing this.
That is to say that unicode you see printed might be happening due to on the fly conversion, set up by DisplayManager.
comment:91 in reply to: ↑ 90 Changed 3 years ago by
Replying to dimpase:
I don't see a full picture here, as it's unclear which sessions are in py2 and which are in py3.
Is there unicode in py3?
The common topic is: In some cases I get a TypeError saying that a str was expected, but got unicode. And a log shows that a variable is of type unicode after an EXPLICIT(!) cast to str.
In all mentioned cases, I proceeded as follows:
- Install the package in Sage-with-py2
- Install the package in Sage-with-py3
- Test in Sage-with-py2
Sage-with-py2 and Sage-with-py3 are of course in different folders. What they share: The git repository (but different git worktrees), and DOT_SAGE.
But it is all very flaky. I just inserted a couple of print statements in my code to investigate more deeply, did the above double-install procedure --- and suddenly the type was str, not unicode.
It might be that iPython installs some kind of display hook that sets
strto convert to unicode. There is code insrc/sage/repl/rich_outputthat might be doing this.
It is not only the output but the type inside of the code. In some cases, I get a TypeError.
comment:92 Changed 3 years ago by
Do you see these errors while running doctests? I gather there is --nodotsage option you can use while running tests, so you may try to see whether you can reproduce this without DOT_SAGE.
Another thing to try is to wipe out DOT_SAGE each time you switch from pyX to py(5-X), and see whether this make the error to go away.
comment:93 Changed 3 years ago by
another way to work around this might be just to convert all your strings to unicode.
comment:94 Changed 3 years ago by
I am afraid that it is currently not possible for anybody else to reproduce the following, because I didn't publish my code yet. But anyway, here is what I did:
- Insert type information, so that the mentioned code snippet now is
root = str(kwds.get('root', COHO.workspace)) coho_logger.debug("Group data are rooted at %r. Type: %r", None, root, type(root))
- Install it in py3
- Install it in py2
- In Py3, run the following:
sage: from pGroupCohomology import CohomologyRing sage: CohomologyRing.doctest_setup() sage: H = CohomologyRing(64,14, from_scratch=True) sage: H.make()
- In Py2, run the following:
sage: from pGroupCohomology import CohomologyRing sage: from pGroupCohomology.cohomology import COHO sage: CohomologyRing.doctest_setup() sage: CohomologyRing.global_options('debug') sage: _ = COHO() <module>: Group data are rooted at u'/home/king/.sage/temp/klap/14917/dir_20mizE/'. Type: <type 'unicode'>
So, the type REALLY is unicode, even though I did an EXPLICIT conversion to str.
Then, I continued:
- Move the old
~/.sageand create a new one. - Start Sage-with-py2, and repeat the test:
Setting permissions of DOT_SAGE directory so only you can read and write it. sage: from pGroupCohomology import CohomologyRing sage: from pGroupCohomology.cohomology import COHO sage: CohomologyRing.doctest_setup() sage: CohomologyRing.global_options('debug') sage: _ = COHO() <module>: Group data are rooted at u'/home/king/.sage/temp/klap/15112/dir_fYCkwo/'. Type: <type 'unicode'> sage: type(COHO.workspace) <type 'str'>
So, removing DOT_SAGE didn't help. For comparison, do the same in Py3:
sage: from pGroupCohomology import CohomologyRing
sage: from pGroupCohomology.cohomology import COHO
sage: CohomologyRing.doctest_setup()
sage: CohomologyRing.global_options('debug')
sage: _ = COHO()
<module>:
Group data are rooted at '/home/king/.sage/temp/klap/15493/dir_81mwix3u/'. Type: <class 'str'>
which of course is no surprise, as python-3 has no unicode.
Note that the line _ = COHO() does not involve taking an old pickle from a repository. It merely creates a new instance of COHO and initialises it only to the point where the current workspace is assigned to the attribute .root.
Summary: The class attribute COHO.workspace is of type str. The dictionary kwds is empty. str(kwds.get('root', COHO.workspace))) returns the unicode version of `COHO.workspace. I.e., converting a str to a str results in a unicode.
comment:95 Changed 3 years ago by
Just to document that we are talking about a Heisenbug: I wanted to test whether it is perhaps the case that str==unicode. Therefore, I changed the code snippet to
root = str(kwds.get('root', COHO.workspace))
coho_logger.debug("Group data are rooted at %r. Type: %r, %r, %r", None, root, type(root), type(COHO.workspace), str)
, installed it in py2 (only), and rerun the test in py2:
sage: from pGroupCohomology import CohomologyRing
sage: from pGroupCohomology.cohomology import COHO
sage: CohomologyRing.doctest_setup()
sage: CohomologyRing.global_options('debug')
sage: _ = COHO()
<module>:
Group data are rooted at '/home/king/.sage/temp/klap/16014/dir_M5saF0/'. Type: <type 'str'>, <type 'str'>, <type 'str'>
Hence, no bug. "Hooray".
Start a sage-with-py3-session and do
sage: from pGroupCohomology import CohomologyRing sage: CohomologyRing.doctest_setup() sage: H = CohomologyRing(64,14, from_scratch=True) sage: H.make()
The hope was of course that it triggers something. But the test in py2 does not have a unicode problem. "Hooray".
But I am sure that the problem will pop up sooner or later.
comment:96 Changed 3 years ago by
To make sure that this bug doesn't pop up again, I inserted a test whether the python version is less than (2,8) and root is of type unicode. In this case, I print some information and then raise a runtime error. I guess it's the best that I can do for now.
After recompiling and rerunning the above tests, they went fine again. But sooner or later it will return, and then the diagnostic tests I inserted will perhaps tell more clearly what is happening.
comment:97 Changed 3 years ago by
Or another idea: Since it is Cython code, I could do cdef str root = ....
comment:98 Changed 3 years ago by
Hooray!
The inserted test already showed something:
sage: from pGroupCohomology import CohomologyRing
sage: from pGroupCohomology.cohomology import COHO
sage: CohomologyRing.doctest_setup()
sage: CohomologyRing.global_options('debug')
sage: _ = COHO()
Critical bug in python.
kwds.get('root', COHO.workspace) is <type 'str'>
COHO.workspace is <type 'str'>
str=<type 'unicode'>, unicode=<type 'unicode'>
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-5-a303ce041f27> in <module>()
----> 1 _ = COHO()
pGroupCohomology/cohomology.pyx in pGroupCohomology.cohomology.COHO.__init__()
RuntimeError: An explicit cast to <str> resulted in a unicode object
Aha! For some silly reason, we have str == unicode!!
EDIT: The above was printed like this:
if (2, 8) > sys.version_info and isinstance(root, unicode): print("Critical bug in python.") print("kwds.get('root', COHO.workspace) is %r"%type(kwds.get('root', COHO.workspace))) print("COHO.workspace is %r"%type(COHO.workspace)) print("str=%r, unicode=%r"%(str,unicode)) raise RuntimeError("An explicit cast to <str> resulted in a unicode object")
comment:99 Changed 3 years ago by
How to cope with it? Shall I test upon importing of the module whether str == unicode in Python-2, and raise an error whose message tells to recompile?
comment:100 follow-up: ↓ 101 Changed 3 years ago by
So, would this still pop up if right before running these tests you removed ~/.sage/ ?
comment:101 in reply to: ↑ 100 Changed 3 years ago by
Replying to dimpase:
So, would this still pop up if right before running these tests you removed
~/.sage/?
See comment:94. There, it didn't help. Now, it doesn't either:
king@klap:~/Sage/git$ mv ~/.sage/ ~/.sageBAK
king@klap:~/Sage/git$ mkdir ~/.sage
king@klap:~/Sage/git$ sage/sage
┌────────────────────────────────────────────────────────────────────┐
│ SageMath version 8.9.beta8, Release Date: 2019-08-25 │
│ Using Python 2.7.15. Type "help()" for help. │
└────────────────────────────────────────────────────────────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Warning: this is a prerelease version, and it may be unstable. ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
Setting permissions of DOT_SAGE directory so only you can read and write it.
sage: from pGroupCohomology import CohomologyRing
sage: from pGroupCohomology.cohomology import COHO
sage: CohomologyRing.doctest_setup()
sage: CohomologyRing.global_options('debug')
sage: _ = COHO()
Critical bug in python.
kwds.get('root', COHO.workspace) is <type 'str'>
COHO.workspace is <type 'str'>
str=<type 'unicode'>, unicode=<type 'unicode'>
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-5-a303ce041f27> in <module>()
----> 1 _ = COHO()
pGroupCohomology/cohomology.pyx in pGroupCohomology.cohomology.COHO.__init__()
RuntimeError: An explicit cast to <str> resulted in a unicode object
comment:102 Changed 3 years ago by
I wonder if the problem occurs because I am not installing the development versions of my package from an actual "package" (using sage -i p_group_cohomology), but directly from the sources. I.e., in the src folder of my development tree, I do
sage -python setup.py build sage -pip install . --upgrade
I know that sage -python setup.py build does create .c files in the src folder, but since the changed module is recompiled both in py2 and py3, I thought that it doesn't matter. And for efficiency, I want that only modules are recompiled that have changed.
So, I would like to test if the problem persists when separating the py2- and py3 build folders. When doing sage -python setup.py build, is it somehow possible to prescribe in what folder the .c and .pyc files are created? Then, I could choose separate folders for py2 and py3.
comment:103 Changed 3 years ago by
I am now putting cythonize(..., build_dir=os.path.join("build","c_files-{}.{}".format(version_info.major, version_info.minor))) into my setup.py. In that way, the c-files will be created in separate directories for both Sage versions.
Let's see if this is enough to solve the problem!
comment:104 Changed 3 years ago by
PS: Even if it solves the problem, I will test upon import whether str is unicode in py2, and if it is, raise a RuntimeError advising to recompile. I hope this error will never pop up...
comment:105 Changed 3 years ago by
After changing setup.py so that the .c files will be in different folders for different python versions, tests in cohomology.pyx pass both with py2 and py3 and then py2 again. So, I can now turn to modular_cohomology.pyx and finally init.py (which contains a long doc string, namely the "general overview" of the package).
comment:106 Changed 3 years ago by
Hooray! When I ran sage -tp 2 --force-lib --long --warn-long 60.1, all tests passed both in python 2 and python 3. I don't know how significant the run time differences are. In any case, here are the details:
Python-3
========
Running doctests with ID 2019-09-25-00-50-30-78efa273.
Using --optional=build,dochtml,gap_packages,libsemigroups,meataxe,memlimit,mpir,python2,sage
Sorting sources by runtime so that slower doctests are run first....
Doctesting 13 files using 2 threads.
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/cochain.pxd
[0 tests, 0.00 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/cochain.pyx
[1481 tests, 350.33 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/cohomology.pyx
[1445 tests, 363.25 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/isomorphism_test.py
**********************************************************************
File "/home/king/Projekte/coho/coho-devel/src/pGroupCohomology/isomorphism_test.py", line 67, in pGroupCohomology.isomorphism_test.IsomorphismTest
Warning, slow doctest:
T.explore_isomorphisms() # long time
Test ran for 122.79 s
[213 tests, 221.05 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/modular_cohomology.pyx
[459 tests, 274.13 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/factory.py
[229 tests, 126.99 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/__init__.py
[343 tests, 96.76 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/dickson.pyx
[28 tests, 1.53 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/auxiliaries.py
[39 tests, 0.79 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/barcode.py
[143 tests, 10.31 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/resolution.pxd
[0 tests, 0.00 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/resolution_bindings.pxd
[0 tests, 0.00 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/resolution.pyx
[882 tests, 22.95 s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: 765.7 seconds
cpu time: 939.6 seconds
cumulative wall time: 1468.1 seconds
Python-2
========
Running doctests with ID 2019-09-25-01-04-47-fa1de7f4.
Git branch: matrix_implementation_keyword
Using --optional=build,ccache,dochtml,frobby,gap_packages,gdb,gfortran,libsemigroups,meataxe,memlimit,mpir,p_group_cohomology,python2,sage
Sorting sources by runtime so that slower doctests are run first....
Doctesting 13 files using 2 threads.
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/cochain.pyx
[1481 tests, 437.44 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/cochain.pxd
[0 tests, 0.00 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/cohomology.pyx
[1445 tests, 465.15 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/isomorphism_test.py
**********************************************************************
File "/home/king/Projekte/coho/coho-devel/src/pGroupCohomology/isomorphism_test.py", line 67, in pGroupCohomology.isomorphism_test.IsomorphismTest
Warning, slow doctest:
T.explore_isomorphisms() # long time
Test ran for 116.79 s
[213 tests, 229.85 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/modular_cohomology.pyx
[459 tests, 312.40 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/factory.py
[229 tests, 124.88 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/__init__.py
[343 tests, 96.86 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/resolution.pxd
[0 tests, 0.00 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/resolution.pyx
[882 tests, 21.38 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/dickson.pyx
[28 tests, 1.63 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/auxiliaries.py
[39 tests, 0.78 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/resolution_bindings.pxd
[0 tests, 0.00 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/barcode.py
[143 tests, 14.90 s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: 873.5 seconds
cpu time: 996.8 seconds
cumulative wall time: 1705.3 seconds
Hence, tomorrow (i.e., in central European time, today in a few hours :-)) I will create a new package version and post it here for review.
comment:107 Changed 3 years ago by
Something more to do: I think I should add a test showing what happens when moving the directory associated with a cohomology ring. The package can cope with moved data, based on some trickery in the attribute _default_filename that is defined by Sage's unpickler. I've turned _default_filename into a property (with getter, setter and deleter), and its documentation should contain the new test.
comment:108 Changed 3 years ago by
- Description modified (diff)
I'm afraid that another test failure popped up. So, I cannot post the new version yet.
comment:109 Changed 3 years ago by
After fixing the failed test (in which I forgot to save something before loading it), I added a new feature to unit_test_64: Previously it was only tested whether the generator degrees and the Poincaré series of a freshly computed cohomology ring is consistent with what is stored in the database; now I test whether the new ring and the stored ring are isomorphic.
Indeed they are, for all groups of order 64. More precisely: In many cases the generators of the relation ideal are not identical. But in all cases, mapping the i-th generator of the new ring to the i-th generator of the old ring results in an isomorphism.
There is one remaining py3 issue, though: When I try to build the documentation, I get the error "No module named sphinx". The documentation does build, when I use py2. What can I do to fix it in py3?
EDIT: Is it not the case that sphinx is installed in Sage for building its documentation, both in py2 and py3? I am not sure if I ever tried to build the documentation of Sage-with-py3, though. Will sphinx be installed when I do a plain make in Sage (i.e., when I build the docs)?
comment:110 follow-up: ↓ 111 Changed 3 years ago by
Sphinx should be installed, yes. Look for SAGE_ROOT/local/lib/python3.7/site-packages/sphinx/. If you run Sage, can you import sphinx?
Edit: this is for a Python 3 build of Sage.
comment:111 in reply to: ↑ 110 Changed 3 years ago by
Replying to jhpalmieri:
Sphinx should be installed, yes. Look for
SAGE_ROOT/local/lib/python3.7/site-packages/sphinx/. If you run Sage, can youimport sphinx?
It was a trivial mistake: In the doc's Makefile, I had SPHINXBUILD = python -msphinx, but since python means python-2 even in a py3-install of Sage, I have to use SPHINXBUILD = sage -python -msphinx
comment:112 Changed 3 years ago by
- Branch set to u/SimonKing/p_group_cohomology_in_py_3
comment:113 Changed 3 years ago by
- Commit set to 463806c6cebb3120e20df0f19a216dd529553741
- Description modified (diff)
- Status changed from new to needs_review
I checked that the package builds and passes tests in Sage-with-py3. I checked that when running the tests directly on the sources (not during package installation) the tests pass in Sage-with-py2. I recommend that whoever reviews this tries the package both in py2 and in py3.
New commits:
| 2e0d11e | Pass unpickling options to pickle.load, default encoding 'latin1'. Accept both str and bytes in mtx_unpickle
|
| 8c105cd | Make str_to_bytes/bytes_to_str accept both str and bytes input.
|
| c8a0748 | Add tests for #28444
|
| 89ac77a | Fix keyword for py3-only test
|
| 3693024 | Fix doc strings in sage.misc.persist
|
| f0828ee | Add a comment regarding the expected data type for an unpickle helper
|
| ba41ebe | Fix two typos in a comment
|
| 463806c | Upgrade p_group_cohomology to version 3.3
|
cmpis also not supported in Python 3, and for a dictionaryd, you should dox in drather thand.has_key(x).