Upgrade to Singular-4.x.x

[jdemeyer]

Lots of stuff has changed in Singular 4. This is a big update. Build system completely changed, and a lot of internal stuff.

The branch here is based on commit a070b84d whose message tells 4.0.3p3:

• ​https://github.com/Singular/Sources/commit/a070b84d59be835a2946d1386edd3221d80f3fe7

The tarball is here:

​http://users.ox.ac.uk/~coml0531/sage/singular-4.0.3p3.tar.bz2

It was made using the following procedure :

env SINGULAR_GIT_COMMIT=a070b84d ./sage --sh build/pkgs/singular/spkg-src
./sage --package fix-checksum singular

This script is loosely based on the make_tar.sh script from upstream.

[jpflori]

Yeah, the update is non trivial, that's a reason to get #17184 (and so #16882) inbetween.

[jdemeyer]

Replying to jpflori:

Yeah, the update is non trivial

I wonder if we even have a Sage developer who is familiar enough with Singular to do this (for the record: not me, I know nothing about Singular).

that's a reason to get #17184 (and so #16882) inbetween.

Of course, I'm not arguing with that.

[jpflori]

Replying to jdemeyer:

Replying to jpflori:

Yeah, the update is non trivial

I wonder if we even have a Sage developer who is familiar enough with Singular to do this (for the record: not me, I know nothing about Singular).

Maybe Burcin would do... IIRC the update was proposed as a GSoC project.

[jakobkroeker]

Q: where to get the singular tarball for this commit? I get a failure at

Trying to download http://www.sagemath.org/packages/upstream/singular/singular-4.0.1p1.tar.bz2

2.

Update to Singular 4.0.1

Maybe Burcin would do...

Could the update to 4.0.1. be done by at least two or three persons to spread the knownledge about Singular internals?

[jpflori]

You have to put a tarball repackaged by the spkg-src script in the upstream folder directly. Once the ticket is merged, the tarball will be put on the official mirror and could be downloaded automatically but not at the moment.

I've put a repackaged tarball at:

• ​http://boxen.math.washington.edu/home/jpflori/upstream/singular-4.0.1.tar.bz2

[jpflori]

(I'm not sure the checksums will the same as what Jeroen branch expect, run ./sage -sh sage-fix-pkg-checksums after you have downloaded the tarball to fix this.)

[jpflori]

And I'd be happy to work on the upgrade and learn more about Singular's internals, but please also review the "easy" #17184 (where I've already learnt some stuff) firsT.

[jakobkroeker]

Singular 4.0.1. compiles now in sage but sage does not yet - I just pushed a branch 'u/jakobkroeker/ticket/17254'. You have to correct the 'singular-4.0.1p1.tar.bz2' tarball to get it work ( the 'm4' folder is missing).

For the moment I removed some patches; we probably will have to adjust them later on. For example there is meanwhile no such thing like 'lomalloc_ndebug' so if we want to disable some debug functionality or output for omalloc we have to do that by passing appropriate flags to omalloc/configure, e,g '--enable-optimizationflags --disable-debug --without-debug' ?

Also I disabled the currently broken '--enable-debugoutput' configure flag for the standalone 'factory' and informed Oleksander Motsak about the issue.

Could someone of you take care of the sage compile errors and figure out what is going wrong and/or ask for help in the Singular forum? I can't do that easily, because within the Singular-team I'm out-of-favor because I constantly criticise their software and thus I will likely get little or no support.

Jakob

[jdemeyer]

New commits:

​4c0bba3

singular 4.0.1 builds now, but sage does not due to changes in libsingular?

[jakobkroeker]

first hint: for example the interface of 'naIsZero' seems changed from

BOOLEAN naIsZero(number a)

to

BOOLEAN naIsZero(number a, const coeffs cf)

Persons who might have an idea about the changes:

Oleksandr Motsak Mohamed Barakat Martin Lee Hans Schönemann

see contact details at ​http://www.mathematik.uni-kl.de/agag/mitglieder/

[jdemeyer]

Replying to jpflori:

I've put a repackaged tarball at:

• ​http://boxen.math.washington.edu/home/jpflori/upstream/singular-4.0.1.tar.bz2

That's an older version than the one that I put in the ticket description, any reason...?

[jdemeyer]

Replying to jakobkroeker:

You have to correct the 'singular-4.0.1p1.tar.bz2' tarball to get it work ( the 'm4' folder is missing).

My tarball compiled fine, so what is the problem?

[jdemeyer]

You should never run ./autogen.sh, that is supposed to be run at packaging-time, not build-time.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​79c079f	Upgrade to Singular-4-0-1
​4c0bba3	singular 4.0.1 builds now, but sage does not due to changes in libsingular?
​80a6b05	Merge commit '4c0bba3' into ticket/17254

[jakobkroeker]

My tarball compiled fine, so what is the problem?

Confirm; cannot reproduce the issue. So likely I didn't use the recent tarball, or I screwed something up.

You should never run ./autogen.sh, that is supposed to be run at packaging-time, not build-time.

I'm guilty ;-)

[jakobkroeker]

singulars spkg-install:

the '--enable-debugoutput' configure flag for standalone factory needs now in additon '--enable-streamio --without-Singular'

[jakobkroeker]

After a little more tweaking and temporarily disabling the extension fields/rings, I was able to build some parts of sage with Singular 4.0.1 on a 'fc18 system' and then.. sage crashes at start! That happens because I didn't update ring creation right - I only recently discovered examples at './libpolys/tests/rings_test.h'. And some sage packages create common rings at loading time.

remaining riddles:

• during build one sage file tried to link to -lsingular (I have no idea why), instead to the libSingular* files (that changed!), so I worked around by manually creating links to libSingular* files.

• on a ScientificLinux? system I get errors like "ImportError?: /home/kroeker/Projects/sage-singular-4.0.1/local/lib/python2.7/site-packages/sage/rings/polynomial/plural.so: undefined symbol: wFunctionalBuch". This symbol is in libSingular*

• fix ring creation. See examples at './libpolys/tests/rings_test.h'

• the extension fields will need some more love. See 'algext.h' and 'transext.h'. @Martin: could you take a look?

• some mapping functions (e.g. nr2mMapZp) now require a source and dst (coefficient) parameter. Not sure if I did it right (source: currentRing, dest: _ring or vice versa?)

• I didn't fix unused interface definitions in singular-cdefs.pxi. TODO

See branch u/jakobkroeker/ticket/17254 and some notes at ​https://github.com/jakobkroeker/jk_test_sage/issues/13

If a build get stuck before errors described above, then maybe a 'make distclean' is necessary. And of course, the Singular tarball ​http://boxen.math.washington.edu/home/jdemeyer/spkg/singular-4.0.1p1.tar.bz2 is required.

Any help appreciated.

---

New commits:

​79c079f	Upgrade to Singular-4-0-1
​2fcaeb5	Update Singular to 4.0.1
​4c0bba3	singular 4.0.1 builds now, but sage does not due to changes in libsingular?
​80a6b05	Merge commit '4c0bba3' into ticket/17254
​2ae243a	messy fixes
​ff2a9dc	required changes for singular to build sage; should be fixed upstream
​3b7877e	hacking sage to build with Singular 4.0.1. needs more love for extension coefficient fields and a refactoring
​f178dfc	fixing nr2mMapZp call

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​8ba17d8

messy fixes

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​2ae243a	messy fixes
​ff2a9dc	required changes for singular to build sage; should be fixed upstream
​3b7877e	hacking sage to build with Singular 4.0.1. needs more love for extension coefficient fields and a refactoring
​f178dfc	fixing nr2mMapZp call
​8e82387	Merge branch 'u/jakobkroeker/jakobkroeker.ticket.17254' of git://trac.sagemath.org/sage into jakobkroeker.ticket.17254
​c2a21bd	fixed some ring creation

[jakobkroeker]

New commits:

​6384b9b

Upgrade to Singular 4.0.1: incomplete, messy patch(first aproach)

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​221c5d6

Upgrade to Singular 4.0.1: incomplete, messy patch(first aproach)

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​1e84789

one more renaming from libsingular to libSingular

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​3a512cd

improved error handling messages when loading Singular

[Snark]

What is the status of this ticket? I see the branch is off 6.5.rc0, which is quite old already...

[jakobkroeker]

What is the status of this ticket? I see the branch is off 6.5.rc0, which is quite old already...

I'm busy and can't work on the ticket currently and expect to have time for the ticket at end of June, not earlier...

with the state as is, sage should build and start (using the Singular tarball from above), but extension fields and transcendental fields are not functional yet and also a cython Singular-interface cleanup has to be done.

[john_perry]

Replying to jakobkroeker:

I'm busy and can't work on the ticket currently and expect to have time for the ticket at end of June, not earlier...

with the state as is, sage should build and start (using the Singular tarball from above), but extension fields and transcendental fields are not functional yet and also a cython Singular-interface cleanup has to be done.

I'm at SD64.25 and am willing to work on this, but I'll need some hand-holding since I haven't worked on a package like this before. Do you have any suggestions for ways to test extension and transcendental fields?

[john_perry]

I just applied this to 6.4.1 and encountered the message,

fatal error: Singular/libsingular.h: No such file or directory

while compiling

build/cythonized/sage/algebras/letterplace/free_algebra_element_letterplace.cpp

Any ideas what causes this?

[fbissey]

I haven't looked yet at the new singular 4.0.x but the first port of call would be to check if the header is indeed installed in $SAGE_LOCAL/include.

[john_perry]

Replying to fbissey:

I haven't looked yet at the new singular 4.0.x but the first port of call would be to check if the header is indeed installed in $SAGE_LOCAL/include.

Shortly after, that installation of Sage fell completely apart (not sure why) so now I'm working on Sage-6.7.

On 6.7, git identifies some issues with merging in singular.pyx and ring.pyx, but they're fairly clear to fix, so I did. (Unless they weren't easy.) Things went much further this time, until I encountered this error:

****************************************************
### Singular spkg-install: choose_patches ###
### Singular spkg-install: apply_patches ###
Applying /Applications/sage-6.7/local/var/tmp/sage/build/singular-4.0.1p1.p0/patches/stricmp.patch
can't find file to patch at input line 7
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|stricmp is being deprecated in Cygwin.
|One should use strcasecmp.
|See https://cygwin.com/ml/cygwin/2014-10/msg00359.html
|diff -druN src/latest/Singular/run.c src/latest/Singular/run.c
|--- latest/Singular/run.c	2014-11-19 14:06:05.000000000 +0100
|+++ latest/Singular/run.c	2015-01-16 09:32:45.771298300 +0100
--------------------------
File to patch: 
Skip this patch? [y] 
Skipping patch.
1 out of 1 hunk ignored
Error applying '/Applications/sage-6.7/local/var/tmp/sage/build/singular-4.0.1p1.p0/patches/stricmp.patch'
Error building Singular (error in apply_patches).

Here's what the file in question suggests:

stricmp is being deprecated in Cygwin.
One should use strcasecmp.
See https://cygwin.com/ml/cygwin/2014-10/msg00359.html
diff -druN src/latest/Singular/run.c src/latest/Singular/run.c
--- latest/Singular/run.c       2014-11-19 14:06:05.000000000 +0100
+++ latest/Singular/run.c       2015-01-16 09:32:45.771298300 +0100
@@ -45,6 +45,7 @@
  #include <sys/stat.h>
  #include <sys/cygwin.h>
  #include <sys/unistd.h>
+ #define stricmp strcasecmp
 //WinMainCRTStartup() { mainCRTStartup(); }
 #else
  #include <direct.h>

Indeed, I do not find run.c in local/var/tmp/sage/build/singular-4.0.1p1.p0/src/latest/Singular/, which I am guessing is where this is supposed to be; instead, I find only run.h.

In my local distribution of Singular, I find a run.c. If I copy it over & try to make, I get the same complaint, & it looks as if the file is actually deleted. Suggestions?

[fbissey]

Did you mean local/var/tmp/sage/build/singular-4.0.1p1.p0/src/Singular/ the latest bit should be stripped by the -p1 option to patch.

[john_perry]

The patch has latest in it; that's why I wrote that in there, so that is what I mean. Is your meaning that latest should not be there? That directory doesn't exist when I look through the paths.

[fbissey]

It is expected for that directory not to exist for you. Whoever wrote the patch was working in a folder named latest but it is their own naming. It doesn't matter anyway. Let me explain to you how the -p option of patch works.

Your patch has a header that helps you locate the file to patch:

--- latest/Singular/run.c       2014-11-19 14:06:05.000000000 +0100
+++ latest/Singular/run.c       2015-01-16 09:32:45.771298300 +0100

patch < mypatch.patch or equivalently patch -p0< mypatch.patch will look for the file latest/Singular/run.c relative to where patch is executed. -pN will strip the N first folders in front of the file name so

patch -p1< mypatch.patch will look for Singular/run.c. In spkg-install for singular, patches are applied with patch -p1 <"$patch" so you shouldn't care about latest it is discarded. Look for local/var/tmp/sage/build/singular-4.0.1p1.p0/src/Singular/run.c OK?

[john_perry]

I think you misunderstand.

Replying to fbissey:

It is expected for that directory not to exist for you.

Why not? That directory does exist for me. The problem is that the file run.c does not exist in it.

Look for local/var/tmp/sage/build/singular-4.0.1p1.p0/src/Singular/run.c OK?

There is no such directory. I can get to local/var/tmp/sage/build/singular-4.0.1p1.p0/src/ but the only directories in there are latest and shared.

[fbissey]

OK you seem to be right. The tarball has been packaged strangely and I am not sure I have the right spkg-install. It is a cygwin patch and cygwin support in singular seems to have changed considerably. I would just remove the patch for now, it doesn't/cannot apply anywway any new fix for cygwin would have to be done from scratch by someone with a cygwin system.

[john_perry]

How do I remove it? If I move it & make anew, the file reappears.

[fbissey]

git rm build/pkgs/singular/patches/stricmp.patch?

[john_perry]

Thanks, that worked. Now I get failure on a line in ring.pyx,

    if is_64_bit:

Namely, Cython (?) claims this is undefined. I guess this is a change in Cython, since that isn't introduced by this patch. I found online another way to test this (sys.maxsize > 2**32) so I'm trying with that. Found another issue, but that was easy. Proceeding with fingers crossed... it's still compiling, which is already an progress...!

[john_perry]

It seems to have built, but Sage doesn't run. Here's the last few lines of the compilation (sorry for so much, but I wanted to show what it was saying at the end of what looks like a successful build, too):

...
byte-compiling /Applications/sage-6.7/local/lib/python2.7/site-packages/sage_setup/find.py to find.pyc
byte-compiling /Applications/sage-6.7/local/lib/python2.7/site-packages/sage_setup/optional_extension.py to optional_extension.pyc
running install_egg_info
Removing /Applications/sage-6.7/local/lib/python2.7/site-packages/sage-6.8.beta0-py2.7.egg-info
Writing /Applications/sage-6.7/local/lib/python2.7/site-packages/sage-6.8.beta0-py2.7.egg-info

real	0m25.849s
user	0m11.855s
sys	0m3.847s
[ -f local/etc/sage-started.txt ] || local/bin/sage-starts
build/pipestatus "./sage --docbuild --no-pdf-links all html  2>&1" "tee -a logs/dochtml.log"
Deleting empty directory /Applications/sage-6.7/src/doc/common/static
Deleting empty directory /Applications/sage-6.7/src/doc/en/reference/combinat/static
Deleting empty directory /Applications/sage-6.7/src/doc/en/reference/combinat/templates
Deleting empty directory /Applications/sage-6.7/src/doc/en/reference/polynomial_rings/static
Deleting empty directory /Applications/sage-6.7/src/doc/en/reference/polynomial_rings/templates
Deleting empty directory /Applications/sage-6.7/src/doc/en/reference/repl/static
Deleting empty directory /Applications/sage-6.7/src/doc/en/reference/repl/templates
Deleting empty directory /Applications/sage-6.7/src/doc/en/reference/tensor_free_modules/static
Deleting empty directory /Applications/sage-6.7/src/doc/en/reference/tensor_free_modules/templates
Deleting empty directory /Applications/sage-6.7/src/doc/output/inventory/en/reference/combinat
Traceback (most recent call last):
  File "/Applications/sage-6.7/src/doc/common/builder.py", line 1619, in <module>
    import sage.all
  File "/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/all.py", line 98, in <module>
    from sage.rings.all      import *
  File "/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/rings/all.py", line 68, in <module>
    from number_field.all import *
  File "/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/rings/number_field/all.py", line 7, in <module>
    from totallyreal import enumerate_totallyreal_fields_prim
  File "sage/rings/number_field/totallyreal_data.pxd", line 16, in init sage.rings.number_field.totallyreal (build/cythonized/sage/rings/number_field/totallyreal.c:10339)
  File "sage/rings/number_field/totallyreal_data.pyx", line 39, in init sage.rings.number_field.totallyreal_data (build/cythonized/sage/rings/number_field/totallyreal_data.c:11133)
  File "/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring_constructor.py", line 465, in PolynomialRing
    R = _single_variate(base_ring, name, sparse, implementation)
  File "/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring_constructor.py", line 539, in _single_variate
    R = m.PolynomialRing_integral_domain(base_ring, name, sparse, implementation)
  File "/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring.py", line 1544, in __init__
    sparse=sparse, element_class=element_class)
  File "/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring.py", line 1451, in __init__
    sparse=sparse, element_class=element_class, category=category)
  File "/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring.py", line 305, in __init__
    from sage.matrix.matrix_space import is_MatrixSpace
  File "/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/matrix/matrix_space.py", line 57, in <module>
    import matrix_mpolynomial_dense
ImportError: dlopen(/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/matrix/matrix_mpolynomial_dense.so, 2): Symbol not found: __Z17initCanonicalFormv
  Referenced from: /Applications/sage-6.7/local/lib/python2.7/site-packages/sage/matrix/matrix_mpolynomial_dense.so
  Expected in: flat namespace
 in /Applications/sage-6.7/local/lib/python2.7/site-packages/sage/matrix/matrix_mpolynomial_dense.so
make: *** [doc-html] Error 1

[fbissey]

It appears that it should be generated by initCanonicalForm but no functions of that name is to be found in the source code I have for singular 4.0.2 here and I am expecting it to be the same for you. I think you may have old singular headers on your system that are interfering (factory.h) we'll have to develop a proper upgrade procedure to remove singular 3.1.x stuff before installing 4.x. That or the singular 4.x headers have not been installed properly. local/include/factory/factory/factory.h and local/include/factory/factory.h both have this function in singular 3.1.7 but it shouldn't be present in the equivalent headers from singular 4.x.

[john_perry]

I don't seem to have a local/include/factory at all...

[fbissey]

I really have to inspect the spkg. I think I went overboard with the factory in the path. Should have been local/include/factory.h and local/include/factory/factory.h. Do you have a factory.h and does it contains initCanonicalForm?

[john_perry]

make distclean then make seems to have fixed my problems. I am apparently running Singular 4, albeit with the changes I mentioned earlier. At startup, I see the following:

ring with rational coefficient field created
_ring.ShortOut 1
_ring.N 2
polynomial ring over integers created
_ring.ShortOut 1
_ring.N 2
ring with rational coefficient field created
_ring.ShortOut 1
_ring.N 2

I found that in the source code, so I'm guessing this was put in to test.

[john_perry]

Well, this is bad.

sage: R=PolynomialRing(GF(32003),'x',7)
sage: I = sage.rings.ideal.Cyclic(R,7).homogenize()
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-2-59ae6a3e91c7> in <module>()
----> 1 I = sage.rings.ideal.Cyclic(R,Integer(7)).homogenize()

/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/rings/ideal.pyc in Cyclic(R, n, homog, singular)
   1607         n = R.ngens()
   1608 
-> 1609     singular.lib("poly")
   1610     R2 = R.change_ring(RationalField())
   1611     R2._singular_().set_ring()

/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/interfaces/singular.pyc in lib(self, lib, reload)
    783         if not reload and lib in self.__libs:
    784             return
--> 785         self.eval('LIB "%s"'%lib)
    786         self.__libs.append(lib)
    787 

/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/interfaces/singular.pyc in eval(self, x, allow_semicolon, strip, **kwds)
    585             x += ';'
    586 
--> 587         s = Expect.eval(self, x, **kwds)
    588 
    589         if s.find("error") != -1 or s.find("Segment fault") != -1:

/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/interfaces/expect.pyc in eval(self, code, strip, synchronize, locals, allow_use_file, split_lines, **kwds)
   1220                 elif split_lines:
   1221                     return '\n'.join([self._eval_line(L, allow_use_file=allow_use_file, **kwds)
-> 1222                                         for L in code.split('\n') if L != ''])
   1223                 else:
   1224                     return self._eval_line(code, allow_use_file=allow_use_file, **kwds)

/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/interfaces/expect.pyc in _eval_line(self, line, allow_use_file, wait_for_prompt, restart_if_needed)
    817         try:
    818             if self._expect is None:
--> 819                 self._start()
    820             E = self._expect
    821             try:

/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/interfaces/singular.pyc in _start(self, alt_message)
    410         """
    411         self.__libs = []
--> 412         Expect._start(self, alt_message)
    413         # Load some standard libraries.
    414         self.lib('general')   # assumed loaded by misc/constants.py

/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/interfaces/expect.pyc in _start(self, alt_message, block_during_init)
    427                 # Change pexpect errors to RuntimeError
    428                 raise RuntimeError("unable to start %s because the command %r failed: %s\n%s" %
--> 429                         (self.name(), cmd, e, self._install_hints()))
    430         except BaseException:
    431             self._expect = None

RuntimeError: unable to start singular because the command 'Singular -t --ticks-per-sec 1000' failed: The command was not found or was not executable: Singular.

[john_perry]

For what it's worth, lots of other things work.: I can, for instance, compute an ideal's Gröbner basis, its Hilbert [numerator, series, polynomial], and so forth.

[john_perry]

Indeed, there is no Singular binary in local/bin. I presume this is the problem, though I have no idea what became of it.

[john_perry]

I'm guessing that failure to install the singular binary is because of a symlink: local/bin/singular' points to local/bin/Singular and removing the old (3.x) Singular wasn't enough to allow the new one in. I removed local/bin/singular` and it's now working.

This may be an issue with Apple's file system, because after installing the new Singular I wasn't allowed to create the symlink.

[jdemeyer]

is_64_bit is deprecated. In Cython, check sizeof(the_type_that_you_care_about) instead.

Do not use sys.maxsize, that's much slower.

[fbissey]

Replying to john_perry:

I'm guessing that failure to install the singular binary is because of a symlink: local/bin/singular' points to local/bin/Singular and removing the old (3.x) Singular wasn't enough to allow the new one in. I removed local/bin/singular` and it's now working.

This may be an issue with Apple's file system, because after installing the new Singular I wasn't allowed to create the symlink.

Yes you may have seen another ticket in which I demonstrated that something like Singular was really singular. OS X file system is not case sensitive but it gives the impression of being one.

[john_perry]

Replying to john_perry:

I just applied this to 6.4.1 and encountered the message,

fatal error: Singular/libsingular.h: No such file or directory

while compiling

build/cythonized/sage/algebras/letterplace/free_algebra_element_letterplace.cpp

Any ideas what causes this?

It's now certain that this, too, was caused by the failure to install Singular because singular exists in the directory (what can I say, it's a Mac). Removing that link and reinstalling the spkg fixed all these problems.

Meanwhile, I discovered some memory-related bugs in the patch. For instance, ring.pyx has the line

assert( exponent == base_ring.degree() )

which causes a failure. I'm not sure why those should be equal (the test I used had Zmod(2^3) where exponent==3 and base_ring.degree()==1), but commenting it out leads to a failure on these lines:

_param.GFChar     = ch;
_param.GFDegree   = base_ring.degree();
_param.GFPar_name = omStrDup(base_ring.gen());   

because _param has not been initialized. Adding

_param = <GFInfo *>omAlloc(sizeof(GFInfo));

before the access gives me a ring. However,

1. I'm not sure whether omAlloc or one of the other omAlloc* commands is more appropriate (e.g., omAlloc0). Anyone know?
2. I suppose this has to be freed somewhere... I know there's an obvious-sounding place, so I'll find it.

There are other places this change has to occur. One more question:

• A few lines before the ones I cited above is the test for 64-bits. I've switched to jdemeyer's test (sizeof(long)>4). However, the code in the patch is exactly the same for both 32- and 64-bit. Is there a recommendation for what to do here? I'm not sufficiently familiar with this to know what to do.

[john_perry]

Replying to john_perry:

Meanwhile, I discovered some memory-related bugs in the patch. For instance, ring.pyx has the line

assert( exponent == base_ring.degree() )

which causes a failure. I'm not sure why those should be equal (the test I used had Zmod(2^3) where exponent==3 and base_ring.degree()==1)...

To wit, the source code for base_ring.degree() is, in total,

    def degree(self):
        """
        Return 1.

        EXAMPLE::

            sage: R = Integers(12345678900)
            sage: R.degree()
            1
        """
        return integer.Integer(1)

I don't know what is meant by the degree of a ring here, so I'll need guidance. For now I've commented it out.

[john_perry]

Pushed code should build on 6.7. Mac users may have problems; if so, delete local/bin/singular and then sage-spkg singular; it should work after that. (In fact, I have to do it again myself now.) I have to run to the train (leaving SD) or I'd do it before leaving.

---

New commits:

​bdb0851	first attempt to resolve
​1f4ebcb	fix for _param and discrepancy between exponent and degree

[john_perry]

Forgot to remove that bad patch; that will come in a subsequent push. Still have the problem with local/bin/Singular because of the local symlink; it's a problem with the package itself, because I deleted both symlink and local copy before running the install. I don't know how to fix that, so advice would be appreciated.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​9aa2456

delete stricmp.patch

[jdemeyer]

Replying to john_perry:

However, the code in the patch is exactly the same for both 32- and 64-bit. Is there a recommendation for what to do here?

Obviously, replace

if condition:
    X
else:
    X

by

X

[jdemeyer]

Replying to john_perry:

• A few lines before the ones I cited above is the test for 64-bits. I've switched to jdemeyer's test (sizeof(long)>4). However, the code in the patch is exactly the same for both 32- and 64-bit. Is there a recommendation for what to do here? I'm not sufficiently familiar with this to know what to do.

Look at the diff for the old corresponding code: this doesn't have the branch.

[jdemeyer]

I would also recommend you to keep the logic of the old code where applicable. For example, the old code started with

if ch.is_power_of(2):
exponent = ch.nbits() -1
    # it seems Singular uses ints somewhere
    # internally, cf. #6051 (Sage) and #138 (Singular)
    if exponent <= 30:

and I see no reason to change this.

[chapoton]

naive rebase.

The spkg is no longer available from boxen..

---

New commits:

​ab68eb0

Merge branch 'u/john_perry/ticket.17254.squashed' into 6.9.b0

[jakobkroeker]

The [singular 4.0.1] spkg is no longer available from boxen..

That is unfortunate. @jdemeyer/@jpflori: could you upload the singular 4.0.1.p1 tarball to somewhere else?

naive rebase. [by chapoton] Conflicts:

src/sage/libs/singular/singular-cdefs.pxi

the file src/sage/libs/singular/singular-cdefs.pxi is even missing in the ticket branch public/ticket/17254

@chapoton Did you set the ticket branch to broken public/ticket/17254 instead of leaving it 'u/john_perry/ticket.17254.squashed' by accident?

[jpflori]

It seems I don't have access to boxen anymore...

[jpflori]

And I don't have the p1 tarball Jeroen posted.

[fbissey]

I have a tarball answering that description. Up on the lmona.de server....

​http://www.lmona.de/files/sage/singular-4.0.1p1.tar.bz2

[jakobkroeker]

bumped to recent sage dvelopment version.

The build may fail with 'undefined ..' error again. Probably this can be fixed with ./sage -ba; if not, make distclean-bazooka should work...

---

New commits:

​8e259d1	merged with recent develop
​aa40c90	check for local tarball

[jakobkroeker]

@chapoton

naive rebase.

except for the issue with src/sage/libs/singular/singular-cdefs.pxi, how did you do the rebase? Did you a rebase by hand, or did you create a patch after merge and then apply the patch to 6.9.b0?

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​72ef927

squashed singular upgrade changes in one commit

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​4adc75d

squashed singular upgrade changes in one commit

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​f492ef2

squashed singular upgrade changes in one commit

[jakobkroeker]

@john_perry thanks for the bugfix!

I am willing to work on this

somehow I overlooked your messages in my emails in the past.Are you still available for help? Maybe we could work together on the remaining issues? (skype?)

An example of creating extension fields can be found in Singulars libpolys/tests/polys_test.h :: test_Q_Ext_a()) while the basic data structure idea is sketched in libpolys/polys/ext_fields/algext.h

[john_perry]

Replying to jakobkroeker:

@john_perry somehow I overlooked your messages in my emails in the past.Are you still available for help? Maybe we could work together on the remaining issues? (skype?)

I'm willing to collaborate on it, but I've had a more pressing project which has consumed my attention/free time. I thought I'd be done with it months ago :-) so I can't promise to be very helpful and/or focus on this at the moment, but I do hope to be available from time to time, until the current project passes in which case I could focus on this a little bit.

[jakobkroeker]

New commits:

​90cefc5

bugfixes

[jakobkroeker]

-added support for extensions over QQ; fixed some bugs of mine

open tasks: -put back si2sa_GFqGivaro, si2sa_GFqNTLGF2E, si2sa_GFq_generic

-and fix failing tests - a lot of tests (about 100) still fail:

naMap00() call in singular.pyx crashes - either I use it incorrectly or there is a bug in Singular.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​b4d0495

removed printouts

[jakobkroeker]

repackaged latest Singular

​https://www.dropbox.com/s/3io9k5srkcjrysr/singular-4.0.2.latest.tar.bz2?dl=0

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​5e14c77

updated to latest singular

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​179ecd8

support building singular in debug mode

[jakobkroeker]

@malb

while upgrading to Singular 4 I'm currently hit a segfault in Singular for

sage: x = var('x')
sage: K.<z> = QQ.extension(x^2 + 1)
sage: P.<v,w> = K[]
sage: P._singular_() 
sage: P(1) # crash

Since I neither familiar enough with the sage interface to Singular nor with Singular internals, I'm currently stuck. Could you take a look?

[malb]

So far, I have no luck even building this. Did you see this before:

ImportError: …/sage-review/local/lib/python2.7/site-packages/sage/matrix/matrix_mpolynomial_dense.so: undefined symbol: _Z17initCanonicalFormv

[jakobkroeker]

So far, I have no luck even building this. Did you see this before: ImportError?: …/sage-review/local/lib/python2.7/site-packages/sage/matrix/matrix_mpolynomial_dense.so: undefined symbol: _Z17initCanonicalFormv

Yes. These are clearly artifacts from singular 3.1.7. In Singular 4 is no 'initCanonicalForm'.

Either this is a caching issue, incorrect dependency issue(in sage/cython) or somewhere are files from 3.1.7 left. On which OS do you build singular( See also comment 55 )? Did you try 'make distclean'? ( If you have a better solution for this, please share it)

[jdemeyer]

Replying to jakobkroeker:

Either this is a caching issue, incorrect dependency issue(in sage/cython) or somewhere are files from 3.1.7 left.

If it is a dependency issue (which looks the most likely), please make a new ticket for it.

[malb]

Indeed, building from scratch made it go away … this will be fun to deal with eventually.

[malb]

Hi, I have rebuild Singular + Sage extension with SAGE_DEBUG=yes and it seems trouble starts a few lines earlier in your minimal example:

sage: x = var('x')
sage: K.<z> = QQ.extension(x^2 + 1)
sage: P.<v,w> = K[]
***omError_WrongSize: wrong size specification of addr                                                                                                                                                           [677/4609]
 occured at: 
  #0 at ??:0                 in ??                  
 occured for addr:0x7f7c9b303190 size:32 specified size:24

// ***dPolyReportError: memory error 
 occured at
 occured for poly: z2+1  addr:0x7f7c9b303190 size:32
***omError_WrongSize: wrong size specification of addr
 occured at: 
  #0 at ??:0                 in ??                  
 occured for addr:0x7f7c9b303190 size:32 specified size:24

// ***dPolyReportError: memory error 
 occured at
 occured for poly: !!longrat: NULL in monomials/p_polys.cc:3598
!!longrat: NULL in monomials/p_polys.cc:3598
oz2+oz2+...  addr:0x7f7c9b303190 size:32
***omError_WrongSize: wrong size specification of addr
 occured at: 
  #0 at ??:0                 in ??                  
 occured for addr:0x7f7c9b303190 size:32 specified size:24

// ***dPolyReportError: memory error 
 occured at
 occured for poly: !!longrat: NULL in monomials/p_polys.cc:3598
!!longrat: NULL in monomials/p_polys.cc:3598
oz2+oz2+...  addr:0x7f7c9b303190 size:32
***omError_WrongSize: wrong size specification of addr
 occured at: 
  #0 at ??:0                 in ??                  
 occured for addr:0x7f7c9b303190 size:32 specified size:24

// ***dPolyReportError: memory error 
 occured at
 occured for poly: !!longrat: NULL in monomials/p_polys.cc:3598
!!longrat: NULL in monomials/p_polys.cc:3598
oz2+oz2+...  addr:0x7f7c9b303190 size:32
***omError_WrongSize: wrong size specification of addr
 occured at: 
  #0 at ??:0                 in ??                  
 occured for addr:0x7f7c9b303190 size:32 specified size:24

// ***dPolyReportError: memory error 
 occured at
 occured for poly: !!longrat: NULL in monomials/p_polys.cc:3598
!!longrat: NULL in monomials/p_polys.cc:3598
oz2+oz2+...  addr:0x7f7c9b303190 size:32

[jakobkroeker]

Replying to malb:

Hi, I have rebuild Singular + Sage extension with SAGE_DEBUG=yes and it seems trouble starts a few lines earlier in your minimal example:

sage: x = var('x')
sage: K.<z> = QQ.extension(x^2 + 1)
sage: P.<v,w> = K[]
***omError_WrongSize: wrong size specification of addr [...] 

Do you know in detail what happens or should happen in sage for

sage: P.<v,w> = K[]
sage: P(1) 

behind the scenes? With that information we could ask Singular's developers if we use libSingular correctly.

[malb]

It'll construct a ring for the extension and then a ring in in v, w. I'll try to investigate further what's going wrong there (sorry, I'm being slow on this)

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7803030

located algext issue

[jakobkroeker]

Replying to malb:

It'll construct a ring for the extension and then a ring in in v, w. I'll try to investigate further what's going wrong there (sorry, I'm being slow on this)

trouble starts a few lines earlier in your minimal example:

Indeed. With printouts I tracked down the issue to ring creation in src/sage/libs/singular/ring.pyx I guess that I incorrectly assembly the minpoly in ring construction (omalloc magic or whatever) Or there is an issue with the AlgExtInfo declaration or in nInitChar() call.

Could you take a look or even fix it? See the corresponding Singular example test_Q_Ext_a() in latest/libpolys/tests/polys_test.h

[jdemeyer]

Replying to jakobkroeker:

So far, I have no luck even building this. Did you see this before: ImportError?: …/sage-review/local/lib/python2.7/site-packages/sage/matrix/matrix_mpolynomial_dense.so: undefined symbol: _Z17initCanonicalFormv

Yes. These are clearly artifacts from singular 3.1.7. In Singular 4 is no 'initCanonicalForm'.

The problem was with the include path containing old files, this is now fixed.

---

New commits:

​1f7f7f1

Fix Singular include path

[jpflori]

Any progress Martin? I'll rebuild a debug version of Sage and give it a try as well in the next few days.

[jpflori]

I looked a little bit into the problem and from what I see when the multivariate gets created the first thing done is to create the number field in Singular. Just after the number field creation, the first memory error Singular complains about is related to the polynomial defining the number field, so I would say the number field creation already went wrong (or the number field is wrongly used once created).

[jpflori]

Here is the backtrace of the first error reporting:

(gdb) bt
#0  omReportError (error=omError_WrongSize, report_error=omError_NoError,
    r=0x3fffa170be00 <omTestBinAddrSize+68>, fmt=0x3fffa171bd80 "")
    at omError.c:83
#1  0x00003fffa17120b0 in omReportAddrError (error=omError_WrongSize,
    report_error=omError_NoError, addr=0x3fffa13001f0, bin_size=0x18,
    flags=262, r=0x3fffa170be00 <omTestBinAddrSize+68>, fmt=0x3fffa171bd80 "")
    at omDebugCheck.c:409
#2  0x00003fffa1710d5c in omDoCheckBinAddr (addr=0x3fffa13001f0,
    bin_size=0x18, flags=262, level=1 '\001', report=omError_NoError,
    r=0x3fffa170be00 <omTestBinAddrSize+68>) at omDebugCheck.c:211
#3  0x00003fffa17108b8 in omDoCheckAddr (addr=0x3fffa13001f0, bin_size=0x18,
    flags=262, level=1 '\001', report=omError_NoError,
    r=0x3fffa170be00 <omTestBinAddrSize+68>) at omDebugCheck.c:169
#4  0x00003fffa170fd2c in _omCheckAddr (addr=0x3fffa13001f0, size_bin=0x18,
    flags=262, check=1 '\001', report=omError_NoError,
    r=0x3fffa170be00 <omTestBinAddrSize+68>) at omDebugCheck.c:46
#5  0x00003fffa170cd8c in _omDebugAddr (addr=0x3fffa13001f0, bin_size=0x18,
    flags=258, check=1 '\001') at omDebug.c:283
#6  0x00003fffa170be00 in omTestBinAddrSize (addr=0x3fffa13001f0, size=24, 
    check_level=1) at omDebug.c:46
#7  0x00003fffa1b64c14 in _p_Test (p=0x3fffa13001f0, r=0x3fffa13344d0, level=0)
    at pDebug.cc:223
#8  0x00003fffa1c1fcd0 in naInitChar (cf=0x3fffa131c9a8, 
    infoStruct=0x3fffffff9240) at ext_fields/algext.cc:1419
#9  0x00003fffa1c5b114 in nInitChar (t=n_algExt, parameter=0x3fffffff9240)
    at numbers.cc:385
#10 0x00003fffa0f81758 in __pyx_f_4sage_4libs_8singular_4ring_singular_ring_new
...
}

[jpflori]

And of the second one:

(gdb) bt
#0  dPolyReportError (p=0x3fffa13001f0, r=0x3fffa13344d0,
    fmt=0x3fffa1cb5140 "%s ") at pDebug.cc:46
#1  0x00003fffa1b64c54 in _p_Test (p=0x3fffa13001f0, r=0x3fffa13344d0, level=0)
    at pDebug.cc:223
#2  0x00003fffa1c1fcd0 in naInitChar (cf=0x3fffa131c9a8,
    infoStruct=0x3fffffff9240) at ext_fields/algext.cc:1419
#3  0x00003fffa1c5b114 in nInitChar (t=n_algExt, parameter=0x3fffffff9240)
    at numbers.cc:385
#4  0x00003fffa0f81758 in __pyx_f_4sage_4libs_8singular_4ring_singular_ring_new
...
}

[jpflori]

Some pointers, I have to quit now and tomorrow I'll have forgotten everything:

• ​https://github.com/hannes14/Singular/blob/master/HOWTO-libsingular
• ​https://github.com/wbhart/Nemo.jl/wiki/Accessing-Singular-from-Julia

[jpflori]

Got a suggestion from Hannes:

• ​https://groups.google.com/d/msg/libsingular-devel/4JRHxVXIF70/r4aXry32BQAJ

Additional issue:

• we should clean up old headers when upgrading.

[jpflori]

The above suggestion fixes the problem.

[jpflori]

I still get a bunch of crashes/errors when testing sage/libs/singular, working on it.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​bea5a27

Correctly initialize Singular polynomials.

[jakobkroeker]

I still get a bunch of crashes/errors when testing sage/libs/singular, working on it.

I was able to fix the issue with naSetMap() and nlInit2gmp() usage in sa2si_NF()) and this reduces the number of segfaults significantly The example

sage: x = var('x')
sage: K.<z> = QQ.extension(x^2 + 1)
sage: P.<v,w> = K[]
sage: P._singular_() 
sage: P(2/3) 

does not crash any more!

A remaining task is to reuse the plain polynomial ring qqr = QQ['x'] over Q in sa2si_NF() instead of rebuilding it everytime. (variable name is irrelevant)

. . Next interesting riddle in sa2si_ZZmod(); simple failing example:

sage: sage: R.<a> = Zmod(5)['a', 'b']  
sage: R(1) #crash

---

New commits:

​e6a30ff

fixed issue with nMapFuncPtr- usage

[jpflori]

New commits:

​71d1873	Merge remote-tracking branch 'trac/develop' into singular
​d252948	Merge remote-tracking branch 'trac/develop' into singular

[jpflori]

The segfault is due to a division by zero (happening in gmp which aborts I guess):

#0  0x00003fffadbeaf28 in __gmpz_mod (rem=0x3fffa03ce230,
    dividend=0x3fffa03ce220, divisor=0x0) at mod.c:29
#1  0x00003fffa0d1558c in nrnMapGMP (from=0x3fffa03ce220, dst=0x3fffa03bc9a8)
    at rmodulon.cc:766
#2  0x00003fffa0d1561c in nrnMapZ (from=0x3fffa03ce220, src=0x3fffa03bc680,
    dst=0x3fffa03bc9a8) at rmodulon.cc:790
#3  0x00003fffa014903c in __pyx_f_4sage_4libs_8singular_8singular_sa2si_ZZmod (
    __pyx_v_d=0x3fff986212e0, __pyx_v__ring=0x3fffa03d43a8)

I guess the ring is wrongly initialized.

[jpflori]

Note that the same thing works when 5 is replaced by anything non-prime. And indeed during ring creation the former is caught at the beginning of the method in sage/libs/singular/ring.pyx as field, finite and prime whereas the latter is caught as integer_mod_ring.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​92304e8

Properly initial prime fields in singular.

[jdemeyer]

Why does this branch change build/bin/sage-spkg???

[jpflori]

No idea, mistake from me surely...

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​97c8bb6

Revert "Properly initial prime fields in singular."

[jpflori]

Ok the issue is indeed a mix between Zmod and GF. The ring was initially created in singular as a "finite field" but then for some reason the conversion is made as for a "modular integer". The former one uses ints whereas the second one uses mpzs.

[jpflori]

I've not checked but I guess the reason is as follow:

• to create the singular ring we ask if the base ring is a prime finite field, not if it's a class derived from FiniteField, so classes derived from IntegerModRing are caught,
• when sending something into the ring we follow the "classes" logic, boom.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​18a6be1

Fix for the GF vs Zmod issue with Singular.

[jpflori]

Ok next issue: napoly and slnumber do not exist anymore, seemingly since a long time...

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​bf2aede

WIP to use new Singular API.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​2ecb686

Fix several doctest failures in libs/singular

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​e949c00

Fix error message formatting

[jpflori]

New commits:

​15d5f12	Fix usage of algebraic extension as base rings in libsingular.
​d3566cb	Merge remote-tracking branch 'trac/u/jdemeyer/ticket/17254' into singular

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​8c02754

Use new Singular overflow limits.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​cab5fc5

groebner_norm Singular function does not exist, groebner does.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​35b9919

Fix some order changes and normalization changes in Singular output.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​ee62dcd

Yet another (doubtful) different normalization (-1 == 2 mod 3).

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​31c98df

New uniform overflow limit for Singular exponents.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​d7a3f36

singclap_isSqrFree was removed from Singular 4.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​b610f9d

Another overflow in Singular related change.

[jdemeyer]

New commits:

​d7fd9ba

Better overflow check

[jpflori]

Strange way to deal with floats accuracy, see: ​https://groups.google.com/forum/#!topic/libsingular-devel/kj3KlcrcNo8

Now looking into the weighted degree issue.

---

New commits:

​bf9ec9d	Change in doctest now that Integers(2)[x, y, ...] uses Singular's Zn type.
​a0a71c9	Merge remote-tracking branch 'trac/u/jdemeyer/ticket/17254' into singular
​a5b45a3	Yet others overflow related changes.

[jpflori]

Note for myself: the issue is that at some point r->pFDeg gets replaced from p_WTotalDegree by p_TotalDegree. Don't know why yet.

[jpflori]

This is triggered here:

  if ((r->pFDeg==p_WTotaldegree) && rOrd_is_MixedDegree_Ordering(r))
    r->pFDeg = p_Totaldegree;

line 3837/8 of ring.cc.

[jpflori]

Made by:

• ​https://github.com/Singular/Sources/commit/d711b46eb29f241f3854ae99e8451d3beb7f96e4

Maybe the 0 in the weights makes think Singular the order is useless.

[jpflori]

Question to upstream here:

• ​https://groups.google.com/forum/?fromgroups=#!topic/libsingular-devel/jregxl-k8Mw

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​199169e

Forgotten overflow stuff + use user defined degree.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​69ea9a4

Order changed in Singular output.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7b5bb32

Use correct function for degree.

[jakobkroeker]

Replying to git:

Branch pushed to git repo; I updated commit sha1. New commits:

​7b5bb32

Use correct function for degree.

I get the following errors for the last change:

Error compiling Cython file:
------------------------------------------------------------
...
    deg = 0
    if p == NULL:
        return -1
    if(r != currRing): rChangeCurrRing(r)
    if x == NULL:
       return r.pDeg(p,&deg,r)
              ^
------------------------------------------------------------

sage/libs/singular/polynomial.pyx:534:15: Object of type 'ring' has no attribute 'pDeg'

Error compiling Cython file:
------------------------------------------------------------
...
    deg = 0
    if p == NULL:
        return -1
    if(r != currRing): rChangeCurrRing(r)
    if x == NULL:
       return r.pDeg(p,&deg,r)
                      ^
------------------------------------------------------------

sage/libs/singular/polynomial.pyx:534:23: Cannot convert 'poly *' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
    deg = 0
    if p == NULL:
        return -1
    if(r != currRing): rChangeCurrRing(r)
    if x == NULL:
       return r.pDeg(p,&deg,r)
                       ^
------------------------------------------------------------

sage/libs/singular/polynomial.pyx:534:24: Cannot convert 'int *' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
    deg = 0
    if p == NULL:
        return -1
    if(r != currRing): rChangeCurrRing(r)
    if x == NULL:
       return r.pDeg(p,&deg,r)
                             ^

[jpflori]

Yes I guess I made a typo and forgot to push the latest commit.

[jpflori]

Or forgot to add the function prototype.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​5c3c320

Add declaration for Singular pDeg.

[jpflori]

Hopefully better now although I did not test the fix.

From what I remember the next issue was a segfault when letterplace stuff elements were multiplied.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​2b76279

Correct fix for the degree in singular.

[jpflori]

Problem with letterplace algebra:

sage: F.<x> = FreeAlgebra(ZZ, implementation='letterplace')
sage: x**2
sage: x**2
sage: x**2

[jpflori]

I seem to remember there was a double free.

[jpflori]

Did you manage to build the branch and reproduce the letterplace error?

[tscrim]

I rebased the branch (as best as I could), and I did get an error with letterplace:

sage: F.<x> = FreeAlgebra(ZZ, implementation='letterplace')
sage: x**2
0

I am also getting a variety of failures in testing the rings/polynomial folder, but most of them seem to be related to constructing ideals. I haven't run tests elsewhere.

---

New commits:

​ef0b4e0

Rebase #17254 on 7.1.rc0.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​dca98ec

Fixing an issue with my rebase.

[jakobkroeker]

Replying to jpflori:

Problem with letterplace algebra:

sage: F.<x> = FreeAlgebra(ZZ, implementation='letterplace')
sage: x**2
sage: x**2
sage: x**2

when I compile the recent 'u/jpflori/ticket/17254' or 'u/tscrim/upgrade_singular-17254' with SAGE_DEBUG=yes on my 32 bit fedora22

  export SAGE_DEBUG=yes
  ./sage -f singular

I get an printout from Singular:

  pmLPshift: too big shift requested

the corresponding Singular source snippet is (look at 'shiftgb.cc')

 if (L+sh-1 > uptodeg)
  {
#ifdef PDEBUG
    PrintS("pmLPshift: too big shift requested\n");
#endif
    return(NULL); /* violation, 2check */
  }

This seems to be a bug, but the next question is, why do we get there and is this an error caused by a previous one or not.

[jakobkroeker]

remaining failing tests (probably not all related to Singular):

./sage -t src/sage/schemes/affine/affine_rational_point.py  # Killed due to abort
./sage -t src/sage/schemes/affine/affine_point.py  # Killed due to segmentation fault
./sage -t src/sage/schemes/affine/affine_morphism.py  # Killed due to segmentation fault
./sage -t src/sage/schemes/generic/algebraic_scheme.py  # Killed due to segmentation fault
./sage -t src/sage/schemes/elliptic_curves/isogeny_small_degree.py  # Killed due to abort
./sage -t src/sage/libs/cremona/newforms.pyx  # IOError in doctesting framework
./sage -t src/sage/algebras/letterplace/free_algebra_letterplace.pyx  # Bad exit: 14
./sage -t src/sage/algebras/letterplace/free_algebra_element_letterplace.pyx  # Timed out (and interrupt failed)
./sage -t src/sage/rings/polynomial/multi_polynomial_sequence.py  # Killed due to segmentation fault
./sage -t src/sage/rings/polynomial/multi_polynomial_ideal.py  # Killed due to segmentation fault
./sage -t src/sage/rings/polynomial/pbori.pyx  # 7 doctests failed
./sage -t src/sage/schemes/projective/projective_morphism.py  # 9 doctests failed
./sage -t src/sage/rings/polynomial/polynomial_element.pyx  # 1 doctest failed
./sage -t src/sage/rings/polynomial/plural.pyx  # 6 doctests failed
./sage -t src/sage/schemes/jacobians/abstract_jacobian.py  # 1 doctest failed
./sage -t src/sage/schemes/plane_curves/projective_curve.py  # 1 doctest failed
./sage -t src/sage/rings/polynomial/polynomial_singular_interface.py  # 1 doctest failed
./sage -t src/sage/rings/number_field/number_field_ideal.py  # 2 doctests failed
./sage -t src/sage/modular/modform_hecketriangle/abstract_space.py  # 1 doctest failed
./sage -t src/sage/rings/polynomial/infinite_polynomial_element.py  # 4 doctests failed
./sage -t src/sage/rings/finite_rings/finite_field_ext_pari.py  # 1 doctest failed
./sage -t src/doc/en/constructions/algebraic_geometry.rst  # 1 doctest failed
./sage -t src/sage/graphs/generic_graph.py  # 1 doctest failed
./sage -t src/sage/modular/modform_hecketriangle/graded_ring_element.py  # 7 doctests failed
./sage -t src/sage/rings/finite_rings/finite_field_givaro.py  # 1 doctest failed
./sage -t src/sage/algebras/free_algebra.py  # 12 doctests failed
./sage -t src/sage/rings/multi_power_series_ring_element.py  # 5 doctests failed
./sage -t src/sage/rings/polynomial/infinite_polynomial_ring.py  # 1 doctest failed
./sage -t src/sage/graphs/base/graph_backends.pyx  # 4 doctests failed
./sage -t src/sage/symbolic/relation.py  # 7 doctests failed
./sage -t src/sage/interfaces/singular.py  # 5 doctests failed
./sage -t src/sage/rings/quotient_ring_element.py  # 1 doctest failed
./sage -t src/sage/structure/element.pyx  # 1 doctest failed
./sage -t src/sage/schemes/product_projective/wehlerK3.py  # 17 doctests failed
./sage -t src/sage/categories/pushout.py  # 1 doctest failed
./sage -t src/sage/rings/quotient_ring.py  # 12 doctests failed
./sage -t src/sage/structure/sage_object.pyx  # 1 doctest failed

[jakobkroeker]

@jpflori

when using 'r.pLDeg()' instead of 'p_Deg()' in 'singular_polynomial_deg()' (src/sage/libs/singular/polynomial.pyx)

some test from above pass, for example:

./sage -t src/sage/schemes/affine/affine_point.py 
./sage -t src/sage/schemes/affine/affine_morphism.py 
./sage -t src/sage/rings/polynomial/pbori.pyx  
./sage -t  src/sage/rings/polynomial/infinite_polynomial_element.py

especially some segfaults disappear but then probably other passing tests get broken?

Which example did you have in mind when you changed 'r.pLDeg()' to 'p_Deg()' in 'singular_polynomial_deg()' (src/sage/libs/singular/polynomial.pyx)?

Maybe we should discuss with Hans his commit in case it breaks something

​https://groups.google.com/forum/#!topic/libsingular-devel/jregxl-k8Mw

[jpflori]

The plDeg function is a Singular internal function, made for ecart, and has some restrictions regarding the ordering it can be applied to (no negative weights or things like that). It gets constructed by Singular at initialization of the object from p_Deg.

What we want in Sage is the "usual" degree which is p_Deg.

Maybe the doctests were just wrong, i have no idea.

[jdemeyer]

Which branch are you using? The branch on this ticket seems to have multiple conflicts with the current develop.

[jdemeyer]

The conflicts seem to be because of #20386.

[jakobkroeker]

fixed some more issues.

A rebase on recent develop branch was too hard for me so I squashed everything in a new commit.

@jdemeyer I had to revert some of your changes in src/module_list.py (removing singular from include path) because I failed to do the same for Singular 4.

[jakobkroeker]

actually, in my recent branch the 'upgrade.patch' file is missing. No idea how that happened, but this is a fact.

I will fix that tomorrow or on friday. At the moment I'm just too tired.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​3312a17

upgrade.patch was not staged, staging it

[jdemeyer]

Replying to jakobkroeker:

A rebase on recent develop branch was too hard for me so I squashed everything in a new commit.

@jdemeyer I had to revert some of your changes in src/module_list.py

Both of these are quite bad. Do you want me to have a look?

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​cc868a0	rebase too hard for me. squashed everything again
​ff98a0b	upgrade.patch was not staged, staging it

[jakobkroeker]

Replying to jdemeyer:

Replying to jakobkroeker:

A rebase on recent develop branch was too hard for me so I squashed everything in a new commit.

@jdemeyer I had to revert some of your changes in src/module_list.py

Both of these are quite bad. Do you want me to have a look?

That would be good.

The latest u/jakobkroeker/ticket.17254.squashed.latest (just committed with force) should have no conflicts with develop/master branch

[jakobkroeker]

Question : why trac shows me that trac's automerging failed?

[jakobkroeker]

I experience several strange doctest failures, for example:

File "src/sage/modular/modform_hecketriangle/abstract_space.py", line 379, in sage.modular.modform_hecketriangle.abstract_space.FormsSpace_abstract.one_element
Failed example:
    MF.one_element()
Expected:
    doctest:...: DeprecationWarning: .one_element() is deprecated. Use .one() instead.
    See http://trac.sagemath.org/17694 for details.
    1 + O(q^5)
Got:
    doctest:1: DeprecationWarning: .one_element() is deprecated. Use .one() instead.
    See http://trac.sagemath.org/17694 for details.
      Running doctests with ID 2016-05-15-20-09-37-2e7003e6.
    1 + O(q^5)
*******************

any idea what is wrong here or how to fix it?

make distclean and ccache --clear did not help

[tscrim]

Is it based off develop or master? These are different as develop is the (released today) 7.3.beta2, whereas master is 7.2.

[tscrim]

I've seen some errors somewhat similar to that before. It is as if something in the doctesting output gets read with an offset to what it should. I forget exactly what I've done in the past to fix this, but I think it came from a specific type of doctest failure...

[jakobkroeker]

Replying to tscrim:

I've seen some errors somewhat similar to that before. It is as if something in the doctesting output gets read with an offset to what it should. I forget exactly what I've done in the past to fix this, but I think it came from a specific type of doctest failure...

are you able to reproduce some of these (false-positive) doctest failures?

[jakobkroeker]

Replying to tscrim:

Is it based off develop or master? These are different as develop is the (released today) 7.3.beta2, whereas master is 7.2.

Based on develop (​https://github.com/sagemath/sage) on my machine it merges without conflicts with master and develop branch.

[tscrim]

Replying to jakobkroeker:

Replying to tscrim:

Is it based off develop or master? These are different as develop is the (released today) 7.3.beta2, whereas master is 7.2.

Based on develop (​https://github.com/sagemath/sage) on my machine it merges without conflicts with master and develop branch.

The automerging that is used on trac is not as good as what local versions of git can do.

As for comment:170, I haven't testing this branch yet.

[jdemeyer]

There was a merge conflict with Sage 7.3.beta2, I fixed it.

---

New commits:

​a80b3a0

Merge tag '7.3.beta2' into t/17254/ticket.17254.squashed.latest

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​e8fbe03

Upgrade to Singular-4.0.2

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​5b8343b

Upgrade to Singular-4.0.2

[jdemeyer]

I cleaned up the patch and removed some unacceptable parts. Right now it doesn't build, that still needs to be fixed.

[jdemeyer]

I think it's best to discuss with upstream about the crazy #include directory structure. It was already bad in Singular 3 and it seems to have gotten worse with Singular 4.

[jakobkroeker]

Replying to jdemeyer:

I cleaned up the patch and removed some unacceptable parts. Right now it doesn't build, that still needs to be fixed.

what has to be done to get it compiling?

[jdemeyer]

I am working on a hack to fix the includes, hang on...

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​1426ad8

Fix singular include directories

[jakobkroeker]

Replying to jdemeyer:

I cleaned up the patch and removed some unacceptable parts. Right now it doesn't build, that still needs to be fixed.

Are these the unacceptable parts? (e.g in src/sage/rings/polynomial/polydict.pyx)

diff --git a/src/sage/rings/polynomial/polydict.pyx b/src/sage/rings/polynomial/polydict.pyx
index fd858f7..0b49f38 100644
--- a/src/sage/rings/polynomial/polydict.pyx
+++ b/src/sage/rings/polynomial/polydict.pyx
@@ -506,7 +506,7 @@ cdef class PolyDict:
         don't put negative signs on the generators. ::
 
             sage: Integers(2)['x,y'].gens()
-            (1*x, 1*y)
+            (x, y)
 
         We make sure that intervals are correctly represented. ::
 

The 1*x output stems from Singular4 and that seems the new behaviour for that ring, for whatever reasons.

same for

diff --git a/src/sage/rings/multi_power_series_ring_element.py b/src/sage/rings/multi_power_series_ring_element.py
index 7293a91..7b9ca6e 100644
--- a/src/sage/rings/multi_power_series_ring_element.py
+++ b/src/sage/rings/multi_power_series_ring_element.py
@@ -32,7 +32,7 @@ Power series arithmetic, tracking precision::
     sage: f*=s; f
     s + 2*s^2 + 3*s^3 + O(s, t)^8
     sage: f%2
-    1*s + 1*s^3 + O(1*s, 1*t)^8
+    s + s^3 + O(s, t)^8
     sage: (f%2).parent()
     Multivariate Power Series Ring in s, t over Ring of integers modulo 2
 
@@ -1078,7 +1078,7 @@ class MPowerSeries(PowerSeries):
             sage: R.<a,b,c> = PowerSeriesRing(ZZ)
             sage: f = -a^3*b*c^2 + a^2*b^2*c^4 - 12*a^3*b^3*c^3 + R.O(10)
             sage: g = f % 2; g
-            1*a^3*b*c^2 + 1*a^2*b^2*c^4 + O(1*a, 1*b, 1*c)^10
+            a^3*b*c^2 + a^2*b^2*c^4 + O(a, b, c)^10
             sage: g in R
             False
             sage: g in R.base_extend(Zmod(2))

also floats are now put in braces, maybe because otherwise parsing the string '1.7E+30*5' in Singular would be wronng or fail? (just guessing)

so removing the following hack

--- a/src/sage/rings/real_double.pyx
+++ b/src/sage/rings/real_double.pyx
@@ -689,21 +689,6 @@ cdef class RealDoubleElement(FieldElement):
             sage: RDF(10^100)
             1e+100
         """
-        if isinstance(x,basestring):
-            # a hack for parsing float strings beginning with braces
-            if len(x)>2 and x.count('(')==1 and  x.count(')')==1:
-                try:
-                    if (x[0] == '(' or x[1] == '(') and x[-1] == ')':
-                        if (x[0] == '-' and x[1] == '(') :
-                            x = x[2:-1]
-                            self._value = - float(x)
-                            return
-                        else:
-                            x = x[1:-1]
-                            self._value = float(x)
-                            return
-                except:
-                    pass
         self._value = float(x)
 
     def _magma_init_(self, magma):

will result in a lot of failing doctests...

[jdemeyer]

It's not because Singular prints something in a certain way that Sage has to print it that way. The correct fix is changing the way how Sage prints Singular objects then.

[jdemeyer]

I added a fix for the include file issue.

[jakobkroeker]

Replying to jdemeyer:

I added a fix for the include file issue.

Thanks!

[jakobkroeker]

Replying to jdemeyer:

It's not because Singular prints something in a certain way that Sage has to print it that way. The correct fix is changing the way how Sage prints Singular objects then.

At least we should understand the reasons

• for 1*x
• and for braces around floats

I do not know whether it is safe to remove the 1* and the braces around floats. I don't know what Sage does for example with _repr_(), (which calls singular_polynomial_str() which in turn calls p_String

And if it' safe, how we should do that? regexping each polynomial string on the fly?

[jdemeyer]

Replying to jakobkroeker:

regexping each polynomial string on the fly?

If that works, why not?

[jakobkroeker]

remaining failing tests:

./sage -t --long src/sage/arith/misc.py  # 1 doctest failed
./sage -t --long src/sage/modular/modform_hecketriangle/graded_ring_element.py  # 7 doctests failed
./sage -t --long src/sage/interfaces/singular.py  # 1 doctest failed
./sage -t --long src/sage/categories/pushout.py  # 1 doctest failed
./sage -t --long src/sage/quivers/algebra_elements.pyx  # Timed out (and interrupt failed)
./sage -t --long src/sage/algebras/free_algebra.py  # 12 doctests failed
./sage -t --long src/sage/algebras/letterplace/free_algebra_letterplace.pyx  # Bad exit: 14
./sage -t --long src/sage/algebras/letterplace/letterplace_ideal.pyx  # Killed due to abort
./sage -t --long src/sage/algebras/letterplace/free_algebra_element_letterplace.pyx  # Killed due to segmentation fault
./sage -t --long src/sage/rings/quotient_ring_element.py  # 1 doctest failed
./sage -t --long src/sage/rings/ring.pyx  # Killed due to abort
./sage -t --long src/sage/rings/quotient_ring.py  # 12 doctests failed
./sage -t --long src/sage/rings/multi_power_series_ring_element.py  # 3 doctests failed
./sage -t --long src/sage/rings/polynomial/infinite_polynomial_ring.py  # 1 doctest failed
./sage -t --long src/sage/rings/polynomial/multi_polynomial_sequence.py  # Killed due to segmentation fault
./sage -t --long src/sage/rings/polynomial/plural.pyx  # 6 doctests failed
./sage -t --long src/sage/rings/polynomial/multi_polynomial_libsingular.pyx  # 2 doctests failed
./sage -t --long src/sage/rings/polynomial/polynomial_singular_interface.py  # 1 doctest failed
./sage -t --long src/sage/rings/polynomial/symmetric_ideal.py  # 12 doctests failed
./sage -t --long src/sage/rings/polynomial/infinite_polynomial_element.py  # 4 doctests failed
./sage -t --long src/sage/rings/polynomial/polydict.pyx  # 1 doctest failed
./sage -t --long src/sage/rings/polynomial/multi_polynomial_ideal.py  # Killed due to segmentation fault
./sage -t --long src/sage/schemes/projective/projective_morphism.py  # Killed due to segmentation fault
./sage -t --long src/sage/schemes/plane_curves/projective_curve.py  # 1 doctest failed
./sage -t --long src/sage/schemes/affine/affine_rational_point.py  # Killed due to abort
./sage -t --long src/sage/schemes/affine/affine_point.py  # Killed due to segmentation fault
./sage -t --long src/sage/schemes/affine/affine_morphism.py  # Killed due to segmentation fault
./sage -t --long src/sage/schemes/jacobians/abstract_jacobian.py  # 1 doctest failed
./sage -t --long src/sage/schemes/generic/algebraic_scheme.py  # Killed due to segmentation fault
./sage -t --long src/sage/schemes/elliptic_curves/isogeny_small_degree.py  # Timed out (with hangup after interrupt)
./sage -t --long src/sage/structure/element.pyx  # 1 doctest failed
./sage -t --long src/doc/en/constructions/algebraic_geometry.rst  # 1 doctest failed

any volunteers?

[jpflori]

Do you still get the letterplace segfault?

[jakobkroeker]

Replying to jpflori:

Do you still get the letterplace segfault?

I did not fix that one because I have no idea what FreeAlgebra? is neither I know something about letterplace implementation, so I did not even try.

[jakobkroeker]

New commits:

​df297d6

add meaninful messages to Singular error

remark: the patch above does not solve the issue with letterplace segfault

[jakobkroeker]

simplest failing example:

sage: A.<x> = AffineSpace(GF(3^2, 't'), 1) ## line 299 ##
Ideal(x).weil_restriction()

[jakobkroeker]

added debug printouts in

u/jakobkroeker/jdemeyer_ticket.17254.latest.debug

for the example

sage: F.<x> = FreeAlgebra(ZZ, implementation='letterplace')
sage: x**2

in free_algebra_element_letterplace.pyx, __pow__() routine after copying the polynomial from one ring to another (with prCopyR) something seems broken (look at the polynomial degree!)

and for the

sage: A.<x> = AffineSpace(GF(3^2, 't'), 1) ## line 299 ##
Ideal(x).weil_restriction()

after the subs() command in the weil_restriction()-Routine something is screwed up (look at the polynomial degree!)

@jpflori Any idea?

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​88ce71f

call rComplete for singular ring

[jakobkroeker]

p_Deg seems to have problems with lex ordering:

k = QQ
S.<a,b>=PolynomialRing(k, 2, {"a","b"}, order='lex')
a.degree() # 65536

k = QQ
S.<a,b>=PolynomialRing(k, 2, {"a","b"}, order='degrevlex')
a.degree() #1

[jakobkroeker]

@jpflori

Singulars degree function calls pLDeg and not p_Deg, (grep for 'jjDEG' in Singular sources), so we should do the same/similar by default. one of the reasons: p_Deg seems returning garbage for lexicographic orderings (see comment 195)

Question: is the behaviour described in the documentation of the 'degree' function in sage

Note that a matrix term ordering alters the grading of the generators
          of the ring; see the tests below.  To avoid this behavior, use either
          ``exponents()`` for the exponents themselves, or the optional
          argument ``std_grading=False``.

natural and is it what a user would expect? Then we could(?) fix it using a different appropriate call:

sage: sage: tord = TermOrder(matrix([3,0,1,1,1,0,1,0,0]))
sage: sage: R.<x,y,z> = PolynomialRing(QQ,'x',3,order=tord)
sage: sage: (x^3*y+x*z^4).degree() # 9
('p_Totaldegree', 4)
('p_WTotaldegree', 9)
('p_WDegree', 9)
('pFDeg', 4)
('pLDeg', 5)

[john_perry]

Replying to jakobkroeker:

Question: is the behaviour described in the documentation of the 'degree' function in sage

Note that a matrix term ordering alters the grading of the generators
          of the ring; see the tests below.  To avoid this behavior, use either
          ``exponents()`` for the exponents themselves, or the optional
          argument ``std_grading=False``.

natural and is it what a user would expect?

I bumped into this a long time ago. It shocked me then, but I adapted & now my code depends on it. If this is true of me, it is probably true of others.

This, for example, is from Sage 6.7:

sage: T = TermOrder(matrix(3,3,[1,2,3,3,4,5,0,0,1]))
sage: R.<x,y,z> = PolynomialRing(QQ,'x',3,order=T)
sage: y.degree()
2
sage: version()
'SageMath Version 6.7, Release Date: 2015-05-17'

So IMHO we should mimic Singular for compatibility reasons (i.e., we're mimicking Sage itself).

[tscrim]

Rebased on latest beta.

---

New commits:

​9ec5134

Merge branch 'u/jakobkroeker/ticket.17254.Jeroen_Demeyer.version' of git://trac.sagemath.org/sage into u/jakobkroeker/ticket.17254.Jeroen_Demeyer.version

[klee]

Replying to jakobkroeker:

Question: is the behaviour described in the documentation of the 'degree' function in sage

Note that a matrix term ordering alters the grading of the generators
          of the ring; see the tests below.  To avoid this behavior, use either
          ``exponents()`` for the exponents themselves, or the optional
          argument ``std_grading=False``.

natural and is it what a user would expect? Then we could(?) fix it using a different appropriate call:

sage: sage: tord = TermOrder(matrix([3,0,1,1,1,0,1,0,0]))
sage: sage: R.<x,y,z> = PolynomialRing(QQ,'x',3,order=tord)
sage: sage: (x^3*y+x*z^4).degree() # 9
('p_Totaldegree', 4)
('p_WTotaldegree', 9)
('p_WDegree', 9)
('pFDeg', 4)
('pLDeg', 5)

As the person who introduced the matrix term ordering into Sage, I think that it was just due to the Singular behavior that a matrix term ordering changes the grading of the generators according to the first row of the matrix in Sage. It shocked everyone including John, and was regarded as a bug. So if we can make the grading and the matrix term ordering independent, switching to Singular 4, then I would be happy. Of course it should go with an appropriate deprecation warning.

[john_perry]

Replying to klee:

Of course it should go with an appropriate deprecation warning.

+1

[mkoeppe]

Polymake (#20892) looks for a binary called libsingular-config. Is this something that will be installed by Singular 4?

[jakobkroeker]

Replying to mkoeppe:

Polymake (#20892) looks for a binary called libsingular-config. Is this something that will be installed by Singular 4?

yes, libsingular-config is a bash script in Singular 4

[leif]

This will hopefully build with GCC 6.x... (cf. #20738) :P

[jakobkroeker]

Currently I implemented singular_polynomial_deg as follows to mimic its behaviour as it was with Singular 3.1.7(and older)

    ...
    cdef long _deg, deg
    
    deg = -1
    _deg = -1 
    if x == NULL:
        while p:  
            _deg = p_WTotaldegree(p,r)
          
            if _deg > deg:
                deg = _deg
            p = pNext(p)

        return deg
    ...

The degree-related tests seem to pass now.

Remakr: pDeg() seems not usable/has different behaviour; see ​https://groups.google.com/forum/?_escaped_fragment_=topic/libsingular-devel/dulzKFTF3g8#!topic/libsingular-devel/dulzKFTF3g8 for details.

Further changes: added some missing rComplete() calls ( otherwise prCopyR cannot be used, for instance, see ​https://groups.google.com/forum/?_escaped_fragment_=topic/libsingular-devel/4JRHxVXIF70#!topic/libsingular-devel/4JRHxVXIF70 )

---

New commits:

​b1e228e	replacement for pLDeg from Singular 3 (behaviour of pLDeg changed in Singular 4)
​a511817	added missing rComplete calls
​738fe43	disable short output for Singular

[jakobkroeker]

only few issues are left;

who will fix the next one? that should be easy!

./sage -t --long src/sage/arith/misc.py  # 1 doctest failed
./sage -t --long src/sage/modular/modform_hecketriangle/graded_ring_element.py  # 7 doctests failed
./sage -t --long src/sage/quivers/algebra_elements.pyx  # Timed out (and interrupt failed)
./sage -t --long src/sage/algebras/letterplace/free_algebra_letterplace.pyx  # Killed due to segmentation fault
./sage -t --long src/sage/algebras/letterplace/letterplace_ideal.pyx  # Killed due to abort
./sage -t --long src/sage/algebras/letterplace/free_algebra_element_letterplace.pyx  # Timed out (and interrupt failed)
./sage -t --long src/sage/rings/ring.pyx  # Killed due to abort
./sage -t --long src/sage/rings/multi_power_series_ring_element.py  # 2 doctests failed
./sage -t --long src/sage/rings/polynomial/multi_polynomial_libsingular.pyx  # 2 doctests failed
./sage -t --long src/sage/rings/polynomial/polynomial_singular_interface.py  # 1 doctest failed
./sage -t --long src/sage/rings/polynomial/polydict.pyx  # 1 doctest failed
./sage -t --long src/sage/rings/polynomial/multi_polynomial_ideal.py  # 1 doctest failed
./sage -t --long src/sage/schemes/projective/projective_morphism.py  # Timed out (with hangup after interrupt)
./sage -t --long src/sage/schemes/affine/affine_rational_point.py  # Killed due to abort
./sage -t --long src/sage/schemes/jacobians/abstract_jacobian.py  # 1 doctest failed
./sage -t --long src/sage/schemes/elliptic_curves/isogeny_small_degree.py  # Killed due to segmentation fault
./sage -t --long src/sage/structure/element.pyx  # 1 doctest failed
./sage -t --long src/doc/en/constructions/algebraic_geometry.rst  # 1 doctest failed

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​3cb4304	explained si_opt_1 and si_opt_2
​ac8e614	fixed an upstream bug in brnoeth.lib and thus the failing test in src/sage/schemes/curves/projective_curve.py. TODO: upstream report/ upstream pull request

[jakobkroeker]

Failing of

sage: F.<x> = FreeAlgebra(ZZ, implementation='letterplace')
sage: x**2

is caused by an upstream bug; (reported: ​https://www.singular.uni-kl.de:8005/trac/ticket/767#ticket)

Letterplace over fields should work (please check)

[jpflori]

Replying to jakobkroeker:

Failing of

sage: F.<x> = FreeAlgebra(ZZ, implementation='letterplace')
sage: x**2

is caused by an upstream bug; (reported: ​https://www.singular.uni-kl.de:8005/trac/ticket/767#ticket)

Letterplace over fields should work (please check)

Thanks for devising this!

[jakobkroeker]

the upsream issue

pathed in ​http://git.sagemath.org/sage.git/commit/?id=ac8e614777f6e5e49149f6227d40ba1cc71585c9 (upgrade.patch)

is now reported on upstream with a pull request:

​https://www.singular.uni-kl.de:8005/trac/ticket/770#ticket

[jakobkroeker]

could someone please look what goes wrong with

./sage -t --long src/sage/modular/modform_hecketriangle/graded_ring_element.py

?

[jpflori]

I've removed a bunch of very old hackish stuff in spkg-install, especially regarding Solaris and sparc. Maybe too much, but the Singular build system changed so much and it does not seem we have real support for such systems anymore that we should only reintroduce removed hacks if found that are still needed.

It also seems this upgrade to git version rebroke some stuff...

---

New commits:

​637b2b7	Merge remote-tracking branch 'trac/u/jakobkroeker/tscrim.upgrade_singular-17254' into singular
​be4f5e7	Update Singular to 4.0.3p1.
​5b98496	Let's play with singular git version.
​3a5baec	Work with Singular git version.
​7e99c41	Use Singular git version.

[jpflori]

In particular, in function.pyx, in __init__ after calling

        if currRingHdl == NULL:
            currRingHdl = ggetid("my_awesome_sage_ring")
            if currRingHdl == NULL:
                currRingHdl = enterid("my_awesome_sage_ring", 0, RING_CMD, &IDROOT, 1)
            currRingHdl.data.uring.ref += 1

the last line segfaults because:

(gdb) print currRingHdl.data
$2 = {
  i = 0, 
  uring = 0x0, 
  p = 0x0, 
  n = 0x0, 
  uideal = 0x0, 
  umap = 0x0, 
  umatrix = 0x0, 
  ustring = 0x0, 
  iv = 0x0, 
  bim = 0x0, 
  l = 0x0, 
  li = 0x0, 
  pack = 0x0, 
  pinf = 0x0
}

I guess some additional funciton should be used to initialize properly currRingHdl.

[jpflori]

The culprit might be ​https://github.com/Singular/Sources/commit/595145c77dbdb42114960862678d5ca6351578be

[leif]

So everyone's always using the latest git version in spkg-src?

FWIW, sed -i isn't portable, e.g. won't work on Oraclis.

SPKG.txt now says "Patches: None" whereas there is upgrade.patch.

[jpflori]

That's just work in progress... This upgrade is hell.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​4a85552

enterid does not initialize underlying ring anymore, fix for function.pyx.

[leif]

Replying to jpflori:

That's just work in progress... This upgrade is hell.

Sure. So you'll (by default) check out a specific commit once you know which one is suited best?

[leif]

Just for the record:

make_tar.sh checks out HEAD, and calls autogen.sh (without checking for errors) which in turn runs (some random version of) autoreconf.

[jpflori]

FYI, explanation for the removal and readdition of nlDelete: ​https://groups.google.com/d/msg/libsingular-devel/EtEblVqdV3I/CyJJ5jaJCAAJ

[jpflori]

Replying to leif:

Replying to jpflori:

That's just work in progress... This upgrade is hell.

Sure. So you'll (by default) check out a specific commit once you know which one is suited best?

Sure! Hopefully it should be 4.1 upstream version though I don't know when it's planned. 4.0.3p1 seems to be a poorer choice as a bunch of bugs reported here where fixed in Singular after that release and the fixes would have to be backported...

[jpflori]

Replying to leif:

Just for the record:

make_tar.sh checks out HEAD, and calls autogen.sh (without checking for errors) which in turn runs (some random version of) autoreconf.

Yes and tries to copy some files from the person who wrote the script :)

[leif]

Replying to jpflori:

Replying to leif:

So you'll (by default) check out a specific commit once you know which one is suited best?

Sure! Hopefully it should be 4.1 upstream version though I don't know when it's planned.

I was just about to ask you that, as there are many recent commits mentioning 4.1.

I'll probably add something to spkg-src regarding checkout and autogen.sh / make_tar.sh unless you're going to as well.

[leif]

Replying to jpflori:

Replying to leif:

make_tar.sh checks out HEAD, and calls autogen.sh (without checking for errors) which in turn runs (some random version of) autoreconf.

Yes and tries to copy some files from the person who wrote the script :)

I just stumbled upon

cp /tmp/wawa-i/share/singular/LIB/all.lib  singular-$VERSION/Singular/LIB/.

Did you mean that, or is there more?

[leif]

After all, it's spielwiese (~= playground).

Is the repo on GitHub their "master" one, or do they (occasionally) just sync from elsewhere?

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​d0c7069

Update singular git version.

[jpflori]

Replying to leif:

Replying to jpflori:

Replying to leif:

make_tar.sh checks out HEAD, and calls autogen.sh (without checking for errors) which in turn runs (some random version of) autoreconf.

Yes and tries to copy some files from the person who wrote the script :)

I just stumbled upon

cp /tmp/wawa-i/share/singular/LIB/all.lib  singular-$VERSION/Singular/LIB/.

Did you mean that, or is there more?

There is also a non-existing fedora (IIRC) file.

[jpflori]

Replying to leif:

After all, it's spielwiese (~= playground).

Is the repo on GitHub their "master" one, or do they (occasionally) just sync from elsewhere?

I would say the spielwiese branch on github is the real master one. As you can see, the following commit: ​https://github.com/Singular/Sources/commit/b412eebf8435fb44680b5160c7f8d524cdb20b2b just followed my post on libsingular-devel: ​https://groups.google.com/forum/#!topic/libsingular-devel/EtEblVqdV3I

[jpflori]

Things seem good. A ptest yeilds:

sage -t --warn-long 121.7 src/doc/en/prep/Advanced-2DPlotting.rst  # Killed due 
to segmentation fault
sage -t --warn-long 121.7 src/sage/calculus/riemann.pyx  # Timed out (and interr
upt failed)
sage -t --warn-long 121.7 src/sage/schemes/elliptic_curves/isogeny_small_degree.
py  # Killed due to segmentation fault
sage -t --warn-long 121.7 src/sage/schemes/elliptic_curves/ell_number_field.py  
# Timed out
sage -t --warn-long 121.7 src/sage/libs/gap/all_documented_functions.py  # Timed
 out
sage -t --warn-long 121.7 src/sage/plot/plot.py  # Timed out
sage -t --warn-long 121.7 src/sage/libs/gap/assigned_names.py  # Timed out
sage -t --warn-long 121.7 src/sage/finance/time_series.pyx  # Timed out
sage -t --warn-long 121.7 src/sage/plot/plot3d/implicit_plot3d.py  # Timed out
sage -t --warn-long 121.7 src/sage/plot/plot3d/parametric_plot3d.py  # Timed out
sage -t --warn-long 121.7 src/sage/plot/plot3d/plot3d.py  # Timed out
sage -t --warn-long 121.7 src/sage/plot/graphics.py  # Timed out
sage -t --warn-long 121.7 src/sage/plot/plot3d/base.pyx  # Timed out
sage -t --warn-long 121.7 src/doc/en/prep/Calculus.rst  # Killed due to segmentation fault
sage -t --warn-long 121.7 src/sage/modular/local_comp/type_space.py  # Timed out
sage -t --warn-long 121.7 src/sage/modular/modform_hecketriangle/graded_ring_element.py  # 7 doctests failed
sage -t --warn-long 121.7 src/sage/schemes/projective/projective_morphism.py  # 4 doctests failed
sage -t --warn-long 121.7 src/sage/misc/randstate.pyx  # 1 doctest failed
sage -t --warn-long 121.7 src/sage/arith/misc.py  # 1 doctest failed
sage -t --warn-long 121.7 src/sage/structure/element.pyx  # 1 doctest failed
sage -t --warn-long 121.7 src/sage/doctest/test.py  # Timed out
sage -t --warn-long 121.7 src/sage/tests/french_book/mpoly.py  # 1 doctest failed
sage -t --warn-long 121.7 src/sage/rings/polynomial/multi_polynomial_ideal.py  # 4 doctests failed
sage -t --warn-long 121.7 src/sage/homology/simplicial_complex.py  # 1 doctest failed
sage -t --warn-long 121.7 src/sage/plot/misc.py  # 1 doctest failed
sage -t --warn-long 121.7 src/sage/interfaces/singular.py  # 3 doctests failed
sage -t --warn-long 121.7 src/sage/rings/polynomial/multi_polynomial_libsingular.pyx  # 4 doctests failedsage -t --warn-long 121.7 src/sage/schemes/affine/affine_rational_point.py  # Killed due to abort
sage -t --warn-long 121.7 src/sage/libs/singular/function.pyx  # 6 doctests failed
sage -t --warn-long 121.7 src/sage/rings/multi_power_series_ring_element.py  # 2 doctests failed
sage -t --warn-long 121.7 src/sage/rings/polynomial/polynomial_singular_interface.py  # 3 doctests failed
sage -t --warn-long 121.7 src/sage/rings/polynomial/multi_polynomial_ideal_libsingular.pyx  # 1 doctest failed  
sage -t --warn-long 121.7 src/sage/schemes/jacobians/abstract_jacobian.py  # 1 doctest failed
sage -t --warn-long 121.7 src/sage/rings/polynomial/polydict.pyx  # 1 doctest failed
----------------------------------------------------------------------

I assume many of them are unrelated to Singular, and due to the POWER7 arch, especially the plots ones.

[jpflori]

The singular related ones seem to be:

• schemes/elliptic_curves/isogeny_small_degree.py: segfault.
• modular/modform_hecketriangle/graded_ring_element.py: lots of singular stuff, including coercion.
• schemes/projective/projective_morphism.py: not sure... something about mpfr!
• arith/misc.py: representation of finite fields changed, easy to fix.
• structure/element.pyx: new limitation in max exponent?
• tests/french_book/mpoly.py: different result, might be just as good as the previous one.
• rings/polynomial/multi_polynomial_ideal.py: some different results, surely just as good and even better
• interfaces/singular.py: different Singular printing.
• rings/polynomial/multi_polynomial_libsingular.pyx: printing and squarefree stuff.
• schemes/affine/affine_rational_point.py: bad abort.
• src/sage/libs/singular/function.pyx: hopefully mostly random values changed in internal singular functions.
• rings/multi_power_series_ring_element.py: printing of power series changed.
• rings/polynomial/polynomial_singular_interface.py: printing changed.
• rings/polynomial/multi_polynomial_ideal_libsingular.pyx: different result, might be just as good.
• schemes/jacobians/abstract_jacobian.py: coercion error?
• rings/polynomial/polydict.pyx: printing of polys changed (x -> 1*x).

[leif]

Automake complains about LIBTOOL not being defined although libtool is used.

And finally I'm getting: curl: command not found... :-)

[jpflori]

Thread for the "1*" issue: ​https://groups.google.com/forum/#!topic/libsingular-devel/X29-H_bv9-4

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​a010eb5	New singular git version including gcc/gmp fix.
​9e14834	Merge remote-tracking branch 'trac/develop' into singular

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​62735be

Let's use Singular finite field struct for finite rings of prime char.

[jpflori]

As a sidenote, if we really want to stick to a finite ring structure rather than a finite field structure for Z/2Z and Z/pZ, we could do so. Hans fixed the "1*" disturbing behavior.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​2d84044	In Singular 4 limit for exponents is the same on 32 and 64 bit.
​75efae0	Let's stick with integer mod ring in Singular at the moment.
​919165a	Integer mod ring printing is not centered anymore in Singular.
​ace4d54	Spurious dot.
​cc921ec	Singular printing changes.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​50ed3b3	Warning disappeared in Singular (better implem?).
​a39f146	Different but (hopefully) equivalent Groebner basis.
​6c88bc3	Singular now yields same result for ZZ and QQ.

[jpflori]

For the segfault in schemes/elliptic_curves/isogeny_small_degree.py it seems that the following commit now triggers a double free or invalid free (use MALLOC_CHECK_=3 or it will just fails somewhere else randomly):

• ​https://github.com/sagemath/sage/commit/14061a1d40e8a27d86eac6dd91559b0131f9fe55

@Volker: could you have a look?

[jpflori]

Replying to jpflori:

For the segfault in schemes/elliptic_curves/isogeny_small_degree.py it seems that the following commit now triggers a double free or invalid free (use MALLOC_CHECK_=3 or it will just fails somewhere else randomly):

• ​https://github.com/sagemath/sage/commit/14061a1d40e8a27d86eac6dd91559b0131f9fe55

Kind of painfuly got it:

• nlNormalize(x) can free whatever is in x and change x from a struct to a disqguised int.
• when calling sa2si on a polynomial we call nlGetDenom or such functions on pointers returned by p_GetCoeff for the coeffs, which in turn call nlNormalize and free the underlying memory and modify our pointer, but not the original pointers in the polynomial
• we call p_Delete on the polynomial which call nlDelete on the coeff which gives no indication the underlying memory is freed
• bang

[jpflori]

The solution would be something like calling p_SetCoeff whenever we call p_GetCoeff to be sure any hackish modification gets transmitted back to the original place.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​cbd3af3	Seemingly equivalent result.
​20e0b22	Fix update of coefficients in algext singular elements.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​f9ae25e	Different singular printing for ring names.
​765b60a	Squarefree not implemented anymore by Singular for rings.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​e9b79db

Remove hack for singular 1*.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​21cf2a5

Hack to parse Singular polys with real coefficients.

[jpflori]

Here is the current state of affairs for make testlong on a POWER7/

sage -t --long src/sage/misc/randstate.pyx  # 1 doctest failed
sage -t --long src/sage/symbolic/function.pyx  # 1 doctest failed
sage -t --long src/sage/modular/modform_hecketriangle/graded_ring_element.py  # 7 doctests failed
sage -t --long src/sage/calculus/riemann.pyx  # 2 doctests failed
sage -t --long src/sage/schemes/projective/projective_morphism.py  # Timed out
sage -t --long src/sage/schemes/jacobians/abstract_jacobian.py  # 1 doctest failed
sage -t --long src/sage/schemes/curves/projective_curve.py  # 2 doctests failed
sage -t --long src/sage/schemes/curves/affine_curve.py  # 1 doctest failed
sage -t --long src/sage/rings/polynomial/multi_polynomial_ideal.py  # 2 doctests failed
sage -t --long src/sage/finance/time_series.pyx  # 6 doctests failed
sage -t --long src/sage/plot/graphics.py  # 1 doctest failed
sage -t --long src/sage/plot/contour_plot.py  # Killed due to segmentation fault
sage -t --long src/sage/plot/misc.py  # 1 doctest failed
sage -t --long src/sage/plot/plot.py  # 1 doctest failed
sage -t --long src/sage/plot/matrix_plot.py  # Killed due to segmentation fault
sage -t --long src/sage/tests/french_book/mpoly.py  # 2 doctests failed
sage -t --long src/sage/libs/singular/function.pyx  # 6 doctests failed
sage -t --long src/doc/en/prep/Advanced-2DPlotting.rst  # Killed due to segmentation fault
sage -t --long src/doc/en/constructions/algebraic_geometry.rst  # 14 doctests failed

[jpflori]

misc/randstate.pyx  # 1 doctest failed: unrelated, gap random different
symbolic/function.pyx  # 1 doctest failed: unrelated, precision issue
modular/modform_hecketriangle/graded_ring_element.py  # 7 doctests failed: singular, real coeffs printing with parenthesis
calculus/riemann.pyx  # 2 doctests failed: unrelated, precision issue
schemes/projective/projective_morphism.py  # Timed out: singular, real parsing 
schemes/jacobians/abstract_jacobian.py  # 1 doctest failed: singular, real parsing
schemes/curves/projective_curve.py  # 2 doctests failed: singular, real parsing
schemes/curves/affine_curve.py  # 1 doctest failed: singular, different result
rings/polynomial/multi_polynomial_ideal.py  # 2 doctests failed: singular, gwalk and skipping text issue
finance/time_series.pyx  # 6 doctests failed: unrelated, precision issue
plot/graphics.py  # 1 doctest failed: unrelated, div by 0
plot/contour_plot.py  # Killed due to segmentation fault: passed the second time
plot/misc.py  # 1 doctest failed: unrelated, div by 0
plot/plot.py  # 1 doctest failed: unrelated, div by 0
plot/matrix_plot.py  # Killed due to segmentation fault: passed the second time
tests/french_book/mpoly.py  # 2 doctests failed: singular, gwalk and skipping text issue
libs/singular/function.pyx  # 6 doctests failed: singular, arity code changes and skipping text issue

en/prep/Advanced-2DPlotting.rst  # Killed due to segmentation fault: passed the second time
en/constructions/algebraic_geometry.rst  # 14 doctests failed: singular, stuff in brnoeth.lib, undefined stuff

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​e705c27	Better handling of Singular 4 floats printing.
​5d18e56	Singular 4 puts sign outside of parentheses for floats.
​cfc8c56	Rely on Singular ring type rather than Sage one.
​c6eaaf4	New singular picks different ratfuncs.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​0192048

Update "arity code" constants for Singular 4.

[jpflori]

So we're hopefully left with:

• bug in mprimedec.lib (​https://groups.google.com/forum/#!topic/libsingular-devel/94d9c574uvg)
• bug in brnoeth.lib (​https://groups.google.com/forum/#!topic/libsingular-devel/DxjhbYqVfFQ)
• gwalk issue
• the "skipping text in parameter" output (when doing GB computations?)

[jpflori]

The grwalk thing must be:

• ​https://github.com/Singular/Sources/commit/53aa95a962b7ad2b6b48f9f308a22948c39cc41f

[jpflori]

In fact the grwalk and brnoeth issues were caused by old lib files in local/share/singular. Already fixed upstream.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​526f0a2

Make sure old Singular lib files are deleted.

[jpflori]

We're almost good now:

• the "skipping text from parameter" text only appears when an error previously happened in Singular, so we it now only appears when testing libs/singular/function.pyx becasuse of the GTZmod error.
• hopefully the GTZmod error will be fixed upstream soon and we can fix a commit to incule here (which should be more or less official Singular 4.0.3).

Can someone else please test the current branch? You'll need to build your own tarball first using spkg-src.

(There will still be some polishing to do about the Singular C++ function we use, see the comments Hans added in some header files, but that can wait for another ticket.)

[jpflori]

Oh crap, we should also reenable the possibility of debug builds. IIRC this was removed at some point in this ticket.

[jpflori]

But that can wait for another ticket as well (at least the replace omalloc by malloc stuff).

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​dcd4e14

More changes for Singular 4.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7366fb6

Better script to package singular tarball.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​9f016ae

Apply patches to singular before debug hacks.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​cfe9a9c

Patch for Singular xalloc.

[jpflori]

Setting SAGE_DEBUG=yes, debug mode with xalloc should be ok although:

• I did not wait for the build to finish,
• it seems auto* wants to reconfigure everything,
• you get a alot of unsued stuff warnings but that's surely normal.

I removed a bunch of hacks from spkg-install.

Please test!

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7dd92be

Fix singular debug build mode.

[dimpase]

the branch does not merge cleanly (so I had to do some manual editing), and after building singular I get

[sagelib-7.4.beta1] ************************************************************************
[sagelib-7.4.beta1] Traceback (most recent call last):
[sagelib-7.4.beta1]   File "setup.py", line 47, in <module>
[sagelib-7.4.beta1]     from module_list import ext_modules, library_order, aliases
[sagelib-7.4.beta1]   File "/home/dima/software/sage/src/module_list.py", line 1587, in <module>
[sagelib-7.4.beta1]     extra_compile_args = givaro_extra_compile_args),
[sagelib-7.4.beta1] NameError: name 'givaro_extra_compile_args' is not defined
[sagelib-7.4.beta1] ************************************************************************
[sagelib-7.4.beta1] Error building the Sage library

although this might be due to me trying to merge this into the branch on #17635 (which should probably become a dependency here)

[tscrim]

It built okay for me on my Ubunutu 16.04 LTS system.

Also, I think we should set a fixed tarball and checksum. Do you want me to do this?

[dimpase]

The upgrade of givaro on #17635 has to be taken into account here.

[Snark]

Replying to tscrim:

Also, I think we should set a fixed tarball and checksum. Do you want me to do this?

I don't follow the ticket very closely, but that makes me jump: what about taking an upstream tarball instead of hacking something?

[leif]

Replying to Snark:

Replying to tscrim:

Also, I think we should set a fixed tarball and checksum. Do you want me to do this?

I don't follow the ticket very closely, but that makes me jump: what about taking an upstream tarball instead of hacking something?

WIP / moving target:

commit 117ae9c7dde9eb9fcaa4cb899cfb7690addd6dfe
Author: Hans Schoenemann <hannes@mathematik.uni-kl.de>
Date:   Fri Aug 26 18:47:06 2016 +0200

    opt: prefer unsigned over int in for loops

commit f5e732af7572071c189ad8185ce830f96ababa46
Author: Hans Schoenemann <hannes@mathematik.uni-kl.de>
Date:   Fri Aug 26 18:46:40 2016 +0200

    compiler warnings

commit 6e28802f29d0fee351ccab022adc6853e46bd3fe
Author: Hans Schoenemann <hannes@mathematik.uni-kl.de>
Date:   Wed Aug 24 17:18:54 2016 +0200

    opt: use id_Delete in syz1.cc

commit 84963dfd52839fd9d4215c5d5b0534e80417dfad
Author: Hans Schoenemann <hannes@mathematik.uni-kl.de>
Date:   Wed Aug 24 16:42:29 2016 +0200

    opt: NULL in syResolvente->*res

commit 495258ce8ff8e5b1376e6c7fa8ae2815387e8b4a
Author: Hans Schoenemann <hannes@mathematik.uni-kl.de>
Date:   Mon Aug 22 16:04:20 2016 +0200

    removed wrong warning (see Tst/Plural/doc-module-ops.tst)

commit 32e84f5b8f065a8070d982ae939de64e4721beab
Author: Hans Schoenemann <hannes@mathematik.uni-kl.de>
Date:   Mon Aug 22 16:03:02 2016 +0200

    removed AlgDep-qso3-dp.tst from Plural/short.lst: also in Plural.lst

commit a070b84d59be835a2946d1386edd3221d80f3fe7
Author: Hans Schoenemann <hannes@mathematik.uni-kl.de>
Date:   Fri Aug 19 16:56:16 2016 +0200

    Singular 4.0.3p3

commit 961c76da92efc86885b6506c8c83262f14e705ff
Author: Hans Schoenemann <hannes@mathematik.uni-kl.de>
Date:   Fri Aug 19 16:34:25 2016 +0200

    chg: n_IsMOne for Z/2: maybe n_Zp, n_Zn, n_Z2m

...

[leif]

Minor thing: All error messages in spkg-install should start with "Error[:] ...".

spkg-src could be beautified a bit as well (e.g. not unconditionally deleting singular.git, and recording what we checked out, error checking), besides that curl is non-standard... ;-)

[leif]

P.S.: I'd prefer to not use -lSingular, but create a symlink from lower to uppercase instead.

[dimpase]

aclocal is currently a dependency, probably due to warnings treated as errors:

[singular-4.0.3p3] config.status: executing libtool commands
[singular-4.0.3p3] ### Singular spkg-install: build_singular ###
[singular-4.0.3p3] make[3]: Entering directory '/home/dima/Sage/sage/local/var/tmp/sage/build/singular-4.0.3p3/src/latest'
[singular-4.0.3p3] CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /home/dima/Sage/sage/local/var/tmp/sage/build/singular-4.0.3p3/src/latest/build-aux/missing aclocal-1.14 -I m4
[singular-4.0.3p3] /home/dima/Sage/sage/local/var/tmp/sage/build/singular-4.0.3p3/src/latest/build-aux/missing: line 81: aclocal-1.14: command not found
[singular-4.0.3p3] WARNING: 'aclocal-1.14' is missing on your system.
[singular-4.0.3p3]          You should only need it if you modified 'acinclude.m4' or
[singular-4.0.3p3]          'configure.ac' or m4 files included by 'configure.ac'.
[singular-4.0.3p3]          The 'aclocal' program is part of the GNU Automake package:
[singular-4.0.3p3]          <http://www.gnu.org/software/automake>
[singular-4.0.3p3]          It also requires GNU Autoconf, GNU m4 and Perl in order to run:
[singular-4.0.3p3]          <http://www.gnu.org/software/autoconf>
[singular-4.0.3p3]          <http://www.gnu.org/software/m4/>
[singular-4.0.3p3]          <http://www.perl.org/>
[singular-4.0.3p3] make[3]: *** [Makefile:485: aclocal.m4] Error 127
[singular-4.0.3p3] make[3]: Leaving directory '/home/dima/Sage/sage/local/var/tmp/sage/build/singular-4.0.3p3/src/latest'
[singular-4.0.3p3] 
[singular-4.0.3p3] real	0m28.318s
[singular-4.0.3p3] user	0m13.437s
[singular-4.0.3p3] sys	0m4.493s
[singular-4.0.3p3] ************************************************************************
[singular-4.0.3p3] Error installing package singular-4.0.3p3
[singular-4.0.3p3] ************************************************************************

only caught this as I'm setting up a new laptop :-)

[dimpase]

I have aclocal-1.15, but it insists on 1.14; weird, no?

[jdemeyer]

Replying to dimpase:

aclocal is currently a dependency

This should be fixed by running the necessary autotools in spkg-src.

[leif]

Replying to jdemeyer:

Replying to dimpase:

aclocal is currently a dependency

This should be fixed by running the necessary autotools in spkg-src.

AC_REQUIRE proper timestamps.

[dimpase]

Replying to jdemeyer:

Replying to dimpase:

aclocal is currently a dependency

This should be fixed by running the necessary autotools in spkg-src.

I did make the tarball using spkg-src, as described. I guess it's an upstream bug: they don't detect presence of aclocal-1.14, even though they don't need it, and bail out, even though it's meant to be a warning.

Any, why 1.14 only? Won't they be happy with 1.14 or newer?

[dimpase]

modulo the following, ptestlong passes with the new branch

sage -t --long --warn-long 77.2 src/sage/schemes/curves/affine_curve.py
**********************************************************************
File "src/sage/schemes/curves/affine_curve.py", line 494, in sage.schemes.curves.affine_curve.AffineCurve.resolution_of_singularities
Failed example:
    C.resolution_of_singularities(extend=True)[0] # long time (2 seconds)
Expected:
    (Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*x^2*ss1^3 + 24*ss1^3 + (a0^3 - 8*a0),
     Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (6*a0^3)*s1,
     Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 8*y^2*s0^4 + (-4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y)
Got:
    (Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*x^2*ss1^3 + 24*ss1^3 + (a0^3 - 8*a0),
     Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (-6*a0^3)*s1,
     Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 8*y^2*s0^4 + (4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y)
**********************************************************************
1 item had failures:
   1 of  23 in sage.schemes.curves.affine_curve.AffineCurve.resolution_of_singularities
    [195 tests, 1 failure, 3.51 s]

I don't know enough about resolution of singularities to decide whether the latter output is correct, or not (notice some signs are changed...)

---

Last 10 new commits:

​5d8c1d9	adding a patch fixing the i386 with AVX issue
​0d27874	Fix int size bug in the interface between singular and givaro (on 32bit arch)
​162c717	preparing upgrade to linbox-1.4.2rc0 fflas-ffpack-2.2.2rc0 and givaro 4.0.2rc0
​20e86b3	update to the lastest releases
​2fb5502	* Merging and fixing conflicts.
​85ce544	Merge branch 'u/cpernet/givaro_fflasffpack_linbox' of git://trac.sagemath.org/sage into givaroupd
​b7233c9	Fix coercion bug: raise a TypeError exception whenever an ArithmeticError is caught, while trying to reduce an element mod the characteristic.
​cd30aec	Merge branch 'u/cpernet/givaro_fflasffpack_linbox' of git://trac.sagemath.org/sage into givaroupd
​0efcdec	Merge branch 'public/singular4' into givaro 4.0.2 branch from #17635
​457bd23	missing capital 'S' in Singular

[leif]

Could we please not mix commits from different upgrades?

I don't see an immediate reason to base this ticket on #17635 either; they're IMHO better first reviewed independently.

[embray]

LGTM at least in principle. I'm going to try over the next day or two to test how this works on Cygwin64.

[embray]

Also I guess the milestone for this ticket should be changed? I don't know what to though.

[dimpase]

Replying to leif:

Could we please not mix commits from different upgrades?

I don't see an immediate reason to base this ticket on #17635 either; they're IMHO better first reviewed independently.

well, they depend, albeit not too deeply. Perhaps wait for the next beta with #17635 in?

[tscrim]

IMO, we should get this in as soon as we can to get as much testing as we can before it goes into a stable release.

[embray]

Replying to tscrim:

IMO, we should get this in as soon as we can to get as much testing as we can before it goes into a stable release.

+1; I'm trying to get cygwin testing ASAP, but first I have to get cygwin builds working against the current develop (since my cygwin branch is from a couple months' old develop). Should be close to that, but I'd be fine if this is merged in the meantime, and I can open a ticket for any issue I discover later.

[jdemeyer]

#17635 is now closed, so you need to make sure that it doesn't conflict with that ticket.

[jdemeyer]

...although Volker closes tickets before fully testing them, so there is still a chance that it will be reopened.

[jpflori]

Replying to embray:

LGTM at least in principle. I'm going to try over the next day or two to test how this works on Cygwin64.

Thx, I absolutely did nothing Cygwin-related on that one.

[jpflori]

Replying to leif:

Minor thing: All error messages in spkg-install should start with "Error[:] ...".

Yes.

spkg-src could be beautified a bit as well (e.g. not unconditionally deleting singular.git, and recording what we checked out, error checking), besides that curl is non-standard... ;-)

Sure but curl was already used in the script anyway and that's only to make the tarball, not for end-users.

[jdemeyer]

I agree that using curl in spkg-src is not a big issue.

[jdemeyer]

This cannot go into Sage until there is a proper tarball.

[jdemeyer]

What's the point of this commented-out code? Just remove it.

# C++11 workaround https://trac.sagemath.org/ticket/20926
# Since https://trac.sagemath.org/ticket/17635 linbox passes a -std=gnu++11 flag, so we no longer need to add it
# if [ "$UNAME" = CYGWIN ]; then
     # Special case for Cygwin https://trac.sagemath.org/ticket/21185
#     CXXFLAGS="$CXXFLAGS -std=gnu++98"
# else
#     CXXFLAGS="$CXXFLAGS -std=c++98"
#fi

[jdemeyer]

In build/pkgs/singular/spkg-install, if you use set -e, it is pointless to do manual error checking like

    if [ $? -ne 0 ]; then
        echo >&2 "Error applying '$patch'"
        return 1
    fi

[jdemeyer]

What's the point of this commented-out code? Just remove it.

 #    Extension('sage.libs.linbox.linbox',
 #             sources = ['sage/libs/linbox/linbox.pyx']),

More generally, better remove all commented-out code unless there is a clear purpose.

[jdemeyer]

In

    cdef enum n_coeffType:
        n_unknown=0
        n_Zp=1 #/**< \F{p < 2^31} */
        n_Q=2   #/**< rational (GMP) numbers */
        [...]

I would remove the values (like =0) because that is only confusing. And please either properly format the comments there or remove them.

[jdemeyer]

This is also silly:

            if sizeof(long) > 4:

                # it seems Singular uses ints somewhere
                # internally, cf. #6051 (Sage) and #138 (Singular)

                if exponent <= 30:  ringtype = n_Z2m
                else:               ringtype = n_Znm
            else:
                if exponent <= 30: ringtype = n_Z2m
                else:              ringtype = n_Znm

You can change code of the form

if condition:
    A
else:
    A

to

A

[jdemeyer]

I think this regression is not acceptable:

             sage: P.<x,y,z> = PolynomialRing(QQ)
             sage: f= x^2 + 2*x*y + 1/2*z
             sage: f.is_squarefree()
             Traceback (most recent call last):
             ...
             NotImplementedError: is_squarefree of multivariate polynomials over rings is not implemented.

Asking whether a multivariate polynomial is squarefree seems like a very natural thing to do and not something you should just break.

[jdemeyer]

Also, I am not so happy with the regression in exponent overflow but that's a less serious issue.

[jdemeyer]

Sorry to say this, but the code is really ugly and certainly uglier than it was before. I see lots of annoying things, like unneeded comments, indentation not a multiple of 4 spaces, needlessly complicated code.

I think this requires somebody to go over all the code and clean it up.

[jdemeyer]

Another example:

    libSingularFound = False

    for extension in ["so", "dylib", "dll"]:
        lib = os.environ['SAGE_LOCAL']+"/lib/libSingular."+extension # libsingular renamed to libSingular
        if os.path.exists(lib):
            libSingularFound = True
            handle = dlopen(lib, RTLD_GLOBAL|RTLD_LAZY)
            if not handle:
                err = dlerror()
                if err:
                    print(err)
            break

    if not libSingularFound :
        raise OSError("did not find libsingular file")


    if handle == NULL:
        dlerrormsg = dlerror()
        if dlerrormsg == NULL:
            dlerrormsg = ""
        raise ImportError("failed to dlopen {} ({})".format(lib, dlerrormsg))

The handle == NULL cannot be reached since we would error out before. The old code is simpler and better:

    for extension in ["so", "dylib", "dll"]:
        lib = os.environ['SAGE_LOCAL']+"/lib/libsingular."+extension
        if os.path.exists(lib):
            handle = dlopen(lib, RTLD_GLOBAL|RTLD_LAZY)
            if not handle:
                err = dlerror()
                if err:
                    print(err)
            break

    if handle == NULL:
        raise ImportError("cannot load libSINGULAR library")

There was no reason to make the code more complicated, why was it done???

[jdemeyer]

Another example in plural.pyx:

cdef n_coeffType unknownRingtype = n_unknown

What's the point of this?

[jdemeyer]

To be clear: I am very happy with the work done on this ticket, really. I just feel that it's not ready to merge yet. I also feel that not much effort was done to re-use the existing code, that code was rewritten without reason. That makes it hard to review because the diff is huge.

[jpflori]

No worry Jeroen, that's exactly the point of making a proper review. To this point this ticket is just a heap of changes, hacks, commits by different people to get the new singular working and at least on my side I did not really look at what the other did, especially given the huge diff. I'll try to polish the points you mentioned this week.

[jdemeyer]

At least we have something that works (modulo the is_squarefree issue). That is a very important starting point.

[dimpase]

Replying to jdemeyer:

At least we have something that works (modulo the is_squarefree issue). That is a very important starting point.

surely Singular 4.0.3 still has the squarefree test, so it's matter of digging up how it is implemented:

> ring r=0,(x,y),dp;
> poly p=x^2+2*x*y+y^2;
>> ring r=0,(x,y),dp;
> poly p=x^2+2*x*y+y^2;
> sqrfree(p);
[1]:
   _[1]=1
   _[2]=x+y
[2]:
   1,2

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7e8eb3a	add pacth fixing the gcc <= 4.9.2 issue by leif.
​aab3eb1	Merge branch 'u/cpernet/givaro_fflasffpack_linbox' of trac.sagemath.org:sage into givaroupd

[dimpase]

merged last commit from givaro upgrade ticket, uploaded the tarball.