Upgrade to Singular-4.x.x

[jdemeyer]

Lots of stuff has changed in Singular 4. This is a big update.

Build system completely changed, and a lot of internal stuff.

Original upstream tarball at www.mathematik.uni-kl.de/ftp/pub/Math/Singular/SOURCES/4-0-3/: singular-4.0.3p3 dot tar dot gz

Modified tarball made with spkg-src at:

• ​http://www.lmona.de/files/sage/singular-4.0.3p3.tar.bz2

This corresponds to commit a070b84d whose message tells 4.0.3p3 and on which this branch was based:

• ​https://github.com/Singular/Sources/commit/a070b84d59be835a2946d1386edd3221d80f3fe7

You can also make your own tarball based on this commit or another one with

env SINGULAR_GIT_COMMIT=a070b84d ./sage --sh build/pkgs/singular/spkg-src
./sage --package fix-checksum singular

This script is loosely based on the make_tar.sh script from upstream.

[jpflori]

Yeah, the update is non trivial, that's a reason to get #17184 (and so #16882) inbetween.

[jdemeyer]

Replying to jpflori:

Yeah, the update is non trivial

I wonder if we even have a Sage developer who is familiar enough with Singular to do this (for the record: not me, I know nothing about Singular).

that's a reason to get #17184 (and so #16882) inbetween.

Of course, I'm not arguing with that.

[jpflori]

Replying to jdemeyer:

Replying to jpflori:

Yeah, the update is non trivial

I wonder if we even have a Sage developer who is familiar enough with Singular to do this (for the record: not me, I know nothing about Singular).

Maybe Burcin would do... IIRC the update was proposed as a GSoC project.

[jakobkroeker]

Q: where to get the singular tarball for this commit? I get a failure at

Trying to download http://www.sagemath.org/packages/upstream/singular/singular-4.0.1p1.tar.bz2

2.

Update to Singular 4.0.1

Maybe Burcin would do...

Could the update to 4.0.1. be done by at least two or three persons to spread the knownledge about Singular internals?

[jpflori]

You have to put a tarball repackaged by the spkg-src script in the upstream folder directly. Once the ticket is merged, the tarball will be put on the official mirror and could be downloaded automatically but not at the moment.

I've put a repackaged tarball at:

• ​http://boxen.math.washington.edu/home/jpflori/upstream/singular-4.0.1.tar.bz2

[jpflori]

(I'm not sure the checksums will the same as what Jeroen branch expect, run ./sage -sh sage-fix-pkg-checksums after you have downloaded the tarball to fix this.)

[jpflori]

And I'd be happy to work on the upgrade and learn more about Singular's internals, but please also review the "easy" #17184 (where I've already learnt some stuff) firsT.

[jakobkroeker]

Singular 4.0.1. compiles now in sage but sage does not yet - I just pushed a branch 'u/jakobkroeker/ticket/17254'. You have to correct the 'singular-4.0.1p1.tar.bz2' tarball to get it work ( the 'm4' folder is missing).

For the moment I removed some patches; we probably will have to adjust them later on. For example there is meanwhile no such thing like 'lomalloc_ndebug' so if we want to disable some debug functionality or output for omalloc we have to do that by passing appropriate flags to omalloc/configure, e,g '--enable-optimizationflags --disable-debug --without-debug' ?

Also I disabled the currently broken '--enable-debugoutput' configure flag for the standalone 'factory' and informed Oleksander Motsak about the issue.

Could someone of you take care of the sage compile errors and figure out what is going wrong and/or ask for help in the Singular forum? I can't do that easily, because within the Singular-team I'm out-of-favor because I constantly criticise their software and thus I will likely get little or no support.

Jakob

[jdemeyer]

New commits:

​4c0bba3

singular 4.0.1 builds now, but sage does not due to changes in libsingular?

[jakobkroeker]

first hint: for example the interface of 'naIsZero' seems changed from

BOOLEAN naIsZero(number a)

to

BOOLEAN naIsZero(number a, const coeffs cf)

Persons who might have an idea about the changes:

Oleksandr Motsak Mohamed Barakat Martin Lee Hans Schönemann

see contact details at ​http://www.mathematik.uni-kl.de/agag/mitglieder/

[jdemeyer]

Replying to jpflori:

I've put a repackaged tarball at:

• ​http://boxen.math.washington.edu/home/jpflori/upstream/singular-4.0.1.tar.bz2

That's an older version than the one that I put in the ticket description, any reason...?

[jdemeyer]

Replying to jakobkroeker:

You have to correct the 'singular-4.0.1p1.tar.bz2' tarball to get it work ( the 'm4' folder is missing).

My tarball compiled fine, so what is the problem?

[jdemeyer]

You should never run ./autogen.sh, that is supposed to be run at packaging-time, not build-time.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​79c079f	Upgrade to Singular-4-0-1
​4c0bba3	singular 4.0.1 builds now, but sage does not due to changes in libsingular?
​80a6b05	Merge commit '4c0bba3' into ticket/17254

[jakobkroeker]

My tarball compiled fine, so what is the problem?

Confirm; cannot reproduce the issue. So likely I didn't use the recent tarball, or I screwed something up.

You should never run ./autogen.sh, that is supposed to be run at packaging-time, not build-time.

I'm guilty ;-)

[jakobkroeker]

singulars spkg-install:

the '--enable-debugoutput' configure flag for standalone factory needs now in additon '--enable-streamio --without-Singular'

[jakobkroeker]

After a little more tweaking and temporarily disabling the extension fields/rings, I was able to build some parts of sage with Singular 4.0.1 on a 'fc18 system' and then.. sage crashes at start! That happens because I didn't update ring creation right - I only recently discovered examples at './libpolys/tests/rings_test.h'. And some sage packages create common rings at loading time.

remaining riddles:

• during build one sage file tried to link to -lsingular (I have no idea why), instead to the libSingular* files (that changed!), so I worked around by manually creating links to libSingular* files.

• on a ScientificLinux? system I get errors like "ImportError?: /home/kroeker/Projects/sage-singular-4.0.1/local/lib/python2.7/site-packages/sage/rings/polynomial/plural.so: undefined symbol: wFunctionalBuch". This symbol is in libSingular*

• fix ring creation. See examples at './libpolys/tests/rings_test.h'

• the extension fields will need some more love. See 'algext.h' and 'transext.h'. @Martin: could you take a look?

• some mapping functions (e.g. nr2mMapZp) now require a source and dst (coefficient) parameter. Not sure if I did it right (source: currentRing, dest: _ring or vice versa?)

• I didn't fix unused interface definitions in singular-cdefs.pxi. TODO

See branch u/jakobkroeker/ticket/17254 and some notes at ​https://github.com/jakobkroeker/jk_test_sage/issues/13

If a build get stuck before errors described above, then maybe a 'make distclean' is necessary. And of course, the Singular tarball ​http://boxen.math.washington.edu/home/jdemeyer/spkg/singular-4.0.1p1.tar.bz2 is required.

Any help appreciated.

---

New commits:

​79c079f	Upgrade to Singular-4-0-1
​2fcaeb5	Update Singular to 4.0.1
​4c0bba3	singular 4.0.1 builds now, but sage does not due to changes in libsingular?
​80a6b05	Merge commit '4c0bba3' into ticket/17254
​2ae243a	messy fixes
​ff2a9dc	required changes for singular to build sage; should be fixed upstream
​3b7877e	hacking sage to build with Singular 4.0.1. needs more love for extension coefficient fields and a refactoring
​f178dfc	fixing nr2mMapZp call

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​8ba17d8

messy fixes

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​2ae243a	messy fixes
​ff2a9dc	required changes for singular to build sage; should be fixed upstream
​3b7877e	hacking sage to build with Singular 4.0.1. needs more love for extension coefficient fields and a refactoring
​f178dfc	fixing nr2mMapZp call
​8e82387	Merge branch 'u/jakobkroeker/jakobkroeker.ticket.17254' of git://trac.sagemath.org/sage into jakobkroeker.ticket.17254
​c2a21bd	fixed some ring creation

[jakobkroeker]

New commits:

​6384b9b

Upgrade to Singular 4.0.1: incomplete, messy patch(first aproach)

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​221c5d6

Upgrade to Singular 4.0.1: incomplete, messy patch(first aproach)

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​1e84789

one more renaming from libsingular to libSingular

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​3a512cd

improved error handling messages when loading Singular

[Snark]

What is the status of this ticket? I see the branch is off 6.5.rc0, which is quite old already...

[jakobkroeker]

What is the status of this ticket? I see the branch is off 6.5.rc0, which is quite old already...

I'm busy and can't work on the ticket currently and expect to have time for the ticket at end of June, not earlier...

with the state as is, sage should build and start (using the Singular tarball from above), but extension fields and transcendental fields are not functional yet and also a cython Singular-interface cleanup has to be done.

[john_perry]

Replying to jakobkroeker:

I'm busy and can't work on the ticket currently and expect to have time for the ticket at end of June, not earlier...

with the state as is, sage should build and start (using the Singular tarball from above), but extension fields and transcendental fields are not functional yet and also a cython Singular-interface cleanup has to be done.

I'm at SD64.25 and am willing to work on this, but I'll need some hand-holding since I haven't worked on a package like this before. Do you have any suggestions for ways to test extension and transcendental fields?

[john_perry]

I just applied this to 6.4.1 and encountered the message,

fatal error: Singular/libsingular.h: No such file or directory

while compiling

build/cythonized/sage/algebras/letterplace/free_algebra_element_letterplace.cpp

Any ideas what causes this?

[fbissey]

I haven't looked yet at the new singular 4.0.x but the first port of call would be to check if the header is indeed installed in $SAGE_LOCAL/include.

[john_perry]

Replying to fbissey:

I haven't looked yet at the new singular 4.0.x but the first port of call would be to check if the header is indeed installed in $SAGE_LOCAL/include.

Shortly after, that installation of Sage fell completely apart (not sure why) so now I'm working on Sage-6.7.

On 6.7, git identifies some issues with merging in singular.pyx and ring.pyx, but they're fairly clear to fix, so I did. (Unless they weren't easy.) Things went much further this time, until I encountered this error:

****************************************************
### Singular spkg-install: choose_patches ###
### Singular spkg-install: apply_patches ###
Applying /Applications/sage-6.7/local/var/tmp/sage/build/singular-4.0.1p1.p0/patches/stricmp.patch
can't find file to patch at input line 7
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|stricmp is being deprecated in Cygwin.
|One should use strcasecmp.
|See https://cygwin.com/ml/cygwin/2014-10/msg00359.html
|diff -druN src/latest/Singular/run.c src/latest/Singular/run.c
|--- latest/Singular/run.c	2014-11-19 14:06:05.000000000 +0100
|+++ latest/Singular/run.c	2015-01-16 09:32:45.771298300 +0100
--------------------------
File to patch: 
Skip this patch? [y] 
Skipping patch.
1 out of 1 hunk ignored
Error applying '/Applications/sage-6.7/local/var/tmp/sage/build/singular-4.0.1p1.p0/patches/stricmp.patch'
Error building Singular (error in apply_patches).

Here's what the file in question suggests:

stricmp is being deprecated in Cygwin.
One should use strcasecmp.
See https://cygwin.com/ml/cygwin/2014-10/msg00359.html
diff -druN src/latest/Singular/run.c src/latest/Singular/run.c
--- latest/Singular/run.c       2014-11-19 14:06:05.000000000 +0100
+++ latest/Singular/run.c       2015-01-16 09:32:45.771298300 +0100
@@ -45,6 +45,7 @@
  #include <sys/stat.h>
  #include <sys/cygwin.h>
  #include <sys/unistd.h>
+ #define stricmp strcasecmp
 //WinMainCRTStartup() { mainCRTStartup(); }
 #else
  #include <direct.h>

Indeed, I do not find run.c in local/var/tmp/sage/build/singular-4.0.1p1.p0/src/latest/Singular/, which I am guessing is where this is supposed to be; instead, I find only run.h.

In my local distribution of Singular, I find a run.c. If I copy it over & try to make, I get the same complaint, & it looks as if the file is actually deleted. Suggestions?

[fbissey]

Did you mean local/var/tmp/sage/build/singular-4.0.1p1.p0/src/Singular/ the latest bit should be stripped by the -p1 option to patch.

[john_perry]

The patch has latest in it; that's why I wrote that in there, so that is what I mean. Is your meaning that latest should not be there? That directory doesn't exist when I look through the paths.

[fbissey]

It is expected for that directory not to exist for you. Whoever wrote the patch was working in a folder named latest but it is their own naming. It doesn't matter anyway. Let me explain to you how the -p option of patch works.

Your patch has a header that helps you locate the file to patch:

--- latest/Singular/run.c       2014-11-19 14:06:05.000000000 +0100
+++ latest/Singular/run.c       2015-01-16 09:32:45.771298300 +0100

patch < mypatch.patch or equivalently patch -p0< mypatch.patch will look for the file latest/Singular/run.c relative to where patch is executed. -pN will strip the N first folders in front of the file name so

patch -p1< mypatch.patch will look for Singular/run.c. In spkg-install for singular, patches are applied with patch -p1 <"$patch" so you shouldn't care about latest it is discarded. Look for local/var/tmp/sage/build/singular-4.0.1p1.p0/src/Singular/run.c OK?

[john_perry]

I think you misunderstand.

Replying to fbissey:

It is expected for that directory not to exist for you.

Why not? That directory does exist for me. The problem is that the file run.c does not exist in it.

Look for local/var/tmp/sage/build/singular-4.0.1p1.p0/src/Singular/run.c OK?

There is no such directory. I can get to local/var/tmp/sage/build/singular-4.0.1p1.p0/src/ but the only directories in there are latest and shared.

[fbissey]

OK you seem to be right. The tarball has been packaged strangely and I am not sure I have the right spkg-install. It is a cygwin patch and cygwin support in singular seems to have changed considerably. I would just remove the patch for now, it doesn't/cannot apply anywway any new fix for cygwin would have to be done from scratch by someone with a cygwin system.

[john_perry]

How do I remove it? If I move it & make anew, the file reappears.

[fbissey]

git rm build/pkgs/singular/patches/stricmp.patch?

[john_perry]

Thanks, that worked. Now I get failure on a line in ring.pyx,

    if is_64_bit:

Namely, Cython (?) claims this is undefined. I guess this is a change in Cython, since that isn't introduced by this patch. I found online another way to test this (sys.maxsize > 2**32) so I'm trying with that. Found another issue, but that was easy. Proceeding with fingers crossed... it's still compiling, which is already an progress...!

[john_perry]

It seems to have built, but Sage doesn't run. Here's the last few lines of the compilation (sorry for so much, but I wanted to show what it was saying at the end of what looks like a successful build, too):

...
byte-compiling /Applications/sage-6.7/local/lib/python2.7/site-packages/sage_setup/find.py to find.pyc
byte-compiling /Applications/sage-6.7/local/lib/python2.7/site-packages/sage_setup/optional_extension.py to optional_extension.pyc
running install_egg_info
Removing /Applications/sage-6.7/local/lib/python2.7/site-packages/sage-6.8.beta0-py2.7.egg-info
Writing /Applications/sage-6.7/local/lib/python2.7/site-packages/sage-6.8.beta0-py2.7.egg-info

real	0m25.849s
user	0m11.855s
sys	0m3.847s
[ -f local/etc/sage-started.txt ] || local/bin/sage-starts
build/pipestatus "./sage --docbuild --no-pdf-links all html  2>&1" "tee -a logs/dochtml.log"
Deleting empty directory /Applications/sage-6.7/src/doc/common/static
Deleting empty directory /Applications/sage-6.7/src/doc/en/reference/combinat/static
Deleting empty directory /Applications/sage-6.7/src/doc/en/reference/combinat/templates
Deleting empty directory /Applications/sage-6.7/src/doc/en/reference/polynomial_rings/static
Deleting empty directory /Applications/sage-6.7/src/doc/en/reference/polynomial_rings/templates
Deleting empty directory /Applications/sage-6.7/src/doc/en/reference/repl/static
Deleting empty directory /Applications/sage-6.7/src/doc/en/reference/repl/templates
Deleting empty directory /Applications/sage-6.7/src/doc/en/reference/tensor_free_modules/static
Deleting empty directory /Applications/sage-6.7/src/doc/en/reference/tensor_free_modules/templates
Deleting empty directory /Applications/sage-6.7/src/doc/output/inventory/en/reference/combinat
Traceback (most recent call last):
  File "/Applications/sage-6.7/src/doc/common/builder.py", line 1619, in <module>
    import sage.all
  File "/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/all.py", line 98, in <module>
    from sage.rings.all      import *
  File "/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/rings/all.py", line 68, in <module>
    from number_field.all import *
  File "/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/rings/number_field/all.py", line 7, in <module>
    from totallyreal import enumerate_totallyreal_fields_prim
  File "sage/rings/number_field/totallyreal_data.pxd", line 16, in init sage.rings.number_field.totallyreal (build/cythonized/sage/rings/number_field/totallyreal.c:10339)
  File "sage/rings/number_field/totallyreal_data.pyx", line 39, in init sage.rings.number_field.totallyreal_data (build/cythonized/sage/rings/number_field/totallyreal_data.c:11133)
  File "/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring_constructor.py", line 465, in PolynomialRing
    R = _single_variate(base_ring, name, sparse, implementation)
  File "/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring_constructor.py", line 539, in _single_variate
    R = m.PolynomialRing_integral_domain(base_ring, name, sparse, implementation)
  File "/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring.py", line 1544, in __init__
    sparse=sparse, element_class=element_class)
  File "/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring.py", line 1451, in __init__
    sparse=sparse, element_class=element_class, category=category)
  File "/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring.py", line 305, in __init__
    from sage.matrix.matrix_space import is_MatrixSpace
  File "/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/matrix/matrix_space.py", line 57, in <module>
    import matrix_mpolynomial_dense
ImportError: dlopen(/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/matrix/matrix_mpolynomial_dense.so, 2): Symbol not found: __Z17initCanonicalFormv
  Referenced from: /Applications/sage-6.7/local/lib/python2.7/site-packages/sage/matrix/matrix_mpolynomial_dense.so
  Expected in: flat namespace
 in /Applications/sage-6.7/local/lib/python2.7/site-packages/sage/matrix/matrix_mpolynomial_dense.so
make: *** [doc-html] Error 1

[fbissey]

It appears that it should be generated by initCanonicalForm but no functions of that name is to be found in the source code I have for singular 4.0.2 here and I am expecting it to be the same for you. I think you may have old singular headers on your system that are interfering (factory.h) we'll have to develop a proper upgrade procedure to remove singular 3.1.x stuff before installing 4.x. That or the singular 4.x headers have not been installed properly. local/include/factory/factory/factory.h and local/include/factory/factory.h both have this function in singular 3.1.7 but it shouldn't be present in the equivalent headers from singular 4.x.

[john_perry]

I don't seem to have a local/include/factory at all...

[fbissey]

I really have to inspect the spkg. I think I went overboard with the factory in the path. Should have been local/include/factory.h and local/include/factory/factory.h. Do you have a factory.h and does it contains initCanonicalForm?

[john_perry]

make distclean then make seems to have fixed my problems. I am apparently running Singular 4, albeit with the changes I mentioned earlier. At startup, I see the following:

ring with rational coefficient field created
_ring.ShortOut 1
_ring.N 2
polynomial ring over integers created
_ring.ShortOut 1
_ring.N 2
ring with rational coefficient field created
_ring.ShortOut 1
_ring.N 2

I found that in the source code, so I'm guessing this was put in to test.

[john_perry]

Well, this is bad.

sage: R=PolynomialRing(GF(32003),'x',7)
sage: I = sage.rings.ideal.Cyclic(R,7).homogenize()
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-2-59ae6a3e91c7> in <module>()
----> 1 I = sage.rings.ideal.Cyclic(R,Integer(7)).homogenize()

/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/rings/ideal.pyc in Cyclic(R, n, homog, singular)
   1607         n = R.ngens()
   1608 
-> 1609     singular.lib("poly")
   1610     R2 = R.change_ring(RationalField())
   1611     R2._singular_().set_ring()

/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/interfaces/singular.pyc in lib(self, lib, reload)
    783         if not reload and lib in self.__libs:
    784             return
--> 785         self.eval('LIB "%s"'%lib)
    786         self.__libs.append(lib)
    787 

/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/interfaces/singular.pyc in eval(self, x, allow_semicolon, strip, **kwds)
    585             x += ';'
    586 
--> 587         s = Expect.eval(self, x, **kwds)
    588 
    589         if s.find("error") != -1 or s.find("Segment fault") != -1:

/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/interfaces/expect.pyc in eval(self, code, strip, synchronize, locals, allow_use_file, split_lines, **kwds)
   1220                 elif split_lines:
   1221                     return '\n'.join([self._eval_line(L, allow_use_file=allow_use_file, **kwds)
-> 1222                                         for L in code.split('\n') if L != ''])
   1223                 else:
   1224                     return self._eval_line(code, allow_use_file=allow_use_file, **kwds)

/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/interfaces/expect.pyc in _eval_line(self, line, allow_use_file, wait_for_prompt, restart_if_needed)
    817         try:
    818             if self._expect is None:
--> 819                 self._start()
    820             E = self._expect
    821             try:

/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/interfaces/singular.pyc in _start(self, alt_message)
    410         """
    411         self.__libs = []
--> 412         Expect._start(self, alt_message)
    413         # Load some standard libraries.
    414         self.lib('general')   # assumed loaded by misc/constants.py

/Applications/sage-6.7/local/lib/python2.7/site-packages/sage/interfaces/expect.pyc in _start(self, alt_message, block_during_init)
    427                 # Change pexpect errors to RuntimeError
    428                 raise RuntimeError("unable to start %s because the command %r failed: %s\n%s" %
--> 429                         (self.name(), cmd, e, self._install_hints()))
    430         except BaseException:
    431             self._expect = None

RuntimeError: unable to start singular because the command 'Singular -t --ticks-per-sec 1000' failed: The command was not found or was not executable: Singular.

[john_perry]

For what it's worth, lots of other things work.: I can, for instance, compute an ideal's Gröbner basis, its Hilbert [numerator, series, polynomial], and so forth.

[john_perry]

Indeed, there is no Singular binary in local/bin. I presume this is the problem, though I have no idea what became of it.

[john_perry]

I'm guessing that failure to install the singular binary is because of a symlink: local/bin/singular' points to local/bin/Singular and removing the old (3.x) Singular wasn't enough to allow the new one in. I removed local/bin/singular` and it's now working.

This may be an issue with Apple's file system, because after installing the new Singular I wasn't allowed to create the symlink.

[jdemeyer]

is_64_bit is deprecated. In Cython, check sizeof(the_type_that_you_care_about) instead.

Do not use sys.maxsize, that's much slower.

[fbissey]

Replying to john_perry:

I'm guessing that failure to install the singular binary is because of a symlink: local/bin/singular' points to local/bin/Singular and removing the old (3.x) Singular wasn't enough to allow the new one in. I removed local/bin/singular` and it's now working.

This may be an issue with Apple's file system, because after installing the new Singular I wasn't allowed to create the symlink.

Yes you may have seen another ticket in which I demonstrated that something like Singular was really singular. OS X file system is not case sensitive but it gives the impression of being one.

[john_perry]

Replying to john_perry:

I just applied this to 6.4.1 and encountered the message,

fatal error: Singular/libsingular.h: No such file or directory

while compiling

build/cythonized/sage/algebras/letterplace/free_algebra_element_letterplace.cpp

Any ideas what causes this?

It's now certain that this, too, was caused by the failure to install Singular because singular exists in the directory (what can I say, it's a Mac). Removing that link and reinstalling the spkg fixed all these problems.

Meanwhile, I discovered some memory-related bugs in the patch. For instance, ring.pyx has the line

assert( exponent == base_ring.degree() )

which causes a failure. I'm not sure why those should be equal (the test I used had Zmod(2^3) where exponent==3 and base_ring.degree()==1), but commenting it out leads to a failure on these lines:

_param.GFChar     = ch;
_param.GFDegree   = base_ring.degree();
_param.GFPar_name = omStrDup(base_ring.gen());   

because _param has not been initialized. Adding

_param = <GFInfo *>omAlloc(sizeof(GFInfo));

before the access gives me a ring. However,

1. I'm not sure whether omAlloc or one of the other omAlloc* commands is more appropriate (e.g., omAlloc0). Anyone know?

1. I suppose this has to be freed somewhere... I know there's an obvious-sounding place, so I'll find it.

There are other places this change has to occur. One more question:

• A few lines before the ones I cited above is the test for 64-bits. I've switched to jdemeyer's test (sizeof(long)>4). However, the code in the patch is exactly the same for both 32- and 64-bit. Is there a recommendation for what to do here? I'm not sufficiently familiar with this to know what to do.

[john_perry]

Replying to john_perry:

Meanwhile, I discovered some memory-related bugs in the patch. For instance, ring.pyx has the line

assert( exponent == base_ring.degree() )

which causes a failure. I'm not sure why those should be equal (the test I used had Zmod(2^3) where exponent==3 and base_ring.degree()==1)...

To wit, the source code for base_ring.degree() is, in total,

    def degree(self):
        """
        Return 1.

        EXAMPLE::

            sage: R = Integers(12345678900)
            sage: R.degree()
            1
        """
        return integer.Integer(1)

I don't know what is meant by the degree of a ring here, so I'll need guidance. For now I've commented it out.

[john_perry]

Pushed code should build on 6.7. Mac users may have problems; if so, delete local/bin/singular and then sage-spkg singular; it should work after that. (In fact, I have to do it again myself now.) I have to run to the train (leaving SD) or I'd do it before leaving.

---

New commits:

​bdb0851	first attempt to resolve
​1f4ebcb	fix for _param and discrepancy between exponent and degree

[john_perry]

Forgot to remove that bad patch; that will come in a subsequent push. Still have the problem with local/bin/Singular because of the local symlink; it's a problem with the package itself, because I deleted both symlink and local copy before running the install. I don't know how to fix that, so advice would be appreciated.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​9aa2456

delete stricmp.patch

[jdemeyer]

Replying to john_perry:

However, the code in the patch is exactly the same for both 32- and 64-bit. Is there a recommendation for what to do here?

Obviously, replace

if condition:
    X
else:
    X

by

X

[jdemeyer]

Replying to john_perry:

• A few lines before the ones I cited above is the test for 64-bits. I've switched to jdemeyer's test (sizeof(long)>4). However, the code in the patch is exactly the same for both 32- and 64-bit. Is there a recommendation for what to do here? I'm not sufficiently familiar with this to know what to do.

Look at the diff for the old corresponding code: this doesn't have the branch.

[jdemeyer]

I would also recommend you to keep the logic of the old code where applicable. For example, the old code started with

if ch.is_power_of(2):
exponent = ch.nbits() -1
    # it seems Singular uses ints somewhere
    # internally, cf. #6051 (Sage) and #138 (Singular)
    if exponent <= 30:

and I see no reason to change this.

[chapoton]

naive rebase.

The spkg is no longer available from boxen..

---

New commits:

​ab68eb0

Merge branch 'u/john_perry/ticket.17254.squashed' into 6.9.b0

[jakobkroeker]

The [singular 4.0.1] spkg is no longer available from boxen..

That is unfortunate. @jdemeyer/@jpflori: could you upload the singular 4.0.1.p1 tarball to somewhere else?

naive rebase. [by chapoton] Conflicts:

src/sage/libs/singular/singular-cdefs.pxi

the file src/sage/libs/singular/singular-cdefs.pxi is even missing in the ticket branch public/ticket/17254

@chapoton Did you set the ticket branch to broken public/ticket/17254 instead of leaving it 'u/john_perry/ticket.17254.squashed' by accident?

[jpflori]

It seems I don't have access to boxen anymore...

[jpflori]

And I don't have the p1 tarball Jeroen posted.

[fbissey]

I have a tarball answering that description. Up on the lmona.de server....

​http://www.lmona.de/files/sage/singular-4.0.1p1.tar.bz2

[jakobkroeker]

bumped to recent sage dvelopment version.

The build may fail with 'undefined ..' error again. Probably this can be fixed with ./sage -ba; if not, make distclean-bazooka should work...

---

New commits:

​8e259d1	merged with recent develop
​aa40c90	check for local tarball

[jakobkroeker]

@chapoton

naive rebase.

except for the issue with src/sage/libs/singular/singular-cdefs.pxi, how did you do the rebase? Did you a rebase by hand, or did you create a patch after merge and then apply the patch to 6.9.b0?

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​72ef927

squashed singular upgrade changes in one commit

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​4adc75d

squashed singular upgrade changes in one commit

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​f492ef2

squashed singular upgrade changes in one commit

[jakobkroeker]

@john_perry thanks for the bugfix!

I am willing to work on this

somehow I overlooked your messages in my emails in the past.Are you still available for help? Maybe we could work together on the remaining issues? (skype?)

An example of creating extension fields can be found in Singulars libpolys/tests/polys_test.h :: test_Q_Ext_a()) while the basic data structure idea is sketched in libpolys/polys/ext_fields/algext.h

[john_perry]

Replying to jakobkroeker:

@john_perry somehow I overlooked your messages in my emails in the past.Are you still available for help? Maybe we could work together on the remaining issues? (skype?)

I'm willing to collaborate on it, but I've had a more pressing project which has consumed my attention/free time. I thought I'd be done with it months ago :-) so I can't promise to be very helpful and/or focus on this at the moment, but I do hope to be available from time to time, until the current project passes in which case I could focus on this a little bit.

[jakobkroeker]

New commits:

​90cefc5

bugfixes

[jakobkroeker]

-added support for extensions over QQ; fixed some bugs of mine

open tasks: -put back si2sa_GFqGivaro, si2sa_GFqNTLGF2E, si2sa_GFq_generic

-and fix failing tests - a lot of tests (about 100) still fail:

naMap00() call in singular.pyx crashes - either I use it incorrectly or there is a bug in Singular.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​b4d0495

removed printouts

[jakobkroeker]

repackaged latest Singular

​https://www.dropbox.com/s/3io9k5srkcjrysr/singular-4.0.2.latest.tar.bz2?dl=0

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​5e14c77

updated to latest singular

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​179ecd8

support building singular in debug mode

[jakobkroeker]

@malb

while upgrading to Singular 4 I'm currently hit a segfault in Singular for

sage: x = var('x')
sage: K.<z> = QQ.extension(x^2 + 1)
sage: P.<v,w> = K[]
sage: P._singular_() 
sage: P(1) # crash

Since I neither familiar enough with the sage interface to Singular nor with Singular internals, I'm currently stuck. Could you take a look?

[malb]

So far, I have no luck even building this. Did you see this before:

ImportError: …/sage-review/local/lib/python2.7/site-packages/sage/matrix/matrix_mpolynomial_dense.so: undefined symbol: _Z17initCanonicalFormv

[jakobkroeker]

So far, I have no luck even building this. Did you see this before: ImportError?: …/sage-review/local/lib/python2.7/site-packages/sage/matrix/matrix_mpolynomial_dense.so: undefined symbol: _Z17initCanonicalFormv

Yes. These are clearly artifacts from singular 3.1.7. In Singular 4 is no 'initCanonicalForm'.

Either this is a caching issue, incorrect dependency issue(in sage/cython) or somewhere are files from 3.1.7 left. On which OS do you build singular( See also comment 55 )? Did you try 'make distclean'? ( If you have a better solution for this, please share it)

[jdemeyer]

Replying to jakobkroeker:

Either this is a caching issue, incorrect dependency issue(in sage/cython) or somewhere are files from 3.1.7 left.

If it is a dependency issue (which looks the most likely), please make a new ticket for it.

[malb]

Indeed, building from scratch made it go away … this will be fun to deal with eventually.

[malb]

Hi, I have rebuild Singular + Sage extension with SAGE_DEBUG=yes and it seems trouble starts a few lines earlier in your minimal example:

sage: x = var('x')
sage: K.<z> = QQ.extension(x^2 + 1)
sage: P.<v,w> = K[]
***omError_WrongSize: wrong size specification of addr                                                                                                                                                           [677/4609]
 occured at: 
  #0 at ??:0                 in ??                  
 occured for addr:0x7f7c9b303190 size:32 specified size:24

// ***dPolyReportError: memory error 
 occured at
 occured for poly: z2+1  addr:0x7f7c9b303190 size:32
***omError_WrongSize: wrong size specification of addr
 occured at: 
  #0 at ??:0                 in ??                  
 occured for addr:0x7f7c9b303190 size:32 specified size:24

// ***dPolyReportError: memory error 
 occured at
 occured for poly: !!longrat: NULL in monomials/p_polys.cc:3598
!!longrat: NULL in monomials/p_polys.cc:3598
oz2+oz2+...  addr:0x7f7c9b303190 size:32
***omError_WrongSize: wrong size specification of addr
 occured at: 
  #0 at ??:0                 in ??                  
 occured for addr:0x7f7c9b303190 size:32 specified size:24

// ***dPolyReportError: memory error 
 occured at
 occured for poly: !!longrat: NULL in monomials/p_polys.cc:3598
!!longrat: NULL in monomials/p_polys.cc:3598
oz2+oz2+...  addr:0x7f7c9b303190 size:32
***omError_WrongSize: wrong size specification of addr
 occured at: 
  #0 at ??:0                 in ??                  
 occured for addr:0x7f7c9b303190 size:32 specified size:24

// ***dPolyReportError: memory error 
 occured at
 occured for poly: !!longrat: NULL in monomials/p_polys.cc:3598
!!longrat: NULL in monomials/p_polys.cc:3598
oz2+oz2+...  addr:0x7f7c9b303190 size:32
***omError_WrongSize: wrong size specification of addr
 occured at: 
  #0 at ??:0                 in ??                  
 occured for addr:0x7f7c9b303190 size:32 specified size:24

// ***dPolyReportError: memory error 
 occured at
 occured for poly: !!longrat: NULL in monomials/p_polys.cc:3598
!!longrat: NULL in monomials/p_polys.cc:3598
oz2+oz2+...  addr:0x7f7c9b303190 size:32

[jakobkroeker]

Replying to malb:

Hi, I have rebuild Singular + Sage extension with SAGE_DEBUG=yes and it seems trouble starts a few lines earlier in your minimal example:

sage: x = var('x')
sage: K.<z> = QQ.extension(x^2 + 1)
sage: P.<v,w> = K[]
***omError_WrongSize: wrong size specification of addr [...] 

Do you know in detail what happens or should happen in sage for

sage: P.<v,w> = K[]
sage: P(1) 

behind the scenes? With that information we could ask Singular's developers if we use libSingular correctly.

[malb]

It'll construct a ring for the extension and then a ring in in v, w. I'll try to investigate further what's going wrong there (sorry, I'm being slow on this)

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7803030

located algext issue

[jakobkroeker]

Replying to malb:

It'll construct a ring for the extension and then a ring in in v, w. I'll try to investigate further what's going wrong there (sorry, I'm being slow on this)

trouble starts a few lines earlier in your minimal example:

Indeed. With printouts I tracked down the issue to ring creation in src/sage/libs/singular/ring.pyx I guess that I incorrectly assembly the minpoly in ring construction (omalloc magic or whatever) Or there is an issue with the AlgExtInfo declaration or in nInitChar() call.

Could you take a look or even fix it? See the corresponding Singular example test_Q_Ext_a() in latest/libpolys/tests/polys_test.h

[jdemeyer]

Replying to jakobkroeker:

So far, I have no luck even building this. Did you see this before: ImportError?: …/sage-review/local/lib/python2.7/site-packages/sage/matrix/matrix_mpolynomial_dense.so: undefined symbol: _Z17initCanonicalFormv

Yes. These are clearly artifacts from singular 3.1.7. In Singular 4 is no 'initCanonicalForm'.

The problem was with the include path containing old files, this is now fixed.

---

New commits:

​1f7f7f1

Fix Singular include path

[jpflori]

Any progress Martin? I'll rebuild a debug version of Sage and give it a try as well in the next few days.

[jpflori]

I looked a little bit into the problem and from what I see when the multivariate gets created the first thing done is to create the number field in Singular. Just after the number field creation, the first memory error Singular complains about is related to the polynomial defining the number field, so I would say the number field creation already went wrong (or the number field is wrongly used once created).

[jpflori]

Here is the backtrace of the first error reporting:

(gdb) bt
#0  omReportError (error=omError_WrongSize, report_error=omError_NoError,
    r=0x3fffa170be00 <omTestBinAddrSize+68>, fmt=0x3fffa171bd80 "")
    at omError.c:83
#1  0x00003fffa17120b0 in omReportAddrError (error=omError_WrongSize,
    report_error=omError_NoError, addr=0x3fffa13001f0, bin_size=0x18,
    flags=262, r=0x3fffa170be00 <omTestBinAddrSize+68>, fmt=0x3fffa171bd80 "")
    at omDebugCheck.c:409
#2  0x00003fffa1710d5c in omDoCheckBinAddr (addr=0x3fffa13001f0,
    bin_size=0x18, flags=262, level=1 '\001', report=omError_NoError,
    r=0x3fffa170be00 <omTestBinAddrSize+68>) at omDebugCheck.c:211
#3  0x00003fffa17108b8 in omDoCheckAddr (addr=0x3fffa13001f0, bin_size=0x18,
    flags=262, level=1 '\001', report=omError_NoError,
    r=0x3fffa170be00 <omTestBinAddrSize+68>) at omDebugCheck.c:169
#4  0x00003fffa170fd2c in _omCheckAddr (addr=0x3fffa13001f0, size_bin=0x18,
    flags=262, check=1 '\001', report=omError_NoError,
    r=0x3fffa170be00 <omTestBinAddrSize+68>) at omDebugCheck.c:46
#5  0x00003fffa170cd8c in _omDebugAddr (addr=0x3fffa13001f0, bin_size=0x18,
    flags=258, check=1 '\001') at omDebug.c:283
#6  0x00003fffa170be00 in omTestBinAddrSize (addr=0x3fffa13001f0, size=24, 
    check_level=1) at omDebug.c:46
#7  0x00003fffa1b64c14 in _p_Test (p=0x3fffa13001f0, r=0x3fffa13344d0, level=0)
    at pDebug.cc:223
#8  0x00003fffa1c1fcd0 in naInitChar (cf=0x3fffa131c9a8, 
    infoStruct=0x3fffffff9240) at ext_fields/algext.cc:1419
#9  0x00003fffa1c5b114 in nInitChar (t=n_algExt, parameter=0x3fffffff9240)
    at numbers.cc:385
#10 0x00003fffa0f81758 in __pyx_f_4sage_4libs_8singular_4ring_singular_ring_new
...
}

[jpflori]

And of the second one:

(gdb) bt
#0  dPolyReportError (p=0x3fffa13001f0, r=0x3fffa13344d0,
    fmt=0x3fffa1cb5140 "%s ") at pDebug.cc:46
#1  0x00003fffa1b64c54 in _p_Test (p=0x3fffa13001f0, r=0x3fffa13344d0, level=0)
    at pDebug.cc:223
#2  0x00003fffa1c1fcd0 in naInitChar (cf=0x3fffa131c9a8,
    infoStruct=0x3fffffff9240) at ext_fields/algext.cc:1419
#3  0x00003fffa1c5b114 in nInitChar (t=n_algExt, parameter=0x3fffffff9240)
    at numbers.cc:385
#4  0x00003fffa0f81758 in __pyx_f_4sage_4libs_8singular_4ring_singular_ring_new
...
}

[jpflori]

Some pointers, I have to quit now and tomorrow I'll have forgotten everything:

• ​https://github.com/hannes14/Singular/blob/master/HOWTO-libsingular
• ​https://github.com/wbhart/Nemo.jl/wiki/Accessing-Singular-from-Julia

[jpflori]

Got a suggestion from Hannes:

• ​https://groups.google.com/d/msg/libsingular-devel/4JRHxVXIF70/r4aXry32BQAJ

Additional issue:

• we should clean up old headers when upgrading.

[jpflori]

The above suggestion fixes the problem.

[jpflori]

I still get a bunch of crashes/errors when testing sage/libs/singular, working on it.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​bea5a27

Correctly initialize Singular polynomials.

[jakobkroeker]

I still get a bunch of crashes/errors when testing sage/libs/singular, working on it.

I was able to fix the issue with naSetMap() and nlInit2gmp() usage in sa2si_NF()) and this reduces the number of segfaults significantly The example

sage: x = var('x')
sage: K.<z> = QQ.extension(x^2 + 1)
sage: P.<v,w> = K[]
sage: P._singular_() 
sage: P(2/3) 

does not crash any more!

A remaining task is to reuse the plain polynomial ring qqr = QQ['x'] over Q in sa2si_NF() instead of rebuilding it everytime. (variable name is irrelevant)

. . Next interesting riddle in sa2si_ZZmod(); simple failing example:

sage: sage: R.<a> = Zmod(5)['a', 'b']  
sage: R(1) #crash

---

New commits:

​e6a30ff

fixed issue with nMapFuncPtr- usage

[jpflori]

New commits:

​71d1873	Merge remote-tracking branch 'trac/develop' into singular
​d252948	Merge remote-tracking branch 'trac/develop' into singular

[jpflori]

The segfault is due to a division by zero (happening in gmp which aborts I guess):

#0  0x00003fffadbeaf28 in __gmpz_mod (rem=0x3fffa03ce230,
    dividend=0x3fffa03ce220, divisor=0x0) at mod.c:29
#1  0x00003fffa0d1558c in nrnMapGMP (from=0x3fffa03ce220, dst=0x3fffa03bc9a8)
    at rmodulon.cc:766
#2  0x00003fffa0d1561c in nrnMapZ (from=0x3fffa03ce220, src=0x3fffa03bc680,
    dst=0x3fffa03bc9a8) at rmodulon.cc:790
#3  0x00003fffa014903c in __pyx_f_4sage_4libs_8singular_8singular_sa2si_ZZmod (
    __pyx_v_d=0x3fff986212e0, __pyx_v__ring=0x3fffa03d43a8)

I guess the ring is wrongly initialized.

[jpflori]

Note that the same thing works when 5 is replaced by anything non-prime. And indeed during ring creation the former is caught at the beginning of the method in sage/libs/singular/ring.pyx as field, finite and prime whereas the latter is caught as integer_mod_ring.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​92304e8

Properly initial prime fields in singular.

[jdemeyer]

Why does this branch change build/bin/sage-spkg???

[jpflori]

No idea, mistake from me surely...

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​97c8bb6

Revert "Properly initial prime fields in singular."

[jpflori]

Ok the issue is indeed a mix between Zmod and GF. The ring was initially created in singular as a "finite field" but then for some reason the conversion is made as for a "modular integer". The former one uses ints whereas the second one uses mpzs.

[jpflori]

I've not checked but I guess the reason is as follow:

• to create the singular ring we ask if the base ring is a prime finite field, not if it's a class derived from FiniteField, so classes derived from IntegerModRing are caught,
• when sending something into the ring we follow the "classes" logic, boom.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​18a6be1

Fix for the GF vs Zmod issue with Singular.

[jpflori]

Ok next issue: napoly and slnumber do not exist anymore, seemingly since a long time...

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​bf2aede

WIP to use new Singular API.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​2ecb686

Fix several doctest failures in libs/singular

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​e949c00

Fix error message formatting

[jpflori]

New commits:

​15d5f12	Fix usage of algebraic extension as base rings in libsingular.
​d3566cb	Merge remote-tracking branch 'trac/u/jdemeyer/ticket/17254' into singular

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​8c02754

Use new Singular overflow limits.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​cab5fc5

groebner_norm Singular function does not exist, groebner does.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​35b9919

Fix some order changes and normalization changes in Singular output.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​ee62dcd

Yet another (doubtful) different normalization (-1 == 2 mod 3).

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​31c98df

New uniform overflow limit for Singular exponents.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​d7a3f36

singclap_isSqrFree was removed from Singular 4.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​b610f9d

Another overflow in Singular related change.

[jdemeyer]

New commits:

​d7fd9ba

Better overflow check

[jpflori]

Strange way to deal with floats accuracy, see: ​https://groups.google.com/forum/#!topic/libsingular-devel/kj3KlcrcNo8

Now looking into the weighted degree issue.

---

New commits:

​bf9ec9d	Change in doctest now that Integers(2)[x, y, ...] uses Singular's Zn type.
​a0a71c9	Merge remote-tracking branch 'trac/u/jdemeyer/ticket/17254' into singular
​a5b45a3	Yet others overflow related changes.

[jpflori]

Note for myself: the issue is that at some point r->pFDeg gets replaced from p_WTotalDegree by p_TotalDegree. Don't know why yet.

[jpflori]

This is triggered here:

  if ((r->pFDeg==p_WTotaldegree) && rOrd_is_MixedDegree_Ordering(r))
    r->pFDeg = p_Totaldegree;

line 3837/8 of ring.cc.

[jpflori]

Made by:

• ​https://github.com/Singular/Sources/commit/d711b46eb29f241f3854ae99e8451d3beb7f96e4

Maybe the 0 in the weights makes think Singular the order is useless.

[jpflori]

Question to upstream here:

• ​https://groups.google.com/forum/?fromgroups=#!topic/libsingular-devel/jregxl-k8Mw

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​199169e

Forgotten overflow stuff + use user defined degree.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​69ea9a4

Order changed in Singular output.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7b5bb32

Use correct function for degree.

[jakobkroeker]

Replying to git:

Branch pushed to git repo; I updated commit sha1. New commits:

​7b5bb32

Use correct function for degree.

I get the following errors for the last change:

Error compiling Cython file:
------------------------------------------------------------
...
    deg = 0
    if p == NULL:
        return -1
    if(r != currRing): rChangeCurrRing(r)
    if x == NULL:
       return r.pDeg(p,&deg,r)
              ^
------------------------------------------------------------

sage/libs/singular/polynomial.pyx:534:15: Object of type 'ring' has no attribute 'pDeg'

Error compiling Cython file:
------------------------------------------------------------
...
    deg = 0
    if p == NULL:
        return -1
    if(r != currRing): rChangeCurrRing(r)
    if x == NULL:
       return r.pDeg(p,&deg,r)
                      ^
------------------------------------------------------------

sage/libs/singular/polynomial.pyx:534:23: Cannot convert 'poly *' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
    deg = 0
    if p == NULL:
        return -1
    if(r != currRing): rChangeCurrRing(r)
    if x == NULL:
       return r.pDeg(p,&deg,r)
                       ^
------------------------------------------------------------

sage/libs/singular/polynomial.pyx:534:24: Cannot convert 'int *' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
    deg = 0
    if p == NULL:
        return -1
    if(r != currRing): rChangeCurrRing(r)
    if x == NULL:
       return r.pDeg(p,&deg,r)
                             ^

[jpflori]

Yes I guess I made a typo and forgot to push the latest commit.

[jpflori]

Or forgot to add the function prototype.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​5c3c320

Add declaration for Singular pDeg.

[jpflori]

Hopefully better now although I did not test the fix.

From what I remember the next issue was a segfault when letterplace stuff elements were multiplied.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​2b76279

Correct fix for the degree in singular.

[jpflori]

Problem with letterplace algebra:

sage: F.<x> = FreeAlgebra(ZZ, implementation='letterplace')
sage: x**2
sage: x**2
sage: x**2

[jpflori]

I seem to remember there was a double free.

[jpflori]

Did you manage to build the branch and reproduce the letterplace error?

[tscrim]

I rebased the branch (as best as I could), and I did get an error with letterplace:

sage: F.<x> = FreeAlgebra(ZZ, implementation='letterplace')
sage: x**2
0

I am also getting a variety of failures in testing the rings/polynomial folder, but most of them seem to be related to constructing ideals. I haven't run tests elsewhere.

---

New commits:

​ef0b4e0

Rebase #17254 on 7.1.rc0.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​dca98ec

Fixing an issue with my rebase.

[jakobkroeker]

Replying to jpflori:

Problem with letterplace algebra:

sage: F.<x> = FreeAlgebra(ZZ, implementation='letterplace')
sage: x**2
sage: x**2
sage: x**2

when I compile the recent 'u/jpflori/ticket/17254' or 'u/tscrim/upgrade_singular-17254' with SAGE_DEBUG=yes on my 32 bit fedora22

  export SAGE_DEBUG=yes
  ./sage -f singular

I get an printout from Singular:

  pmLPshift: too big shift requested

the corresponding Singular source snippet is (look at 'shiftgb.cc')

 if (L+sh-1 > uptodeg)
  {
#ifdef PDEBUG
    PrintS("pmLPshift: too big shift requested\n");
#endif
    return(NULL); /* violation, 2check */
  }

This seems to be a bug, but the next question is, why do we get there and is this an error caused by a previous one or not.

[jakobkroeker]

remaining failing tests (probably not all related to Singular):

./sage -t src/sage/schemes/affine/affine_rational_point.py  # Killed due to abort
./sage -t src/sage/schemes/affine/affine_point.py  # Killed due to segmentation fault
./sage -t src/sage/schemes/affine/affine_morphism.py  # Killed due to segmentation fault
./sage -t src/sage/schemes/generic/algebraic_scheme.py  # Killed due to segmentation fault
./sage -t src/sage/schemes/elliptic_curves/isogeny_small_degree.py  # Killed due to abort
./sage -t src/sage/libs/cremona/newforms.pyx  # IOError in doctesting framework
./sage -t src/sage/algebras/letterplace/free_algebra_letterplace.pyx  # Bad exit: 14
./sage -t src/sage/algebras/letterplace/free_algebra_element_letterplace.pyx  # Timed out (and interrupt failed)
./sage -t src/sage/rings/polynomial/multi_polynomial_sequence.py  # Killed due to segmentation fault
./sage -t src/sage/rings/polynomial/multi_polynomial_ideal.py  # Killed due to segmentation fault
./sage -t src/sage/rings/polynomial/pbori.pyx  # 7 doctests failed
./sage -t src/sage/schemes/projective/projective_morphism.py  # 9 doctests failed
./sage -t src/sage/rings/polynomial/polynomial_element.pyx  # 1 doctest failed
./sage -t src/sage/rings/polynomial/plural.pyx  # 6 doctests failed
./sage -t src/sage/schemes/jacobians/abstract_jacobian.py  # 1 doctest failed
./sage -t src/sage/schemes/plane_curves/projective_curve.py  # 1 doctest failed
./sage -t src/sage/rings/polynomial/polynomial_singular_interface.py  # 1 doctest failed
./sage -t src/sage/rings/number_field/number_field_ideal.py  # 2 doctests failed
./sage -t src/sage/modular/modform_hecketriangle/abstract_space.py  # 1 doctest failed
./sage -t src/sage/rings/polynomial/infinite_polynomial_element.py  # 4 doctests failed
./sage -t src/sage/rings/finite_rings/finite_field_ext_pari.py  # 1 doctest failed
./sage -t src/doc/en/constructions/algebraic_geometry.rst  # 1 doctest failed
./sage -t src/sage/graphs/generic_graph.py  # 1 doctest failed
./sage -t src/sage/modular/modform_hecketriangle/graded_ring_element.py  # 7 doctests failed
./sage -t src/sage/rings/finite_rings/finite_field_givaro.py  # 1 doctest failed
./sage -t src/sage/algebras/free_algebra.py  # 12 doctests failed
./sage -t src/sage/rings/multi_power_series_ring_element.py  # 5 doctests failed
./sage -t src/sage/rings/polynomial/infinite_polynomial_ring.py  # 1 doctest failed
./sage -t src/sage/graphs/base/graph_backends.pyx  # 4 doctests failed
./sage -t src/sage/symbolic/relation.py  # 7 doctests failed
./sage -t src/sage/interfaces/singular.py  # 5 doctests failed
./sage -t src/sage/rings/quotient_ring_element.py  # 1 doctest failed
./sage -t src/sage/structure/element.pyx  # 1 doctest failed
./sage -t src/sage/schemes/product_projective/wehlerK3.py  # 17 doctests failed
./sage -t src/sage/categories/pushout.py  # 1 doctest failed
./sage -t src/sage/rings/quotient_ring.py  # 12 doctests failed
./sage -t src/sage/structure/sage_object.pyx  # 1 doctest failed

[jakobkroeker]

@jpflori

when using 'r.pLDeg()' instead of 'p_Deg()' in 'singular_polynomial_deg()' (src/sage/libs/singular/polynomial.pyx)

some test from above pass, for example:

./sage -t src/sage/schemes/affine/affine_point.py 
./sage -t src/sage/schemes/affine/affine_morphism.py 
./sage -t src/sage/rings/polynomial/pbori.pyx  
./sage -t  src/sage/rings/polynomial/infinite_polynomial_element.py

especially some segfaults disappear but then probably other passing tests get broken?

Which example did you have in mind when you changed 'r.pLDeg()' to 'p_Deg()' in 'singular_polynomial_deg()' (src/sage/libs/singular/polynomial.pyx)?

Maybe we should discuss with Hans his commit in case it breaks something

​https://groups.google.com/forum/#!topic/libsingular-devel/jregxl-k8Mw

[jpflori]

The plDeg function is a Singular internal function, made for ecart, and has some restrictions regarding the ordering it can be applied to (no negative weights or things like that). It gets constructed by Singular at initialization of the object from p_Deg.

What we want in Sage is the "usual" degree which is p_Deg.

Maybe the doctests were just wrong, i have no idea.

[jdemeyer]

Which branch are you using? The branch on this ticket seems to have multiple conflicts with the current develop.

[jdemeyer]

The conflicts seem to be because of #20386.

[jakobkroeker]

fixed some more issues.

A rebase on recent develop branch was too hard for me so I squashed everything in a new commit.

@jdemeyer I had to revert some of your changes in src/module_list.py (removing singular from include path) because I failed to do the same for Singular 4.

[jakobkroeker]

actually, in my recent branch the 'upgrade.patch' file is missing. No idea how that happened, but this is a fact.

I will fix that tomorrow or on friday. At the moment I'm just too tired.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​3312a17

upgrade.patch was not staged, staging it

[jdemeyer]

Replying to jakobkroeker:

A rebase on recent develop branch was too hard for me so I squashed everything in a new commit.

@jdemeyer I had to revert some of your changes in src/module_list.py

Both of these are quite bad. Do you want me to have a look?

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​cc868a0	rebase too hard for me. squashed everything again
​ff98a0b	upgrade.patch was not staged, staging it

[jakobkroeker]

Replying to jdemeyer:

Replying to jakobkroeker:

A rebase on recent develop branch was too hard for me so I squashed everything in a new commit.

@jdemeyer I had to revert some of your changes in src/module_list.py

Both of these are quite bad. Do you want me to have a look?

That would be good.

The latest u/jakobkroeker/ticket.17254.squashed.latest (just committed with force) should have no conflicts with develop/master branch

[jakobkroeker]

Question : why trac shows me that trac's automerging failed?

[jakobkroeker]

I experience several strange doctest failures, for example:

File "src/sage/modular/modform_hecketriangle/abstract_space.py", line 379, in sage.modular.modform_hecketriangle.abstract_space.FormsSpace_abstract.one_element
Failed example:
    MF.one_element()
Expected:
    doctest:...: DeprecationWarning: .one_element() is deprecated. Use .one() instead.
    See http://trac.sagemath.org/17694 for details.
    1 + O(q^5)
Got:
    doctest:1: DeprecationWarning: .one_element() is deprecated. Use .one() instead.
    See http://trac.sagemath.org/17694 for details.
      Running doctests with ID 2016-05-15-20-09-37-2e7003e6.
    1 + O(q^5)
*******************

any idea what is wrong here or how to fix it?

make distclean and ccache --clear did not help

[tscrim]

Is it based off develop or master? These are different as develop is the (released today) 7.3.beta2, whereas master is 7.2.

[tscrim]

I've seen some errors somewhat similar to that before. It is as if something in the doctesting output gets read with an offset to what it should. I forget exactly what I've done in the past to fix this, but I think it came from a specific type of doctest failure...

[jakobkroeker]

Replying to tscrim:

I've seen some errors somewhat similar to that before. It is as if something in the doctesting output gets read with an offset to what it should. I forget exactly what I've done in the past to fix this, but I think it came from a specific type of doctest failure...

are you able to reproduce some of these (false-positive) doctest failures?

[jakobkroeker]

Replying to tscrim:

Is it based off develop or master? These are different as develop is the (released today) 7.3.beta2, whereas master is 7.2.

Based on develop (​https://github.com/sagemath/sage) on my machine it merges without conflicts with master and develop branch.

[tscrim]

Replying to jakobkroeker:

Replying to tscrim:

Is it based off develop or master? These are different as develop is the (released today) 7.3.beta2, whereas master is 7.2.

Based on develop (​https://github.com/sagemath/sage) on my machine it merges without conflicts with master and develop branch.

The automerging that is used on trac is not as good as what local versions of git can do.

As for comment:170, I haven't testing this branch yet.

[jdemeyer]

There was a merge conflict with Sage 7.3.beta2, I fixed it.

---

New commits:

​a80b3a0

Merge tag '7.3.beta2' into t/17254/ticket.17254.squashed.latest

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​e8fbe03

Upgrade to Singular-4.0.2

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​5b8343b

Upgrade to Singular-4.0.2

[jdemeyer]

I cleaned up the patch and removed some unacceptable parts. Right now it doesn't build, that still needs to be fixed.

[jdemeyer]

I think it's best to discuss with upstream about the crazy #include directory structure. It was already bad in Singular 3 and it seems to have gotten worse with Singular 4.

[jakobkroeker]

Replying to jdemeyer:

I cleaned up the patch and removed some unacceptable parts. Right now it doesn't build, that still needs to be fixed.

what has to be done to get it compiling?

[jdemeyer]

I am working on a hack to fix the includes, hang on...

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​1426ad8

Fix singular include directories

[jakobkroeker]

Replying to jdemeyer:

I cleaned up the patch and removed some unacceptable parts. Right now it doesn't build, that still needs to be fixed.

Are these the unacceptable parts? (e.g in src/sage/rings/polynomial/polydict.pyx)

diff --git a/src/sage/rings/polynomial/polydict.pyx b/src/sage/rings/polynomial/polydict.pyx
index fd858f7..0b49f38 100644
--- a/src/sage/rings/polynomial/polydict.pyx
+++ b/src/sage/rings/polynomial/polydict.pyx
@@ -506,7 +506,7 @@ cdef class PolyDict:
         don't put negative signs on the generators. ::
 
             sage: Integers(2)['x,y'].gens()
-            (1*x, 1*y)
+            (x, y)
 
         We make sure that intervals are correctly represented. ::
 

The 1*x output stems from Singular4 and that seems the new behaviour for that ring, for whatever reasons.

same for

diff --git a/src/sage/rings/multi_power_series_ring_element.py b/src/sage/rings/multi_power_series_ring_element.py
index 7293a91..7b9ca6e 100644
--- a/src/sage/rings/multi_power_series_ring_element.py
+++ b/src/sage/rings/multi_power_series_ring_element.py
@@ -32,7 +32,7 @@ Power series arithmetic, tracking precision::
     sage: f*=s; f
     s + 2*s^2 + 3*s^3 + O(s, t)^8
     sage: f%2
-    1*s + 1*s^3 + O(1*s, 1*t)^8
+    s + s^3 + O(s, t)^8
     sage: (f%2).parent()
     Multivariate Power Series Ring in s, t over Ring of integers modulo 2
 
@@ -1078,7 +1078,7 @@ class MPowerSeries(PowerSeries):
             sage: R.<a,b,c> = PowerSeriesRing(ZZ)
             sage: f = -a^3*b*c^2 + a^2*b^2*c^4 - 12*a^3*b^3*c^3 + R.O(10)
             sage: g = f % 2; g
-            1*a^3*b*c^2 + 1*a^2*b^2*c^4 + O(1*a, 1*b, 1*c)^10
+            a^3*b*c^2 + a^2*b^2*c^4 + O(a, b, c)^10
             sage: g in R
             False
             sage: g in R.base_extend(Zmod(2))

also floats are now put in braces, maybe because otherwise parsing the string '1.7E+30*5' in Singular would be wronng or fail? (just guessing)

so removing the following hack

--- a/src/sage/rings/real_double.pyx
+++ b/src/sage/rings/real_double.pyx
@@ -689,21 +689,6 @@ cdef class RealDoubleElement(FieldElement):
             sage: RDF(10^100)
             1e+100
         """
-        if isinstance(x,basestring):
-            # a hack for parsing float strings beginning with braces
-            if len(x)>2 and x.count('(')==1 and  x.count(')')==1:
-                try:
-                    if (x[0] == '(' or x[1] == '(') and x[-1] == ')':
-                        if (x[0] == '-' and x[1] == '(') :
-                            x = x[2:-1]
-                            self._value = - float(x)
-                            return
-                        else:
-                            x = x[1:-1]
-                            self._value = float(x)
-                            return
-                except:
-                    pass
         self._value = float(x)
 
     def _magma_init_(self, magma):

will result in a lot of failing doctests...

[jdemeyer]

It's not because Singular prints something in a certain way that Sage has to print it that way. The correct fix is changing the way how Sage prints Singular objects then.

[jdemeyer]

I added a fix for the include file issue.

[jakobkroeker]

Replying to jdemeyer:

I added a fix for the include file issue.

Thanks!

[jakobkroeker]

Replying to jdemeyer:

It's not because Singular prints something in a certain way that Sage has to print it that way. The correct fix is changing the way how Sage prints Singular objects then.

At least we should understand the reasons

• for 1*x
• and for braces around floats

I do not know whether it is safe to remove the 1* and the braces around floats. I don't know what Sage does for example with _repr_(), (which calls singular_polynomial_str() which in turn calls p_String

And if it' safe, how we should do that? regexping each polynomial string on the fly?

[jdemeyer]

Replying to jakobkroeker:

regexping each polynomial string on the fly?

If that works, why not?

[jakobkroeker]

remaining failing tests:

./sage -t --long src/sage/arith/misc.py  # 1 doctest failed
./sage -t --long src/sage/modular/modform_hecketriangle/graded_ring_element.py  # 7 doctests failed
./sage -t --long src/sage/interfaces/singular.py  # 1 doctest failed
./sage -t --long src/sage/categories/pushout.py  # 1 doctest failed
./sage -t --long src/sage/quivers/algebra_elements.pyx  # Timed out (and interrupt failed)
./sage -t --long src/sage/algebras/free_algebra.py  # 12 doctests failed
./sage -t --long src/sage/algebras/letterplace/free_algebra_letterplace.pyx  # Bad exit: 14
./sage -t --long src/sage/algebras/letterplace/letterplace_ideal.pyx  # Killed due to abort
./sage -t --long src/sage/algebras/letterplace/free_algebra_element_letterplace.pyx  # Killed due to segmentation fault
./sage -t --long src/sage/rings/quotient_ring_element.py  # 1 doctest failed
./sage -t --long src/sage/rings/ring.pyx  # Killed due to abort
./sage -t --long src/sage/rings/quotient_ring.py  # 12 doctests failed
./sage -t --long src/sage/rings/multi_power_series_ring_element.py  # 3 doctests failed
./sage -t --long src/sage/rings/polynomial/infinite_polynomial_ring.py  # 1 doctest failed
./sage -t --long src/sage/rings/polynomial/multi_polynomial_sequence.py  # Killed due to segmentation fault
./sage -t --long src/sage/rings/polynomial/plural.pyx  # 6 doctests failed
./sage -t --long src/sage/rings/polynomial/multi_polynomial_libsingular.pyx  # 2 doctests failed
./sage -t --long src/sage/rings/polynomial/polynomial_singular_interface.py  # 1 doctest failed
./sage -t --long src/sage/rings/polynomial/symmetric_ideal.py  # 12 doctests failed
./sage -t --long src/sage/rings/polynomial/infinite_polynomial_element.py  # 4 doctests failed
./sage -t --long src/sage/rings/polynomial/polydict.pyx  # 1 doctest failed
./sage -t --long src/sage/rings/polynomial/multi_polynomial_ideal.py  # Killed due to segmentation fault
./sage -t --long src/sage/schemes/projective/projective_morphism.py  # Killed due to segmentation fault
./sage -t --long src/sage/schemes/plane_curves/projective_curve.py  # 1 doctest failed
./sage -t --long src/sage/schemes/affine/affine_rational_point.py  # Killed due to abort
./sage -t --long src/sage/schemes/affine/affine_point.py  # Killed due to segmentation fault
./sage -t --long src/sage/schemes/affine/affine_morphism.py  # Killed due to segmentation fault
./sage -t --long src/sage/schemes/jacobians/abstract_jacobian.py  # 1 doctest failed
./sage -t --long src/sage/schemes/generic/algebraic_scheme.py  # Killed due to segmentation fault
./sage -t --long src/sage/schemes/elliptic_curves/isogeny_small_degree.py  # Timed out (with hangup after interrupt)
./sage -t --long src/sage/structure/element.pyx  # 1 doctest failed
./sage -t --long src/doc/en/constructions/algebraic_geometry.rst  # 1 doctest failed

any volunteers?

[jpflori]

Do you still get the letterplace segfault?

[jakobkroeker]

Replying to jpflori:

Do you still get the letterplace segfault?

I did not fix that one because I have no idea what FreeAlgebra? is neither I know something about letterplace implementation, so I did not even try.

[jakobkroeker]

New commits:

​df297d6

add meaninful messages to Singular error

remark: the patch above does not solve the issue with letterplace segfault

[jakobkroeker]

simplest failing example:

sage: A.<x> = AffineSpace(GF(3^2, 't'), 1) ## line 299 ##
Ideal(x).weil_restriction()

[jakobkroeker]

added debug printouts in

u/jakobkroeker/jdemeyer_ticket.17254.latest.debug

for the example

sage: F.<x> = FreeAlgebra(ZZ, implementation='letterplace')
sage: x**2

in free_algebra_element_letterplace.pyx, __pow__() routine after copying the polynomial from one ring to another (with prCopyR) something seems broken (look at the polynomial degree!)

and for the

sage: A.<x> = AffineSpace(GF(3^2, 't'), 1) ## line 299 ##
Ideal(x).weil_restriction()

after the subs() command in the weil_restriction()-Routine something is screwed up (look at the polynomial degree!)

@jpflori Any idea?

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​88ce71f

call rComplete for singular ring

[jakobkroeker]

p_Deg seems to have problems with lex ordering:

k = QQ
S.<a,b>=PolynomialRing(k, 2, {"a","b"}, order='lex')
a.degree() # 65536

k = QQ
S.<a,b>=PolynomialRing(k, 2, {"a","b"}, order='degrevlex')
a.degree() #1

[jakobkroeker]

@jpflori

Singulars degree function calls pLDeg and not p_Deg, (grep for 'jjDEG' in Singular sources), so we should do the same/similar by default. one of the reasons: p_Deg seems returning garbage for lexicographic orderings (see comment 195)

Question: is the behaviour described in the documentation of the 'degree' function in sage

Note that a matrix term ordering alters the grading of the generators
          of the ring; see the tests below.  To avoid this behavior, use either
          ``exponents()`` for the exponents themselves, or the optional
          argument ``std_grading=False``.

natural and is it what a user would expect? Then we could(?) fix it using a different appropriate call:

sage: sage: tord = TermOrder(matrix([3,0,1,1,1,0,1,0,0]))
sage: sage: R.<x,y,z> = PolynomialRing(QQ,'x',3,order=tord)
sage: sage: (x^3*y+x*z^4).degree() # 9
('p_Totaldegree', 4)
('p_WTotaldegree', 9)
('p_WDegree', 9)
('pFDeg', 4)
('pLDeg', 5)

[john_perry]

Replying to jakobkroeker:

Question: is the behaviour described in the documentation of the 'degree' function in sage

Note that a matrix term ordering alters the grading of the generators
          of the ring; see the tests below.  To avoid this behavior, use either
          ``exponents()`` for the exponents themselves, or the optional
          argument ``std_grading=False``.

natural and is it what a user would expect?

I bumped into this a long time ago. It shocked me then, but I adapted & now my code depends on it. If this is true of me, it is probably true of others.

This, for example, is from Sage 6.7:

sage: T = TermOrder(matrix(3,3,[1,2,3,3,4,5,0,0,1]))
sage: R.<x,y,z> = PolynomialRing(QQ,'x',3,order=T)
sage: y.degree()
2
sage: version()
'SageMath Version 6.7, Release Date: 2015-05-17'

So IMHO we should mimic Singular for compatibility reasons (i.e., we're mimicking Sage itself).

[tscrim]

Rebased on latest beta.

---

New commits:

​9ec5134

Merge branch 'u/jakobkroeker/ticket.17254.Jeroen_Demeyer.version' of git://trac.sagemath.org/sage into u/jakobkroeker/ticket.17254.Jeroen_Demeyer.version

[klee]

Replying to jakobkroeker:

Question: is the behaviour described in the documentation of the 'degree' function in sage

Note that a matrix term ordering alters the grading of the generators
          of the ring; see the tests below.  To avoid this behavior, use either
          ``exponents()`` for the exponents themselves, or the optional
          argument ``std_grading=False``.

natural and is it what a user would expect? Then we could(?) fix it using a different appropriate call:

sage: sage: tord = TermOrder(matrix([3,0,1,1,1,0,1,0,0]))
sage: sage: R.<x,y,z> = PolynomialRing(QQ,'x',3,order=tord)
sage: sage: (x^3*y+x*z^4).degree() # 9
('p_Totaldegree', 4)
('p_WTotaldegree', 9)
('p_WDegree', 9)
('pFDeg', 4)
('pLDeg', 5)

As the person who introduced the matrix term ordering into Sage, I think that it was just due to the Singular behavior that a matrix term ordering changes the grading of the generators according to the first row of the matrix in Sage. It shocked everyone including John, and was regarded as a bug. So if we can make the grading and the matrix term ordering independent, switching to Singular 4, then I would be happy. Of course it should go with an appropriate deprecation warning.

[john_perry]

Replying to klee:

Of course it should go with an appropriate deprecation warning.

+1

[mkoeppe]

Polymake (#20892) looks for a binary called libsingular-config. Is this something that will be installed by Singular 4?

[jakobkroeker]

Replying to mkoeppe:

Polymake (#20892) looks for a binary called libsingular-config. Is this something that will be installed by Singular 4?

yes, libsingular-config is a bash script in Singular 4

[leif]

This will hopefully build with GCC 6.x... (cf. #20738) :P

[jakobkroeker]

Currently I implemented singular_polynomial_deg as follows to mimic its behaviour as it was with Singular 3.1.7(and older)

    ...
    cdef long _deg, deg
    
    deg = -1
    _deg = -1 
    if x == NULL:
        while p:  
            _deg = p_WTotaldegree(p,r)
          
            if _deg > deg:
                deg = _deg
            p = pNext(p)

        return deg
    ...

The degree-related tests seem to pass now.

Remakr: pDeg() seems not usable/has different behaviour; see ​https://groups.google.com/forum/?_escaped_fragment_=topic/libsingular-devel/dulzKFTF3g8#!topic/libsingular-devel/dulzKFTF3g8 for details.

Further changes: added some missing rComplete() calls ( otherwise prCopyR cannot be used, for instance, see ​https://groups.google.com/forum/?_escaped_fragment_=topic/libsingular-devel/4JRHxVXIF70#!topic/libsingular-devel/4JRHxVXIF70 )

---

New commits:

​b1e228e	replacement for pLDeg from Singular 3 (behaviour of pLDeg changed in Singular 4)
​a511817	added missing rComplete calls
​738fe43	disable short output for Singular

[jakobkroeker]

only few issues are left;

who will fix the next one? that should be easy!

./sage -t --long src/sage/arith/misc.py  # 1 doctest failed
./sage -t --long src/sage/modular/modform_hecketriangle/graded_ring_element.py  # 7 doctests failed
./sage -t --long src/sage/quivers/algebra_elements.pyx  # Timed out (and interrupt failed)
./sage -t --long src/sage/algebras/letterplace/free_algebra_letterplace.pyx  # Killed due to segmentation fault
./sage -t --long src/sage/algebras/letterplace/letterplace_ideal.pyx  # Killed due to abort
./sage -t --long src/sage/algebras/letterplace/free_algebra_element_letterplace.pyx  # Timed out (and interrupt failed)
./sage -t --long src/sage/rings/ring.pyx  # Killed due to abort
./sage -t --long src/sage/rings/multi_power_series_ring_element.py  # 2 doctests failed
./sage -t --long src/sage/rings/polynomial/multi_polynomial_libsingular.pyx  # 2 doctests failed
./sage -t --long src/sage/rings/polynomial/polynomial_singular_interface.py  # 1 doctest failed
./sage -t --long src/sage/rings/polynomial/polydict.pyx  # 1 doctest failed
./sage -t --long src/sage/rings/polynomial/multi_polynomial_ideal.py  # 1 doctest failed
./sage -t --long src/sage/schemes/projective/projective_morphism.py  # Timed out (with hangup after interrupt)
./sage -t --long src/sage/schemes/affine/affine_rational_point.py  # Killed due to abort
./sage -t --long src/sage/schemes/jacobians/abstract_jacobian.py  # 1 doctest failed
./sage -t --long src/sage/schemes/elliptic_curves/isogeny_small_degree.py  # Killed due to segmentation fault
./sage -t --long src/sage/structure/element.pyx  # 1 doctest failed
./sage -t --long src/doc/en/constructions/algebraic_geometry.rst  # 1 doctest failed

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​3cb4304	explained si_opt_1 and si_opt_2
​ac8e614	fixed an upstream bug in brnoeth.lib and thus the failing test in src/sage/schemes/curves/projective_curve.py. TODO: upstream report/ upstream pull request

[jakobkroeker]

Failing of

sage: F.<x> = FreeAlgebra(ZZ, implementation='letterplace')
sage: x**2

is caused by an upstream bug; (reported: ​https://www.singular.uni-kl.de:8005/trac/ticket/767#ticket)

Letterplace over fields should work (please check)

[jpflori]

Replying to jakobkroeker:

Failing of

sage: F.<x> = FreeAlgebra(ZZ, implementation='letterplace')
sage: x**2

is caused by an upstream bug; (reported: ​https://www.singular.uni-kl.de:8005/trac/ticket/767#ticket)

Letterplace over fields should work (please check)

Thanks for devising this!

[jakobkroeker]

the upsream issue

pathed in ​http://git.sagemath.org/sage.git/commit/?id=ac8e614777f6e5e49149f6227d40ba1cc71585c9 (upgrade.patch)

is now reported on upstream with a pull request:

​https://www.singular.uni-kl.de:8005/trac/ticket/770#ticket

[jakobkroeker]

could someone please look what goes wrong with

./sage -t --long src/sage/modular/modform_hecketriangle/graded_ring_element.py

?

[jpflori]

I've removed a bunch of very old hackish stuff in spkg-install, especially regarding Solaris and sparc. Maybe too much, but the Singular build system changed so much and it does not seem we have real support for such systems anymore that we should only reintroduce removed hacks if found that are still needed.

It also seems this upgrade to git version rebroke some stuff...

---

New commits:

​637b2b7	Merge remote-tracking branch 'trac/u/jakobkroeker/tscrim.upgrade_singular-17254' into singular
​be4f5e7	Update Singular to 4.0.3p1.
​5b98496	Let's play with singular git version.
​3a5baec	Work with Singular git version.
​7e99c41	Use Singular git version.

[jpflori]

In particular, in function.pyx, in __init__ after calling

        if currRingHdl == NULL:
            currRingHdl = ggetid("my_awesome_sage_ring")
            if currRingHdl == NULL:
                currRingHdl = enterid("my_awesome_sage_ring", 0, RING_CMD, &IDROOT, 1)
            currRingHdl.data.uring.ref += 1

the last line segfaults because:

(gdb) print currRingHdl.data
$2 = {
  i = 0, 
  uring = 0x0, 
  p = 0x0, 
  n = 0x0, 
  uideal = 0x0, 
  umap = 0x0, 
  umatrix = 0x0, 
  ustring = 0x0, 
  iv = 0x0, 
  bim = 0x0, 
  l = 0x0, 
  li = 0x0, 
  pack = 0x0, 
  pinf = 0x0
}

I guess some additional funciton should be used to initialize properly currRingHdl.

[jpflori]

The culprit might be ​https://github.com/Singular/Sources/commit/595145c77dbdb42114960862678d5ca6351578be

[leif]

So everyone's always using the latest git version in spkg-src?

FWIW, sed -i isn't portable, e.g. won't work on Oraclis.

SPKG.txt now says "Patches: None" whereas there is upgrade.patch.

[jpflori]

That's just work in progress... This upgrade is hell.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​4a85552

enterid does not initialize underlying ring anymore, fix for function.pyx.

[leif]

Replying to jpflori:

That's just work in progress... This upgrade is hell.

Sure. So you'll (by default) check out a specific commit once you know which one is suited best?

[leif]

Just for the record:

make_tar.sh checks out HEAD, and calls autogen.sh (without checking for errors) which in turn runs (some random version of) autoreconf.

[jpflori]

FYI, explanation for the removal and readdition of nlDelete: ​https://groups.google.com/d/msg/libsingular-devel/EtEblVqdV3I/CyJJ5jaJCAAJ

[jpflori]

Replying to leif:

Replying to jpflori:

That's just work in progress... This upgrade is hell.

Sure. So you'll (by default) check out a specific commit once you know which one is suited best?

Sure! Hopefully it should be 4.1 upstream version though I don't know when it's planned. 4.0.3p1 seems to be a poorer choice as a bunch of bugs reported here where fixed in Singular after that release and the fixes would have to be backported...

[jpflori]

Replying to leif:

Just for the record:

make_tar.sh checks out HEAD, and calls autogen.sh (without checking for errors) which in turn runs (some random version of) autoreconf.

Yes and tries to copy some files from the person who wrote the script :)

[leif]

Replying to jpflori:

Replying to leif:

So you'll (by default) check out a specific commit once you know which one is suited best?

Sure! Hopefully it should be 4.1 upstream version though I don't know when it's planned.

I was just about to ask you that, as there are many recent commits mentioning 4.1.

I'll probably add something to spkg-src regarding checkout and autogen.sh / make_tar.sh unless you're going to as well.

[leif]

Replying to jpflori:

Replying to leif:

make_tar.sh checks out HEAD, and calls autogen.sh (without checking for errors) which in turn runs (some random version of) autoreconf.

Yes and tries to copy some files from the person who wrote the script :)

I just stumbled upon

cp /tmp/wawa-i/share/singular/LIB/all.lib  singular-$VERSION/Singular/LIB/.

Did you mean that, or is there more?

[leif]

After all, it's spielwiese (~= playground).

Is the repo on GitHub their "master" one, or do they (occasionally) just sync from elsewhere?

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​d0c7069

Update singular git version.

[jpflori]

Replying to leif:

Replying to jpflori:

Replying to leif:

make_tar.sh checks out HEAD, and calls autogen.sh (without checking for errors) which in turn runs (some random version of) autoreconf.

Yes and tries to copy some files from the person who wrote the script :)

I just stumbled upon

cp /tmp/wawa-i/share/singular/LIB/all.lib  singular-$VERSION/Singular/LIB/.

Did you mean that, or is there more?

There is also a non-existing fedora (IIRC) file.

[jpflori]

Replying to leif:

After all, it's spielwiese (~= playground).

Is the repo on GitHub their "master" one, or do they (occasionally) just sync from elsewhere?

I would say the spielwiese branch on github is the real master one. As you can see, the following commit: ​https://github.com/Singular/Sources/commit/b412eebf8435fb44680b5160c7f8d524cdb20b2b just followed my post on libsingular-devel: ​https://groups.google.com/forum/#!topic/libsingular-devel/EtEblVqdV3I

[jpflori]

Things seem good. A ptest yeilds:

sage -t --warn-long 121.7 src/doc/en/prep/Advanced-2DPlotting.rst  # Killed due 
to segmentation fault
sage -t --warn-long 121.7 src/sage/calculus/riemann.pyx  # Timed out (and interr
upt failed)
sage -t --warn-long 121.7 src/sage/schemes/elliptic_curves/isogeny_small_degree.
py  # Killed due to segmentation fault
sage -t --warn-long 121.7 src/sage/schemes/elliptic_curves/ell_number_field.py  
# Timed out
sage -t --warn-long 121.7 src/sage/libs/gap/all_documented_functions.py  # Timed
 out
sage -t --warn-long 121.7 src/sage/plot/plot.py  # Timed out
sage -t --warn-long 121.7 src/sage/libs/gap/assigned_names.py  # Timed out
sage -t --warn-long 121.7 src/sage/finance/time_series.pyx  # Timed out
sage -t --warn-long 121.7 src/sage/plot/plot3d/implicit_plot3d.py  # Timed out
sage -t --warn-long 121.7 src/sage/plot/plot3d/parametric_plot3d.py  # Timed out
sage -t --warn-long 121.7 src/sage/plot/plot3d/plot3d.py  # Timed out
sage -t --warn-long 121.7 src/sage/plot/graphics.py  # Timed out
sage -t --warn-long 121.7 src/sage/plot/plot3d/base.pyx  # Timed out
sage -t --warn-long 121.7 src/doc/en/prep/Calculus.rst  # Killed due to segmentation fault
sage -t --warn-long 121.7 src/sage/modular/local_comp/type_space.py  # Timed out
sage -t --warn-long 121.7 src/sage/modular/modform_hecketriangle/graded_ring_element.py  # 7 doctests failed
sage -t --warn-long 121.7 src/sage/schemes/projective/projective_morphism.py  # 4 doctests failed
sage -t --warn-long 121.7 src/sage/misc/randstate.pyx  # 1 doctest failed
sage -t --warn-long 121.7 src/sage/arith/misc.py  # 1 doctest failed
sage -t --warn-long 121.7 src/sage/structure/element.pyx  # 1 doctest failed
sage -t --warn-long 121.7 src/sage/doctest/test.py  # Timed out
sage -t --warn-long 121.7 src/sage/tests/french_book/mpoly.py  # 1 doctest failed
sage -t --warn-long 121.7 src/sage/rings/polynomial/multi_polynomial_ideal.py  # 4 doctests failed
sage -t --warn-long 121.7 src/sage/homology/simplicial_complex.py  # 1 doctest failed
sage -t --warn-long 121.7 src/sage/plot/misc.py  # 1 doctest failed
sage -t --warn-long 121.7 src/sage/interfaces/singular.py  # 3 doctests failed
sage -t --warn-long 121.7 src/sage/rings/polynomial/multi_polynomial_libsingular.pyx  # 4 doctests failedsage -t --warn-long 121.7 src/sage/schemes/affine/affine_rational_point.py  # Killed due to abort
sage -t --warn-long 121.7 src/sage/libs/singular/function.pyx  # 6 doctests failed
sage -t --warn-long 121.7 src/sage/rings/multi_power_series_ring_element.py  # 2 doctests failed
sage -t --warn-long 121.7 src/sage/rings/polynomial/polynomial_singular_interface.py  # 3 doctests failed
sage -t --warn-long 121.7 src/sage/rings/polynomial/multi_polynomial_ideal_libsingular.pyx  # 1 doctest failed  
sage -t --warn-long 121.7 src/sage/schemes/jacobians/abstract_jacobian.py  # 1 doctest failed
sage -t --warn-long 121.7 src/sage/rings/polynomial/polydict.pyx  # 1 doctest failed
----------------------------------------------------------------------

I assume many of them are unrelated to Singular, and due to the POWER7 arch, especially the plots ones.

[jpflori]

The singular related ones seem to be:

• schemes/elliptic_curves/isogeny_small_degree.py: segfault.
• modular/modform_hecketriangle/graded_ring_element.py: lots of singular stuff, including coercion.
• schemes/projective/projective_morphism.py: not sure... something about mpfr!
• arith/misc.py: representation of finite fields changed, easy to fix.
• structure/element.pyx: new limitation in max exponent?
• tests/french_book/mpoly.py: different result, might be just as good as the previous one.
• rings/polynomial/multi_polynomial_ideal.py: some different results, surely just as good and even better
• interfaces/singular.py: different Singular printing.
• rings/polynomial/multi_polynomial_libsingular.pyx: printing and squarefree stuff.
• schemes/affine/affine_rational_point.py: bad abort.
• src/sage/libs/singular/function.pyx: hopefully mostly random values changed in internal singular functions.
• rings/multi_power_series_ring_element.py: printing of power series changed.
• rings/polynomial/polynomial_singular_interface.py: printing changed.
• rings/polynomial/multi_polynomial_ideal_libsingular.pyx: different result, might be just as good.
• schemes/jacobians/abstract_jacobian.py: coercion error?
• rings/polynomial/polydict.pyx: printing of polys changed (x -> 1*x).

[leif]

Automake complains about LIBTOOL not being defined although libtool is used.

And finally I'm getting: curl: command not found... :-)

[jpflori]

Thread for the "1*" issue: ​https://groups.google.com/forum/#!topic/libsingular-devel/X29-H_bv9-4

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​a010eb5	New singular git version including gcc/gmp fix.
​9e14834	Merge remote-tracking branch 'trac/develop' into singular

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​62735be

Let's use Singular finite field struct for finite rings of prime char.

[jpflori]

As a sidenote, if we really want to stick to a finite ring structure rather than a finite field structure for Z/2Z and Z/pZ, we could do so. Hans fixed the "1*" disturbing behavior.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​2d84044	In Singular 4 limit for exponents is the same on 32 and 64 bit.
​75efae0	Let's stick with integer mod ring in Singular at the moment.
​919165a	Integer mod ring printing is not centered anymore in Singular.
​ace4d54	Spurious dot.
​cc921ec	Singular printing changes.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​50ed3b3	Warning disappeared in Singular (better implem?).
​a39f146	Different but (hopefully) equivalent Groebner basis.
​6c88bc3	Singular now yields same result for ZZ and QQ.

[jpflori]

For the segfault in schemes/elliptic_curves/isogeny_small_degree.py it seems that the following commit now triggers a double free or invalid free (use MALLOC_CHECK_=3 or it will just fails somewhere else randomly):

• ​https://github.com/sagemath/sage/commit/14061a1d40e8a27d86eac6dd91559b0131f9fe55

@Volker: could you have a look?

[jpflori]

Replying to jpflori:

For the segfault in schemes/elliptic_curves/isogeny_small_degree.py it seems that the following commit now triggers a double free or invalid free (use MALLOC_CHECK_=3 or it will just fails somewhere else randomly):

• ​https://github.com/sagemath/sage/commit/14061a1d40e8a27d86eac6dd91559b0131f9fe55

Kind of painfuly got it:

• nlNormalize(x) can free whatever is in x and change x from a struct to a disqguised int.
• when calling sa2si on a polynomial we call nlGetDenom or such functions on pointers returned by p_GetCoeff for the coeffs, which in turn call nlNormalize and free the underlying memory and modify our pointer, but not the original pointers in the polynomial
• we call p_Delete on the polynomial which call nlDelete on the coeff which gives no indication the underlying memory is freed
• bang

[jpflori]

The solution would be something like calling p_SetCoeff whenever we call p_GetCoeff to be sure any hackish modification gets transmitted back to the original place.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​cbd3af3	Seemingly equivalent result.
​20e0b22	Fix update of coefficients in algext singular elements.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​f9ae25e	Different singular printing for ring names.
​765b60a	Squarefree not implemented anymore by Singular for rings.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​e9b79db

Remove hack for singular 1*.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​21cf2a5

Hack to parse Singular polys with real coefficients.

[jpflori]

Here is the current state of affairs for make testlong on a POWER7/

sage -t --long src/sage/misc/randstate.pyx  # 1 doctest failed
sage -t --long src/sage/symbolic/function.pyx  # 1 doctest failed
sage -t --long src/sage/modular/modform_hecketriangle/graded_ring_element.py  # 7 doctests failed
sage -t --long src/sage/calculus/riemann.pyx  # 2 doctests failed
sage -t --long src/sage/schemes/projective/projective_morphism.py  # Timed out
sage -t --long src/sage/schemes/jacobians/abstract_jacobian.py  # 1 doctest failed
sage -t --long src/sage/schemes/curves/projective_curve.py  # 2 doctests failed
sage -t --long src/sage/schemes/curves/affine_curve.py  # 1 doctest failed
sage -t --long src/sage/rings/polynomial/multi_polynomial_ideal.py  # 2 doctests failed
sage -t --long src/sage/finance/time_series.pyx  # 6 doctests failed
sage -t --long src/sage/plot/graphics.py  # 1 doctest failed
sage -t --long src/sage/plot/contour_plot.py  # Killed due to segmentation fault
sage -t --long src/sage/plot/misc.py  # 1 doctest failed
sage -t --long src/sage/plot/plot.py  # 1 doctest failed
sage -t --long src/sage/plot/matrix_plot.py  # Killed due to segmentation fault
sage -t --long src/sage/tests/french_book/mpoly.py  # 2 doctests failed
sage -t --long src/sage/libs/singular/function.pyx  # 6 doctests failed
sage -t --long src/doc/en/prep/Advanced-2DPlotting.rst  # Killed due to segmentation fault
sage -t --long src/doc/en/constructions/algebraic_geometry.rst  # 14 doctests failed

[jpflori]

misc/randstate.pyx  # 1 doctest failed: unrelated, gap random different
symbolic/function.pyx  # 1 doctest failed: unrelated, precision issue
modular/modform_hecketriangle/graded_ring_element.py  # 7 doctests failed: singular, real coeffs printing with parenthesis
calculus/riemann.pyx  # 2 doctests failed: unrelated, precision issue
schemes/projective/projective_morphism.py  # Timed out: singular, real parsing 
schemes/jacobians/abstract_jacobian.py  # 1 doctest failed: singular, real parsing
schemes/curves/projective_curve.py  # 2 doctests failed: singular, real parsing
schemes/curves/affine_curve.py  # 1 doctest failed: singular, different result
rings/polynomial/multi_polynomial_ideal.py  # 2 doctests failed: singular, gwalk and skipping text issue
finance/time_series.pyx  # 6 doctests failed: unrelated, precision issue
plot/graphics.py  # 1 doctest failed: unrelated, div by 0
plot/contour_plot.py  # Killed due to segmentation fault: passed the second time
plot/misc.py  # 1 doctest failed: unrelated, div by 0
plot/plot.py  # 1 doctest failed: unrelated, div by 0
plot/matrix_plot.py  # Killed due to segmentation fault: passed the second time
tests/french_book/mpoly.py  # 2 doctests failed: singular, gwalk and skipping text issue
libs/singular/function.pyx  # 6 doctests failed: singular, arity code changes and skipping text issue

en/prep/Advanced-2DPlotting.rst  # Killed due to segmentation fault: passed the second time
en/constructions/algebraic_geometry.rst  # 14 doctests failed: singular, stuff in brnoeth.lib, undefined stuff

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​e705c27	Better handling of Singular 4 floats printing.
​5d18e56	Singular 4 puts sign outside of parentheses for floats.
​cfc8c56	Rely on Singular ring type rather than Sage one.
​c6eaaf4	New singular picks different ratfuncs.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​0192048

Update "arity code" constants for Singular 4.

[jpflori]

So we're hopefully left with:

• bug in mprimedec.lib (​https://groups.google.com/forum/#!topic/libsingular-devel/94d9c574uvg)
• bug in brnoeth.lib (​https://groups.google.com/forum/#!topic/libsingular-devel/DxjhbYqVfFQ)
• gwalk issue
• the "skipping text in parameter" output (when doing GB computations?)

[jpflori]

The grwalk thing must be:

• ​https://github.com/Singular/Sources/commit/53aa95a962b7ad2b6b48f9f308a22948c39cc41f

[jpflori]

In fact the grwalk and brnoeth issues were caused by old lib files in local/share/singular. Already fixed upstream.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​526f0a2

Make sure old Singular lib files are deleted.

[jpflori]

We're almost good now:

• the "skipping text from parameter" text only appears when an error previously happened in Singular, so we it now only appears when testing libs/singular/function.pyx becasuse of the GTZmod error.
• hopefully the GTZmod error will be fixed upstream soon and we can fix a commit to incule here (which should be more or less official Singular 4.0.3).

Can someone else please test the current branch? You'll need to build your own tarball first using spkg-src.

(There will still be some polishing to do about the Singular C++ function we use, see the comments Hans added in some header files, but that can wait for another ticket.)

[jpflori]

Oh crap, we should also reenable the possibility of debug builds. IIRC this was removed at some point in this ticket.

[jpflori]

But that can wait for another ticket as well (at least the replace omalloc by malloc stuff).

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​dcd4e14

More changes for Singular 4.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7366fb6

Better script to package singular tarball.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​9f016ae

Apply patches to singular before debug hacks.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​cfe9a9c

Patch for Singular xalloc.

[jpflori]

Setting SAGE_DEBUG=yes, debug mode with xalloc should be ok although:

• I did not wait for the build to finish,
• it seems auto* wants to reconfigure everything,
• you get a alot of unsued stuff warnings but that's surely normal.

I removed a bunch of hacks from spkg-install.

Please test!

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7dd92be

Fix singular debug build mode.

[dimpase]

the branch does not merge cleanly (so I had to do some manual editing), and after building singular I get

[sagelib-7.4.beta1] ************************************************************************
[sagelib-7.4.beta1] Traceback (most recent call last):
[sagelib-7.4.beta1]   File "setup.py", line 47, in <module>
[sagelib-7.4.beta1]     from module_list import ext_modules, library_order, aliases
[sagelib-7.4.beta1]   File "/home/dima/software/sage/src/module_list.py", line 1587, in <module>
[sagelib-7.4.beta1]     extra_compile_args = givaro_extra_compile_args),
[sagelib-7.4.beta1] NameError: name 'givaro_extra_compile_args' is not defined
[sagelib-7.4.beta1] ************************************************************************
[sagelib-7.4.beta1] Error building the Sage library

although this might be due to me trying to merge this into the branch on #17635 (which should probably become a dependency here)

[tscrim]

It built okay for me on my Ubunutu 16.04 LTS system.

Also, I think we should set a fixed tarball and checksum. Do you want me to do this?

[dimpase]

The upgrade of givaro on #17635 has to be taken into account here.

[Snark]

Replying to tscrim:

Also, I think we should set a fixed tarball and checksum. Do you want me to do this?

I don't follow the ticket very closely, but that makes me jump: what about taking an upstream tarball instead of hacking something?

[leif]

Replying to Snark:

Replying to tscrim:

Also, I think we should set a fixed tarball and checksum. Do you want me to do this?

I don't follow the ticket very closely, but that makes me jump: what about taking an upstream tarball instead of hacking something?

WIP / moving target:

commit 117ae9c7dde9eb9fcaa4cb899cfb7690addd6dfe
Author: Hans Schoenemann <hannes@mathematik.uni-kl.de>
Date:   Fri Aug 26 18:47:06 2016 +0200

    opt: prefer unsigned over int in for loops

commit f5e732af7572071c189ad8185ce830f96ababa46
Author: Hans Schoenemann <hannes@mathematik.uni-kl.de>
Date:   Fri Aug 26 18:46:40 2016 +0200

    compiler warnings

commit 6e28802f29d0fee351ccab022adc6853e46bd3fe
Author: Hans Schoenemann <hannes@mathematik.uni-kl.de>
Date:   Wed Aug 24 17:18:54 2016 +0200

    opt: use id_Delete in syz1.cc

commit 84963dfd52839fd9d4215c5d5b0534e80417dfad
Author: Hans Schoenemann <hannes@mathematik.uni-kl.de>
Date:   Wed Aug 24 16:42:29 2016 +0200

    opt: NULL in syResolvente->*res

commit 495258ce8ff8e5b1376e6c7fa8ae2815387e8b4a
Author: Hans Schoenemann <hannes@mathematik.uni-kl.de>
Date:   Mon Aug 22 16:04:20 2016 +0200

    removed wrong warning (see Tst/Plural/doc-module-ops.tst)

commit 32e84f5b8f065a8070d982ae939de64e4721beab
Author: Hans Schoenemann <hannes@mathematik.uni-kl.de>
Date:   Mon Aug 22 16:03:02 2016 +0200

    removed AlgDep-qso3-dp.tst from Plural/short.lst: also in Plural.lst

commit a070b84d59be835a2946d1386edd3221d80f3fe7
Author: Hans Schoenemann <hannes@mathematik.uni-kl.de>
Date:   Fri Aug 19 16:56:16 2016 +0200

    Singular 4.0.3p3

commit 961c76da92efc86885b6506c8c83262f14e705ff
Author: Hans Schoenemann <hannes@mathematik.uni-kl.de>
Date:   Fri Aug 19 16:34:25 2016 +0200

    chg: n_IsMOne for Z/2: maybe n_Zp, n_Zn, n_Z2m

...

[leif]

Minor thing: All error messages in spkg-install should start with "Error[:] ...".

spkg-src could be beautified a bit as well (e.g. not unconditionally deleting singular.git, and recording what we checked out, error checking), besides that curl is non-standard... ;-)

[leif]

P.S.: I'd prefer to not use -lSingular, but create a symlink from lower to uppercase instead.

[dimpase]

aclocal is currently a dependency, probably due to warnings treated as errors:

[singular-4.0.3p3] config.status: executing libtool commands
[singular-4.0.3p3] ### Singular spkg-install: build_singular ###
[singular-4.0.3p3] make[3]: Entering directory '/home/dima/Sage/sage/local/var/tmp/sage/build/singular-4.0.3p3/src/latest'
[singular-4.0.3p3] CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /home/dima/Sage/sage/local/var/tmp/sage/build/singular-4.0.3p3/src/latest/build-aux/missing aclocal-1.14 -I m4
[singular-4.0.3p3] /home/dima/Sage/sage/local/var/tmp/sage/build/singular-4.0.3p3/src/latest/build-aux/missing: line 81: aclocal-1.14: command not found
[singular-4.0.3p3] WARNING: 'aclocal-1.14' is missing on your system.
[singular-4.0.3p3]          You should only need it if you modified 'acinclude.m4' or
[singular-4.0.3p3]          'configure.ac' or m4 files included by 'configure.ac'.
[singular-4.0.3p3]          The 'aclocal' program is part of the GNU Automake package:
[singular-4.0.3p3]          <http://www.gnu.org/software/automake>
[singular-4.0.3p3]          It also requires GNU Autoconf, GNU m4 and Perl in order to run:
[singular-4.0.3p3]          <http://www.gnu.org/software/autoconf>
[singular-4.0.3p3]          <http://www.gnu.org/software/m4/>
[singular-4.0.3p3]          <http://www.perl.org/>
[singular-4.0.3p3] make[3]: *** [Makefile:485: aclocal.m4] Error 127
[singular-4.0.3p3] make[3]: Leaving directory '/home/dima/Sage/sage/local/var/tmp/sage/build/singular-4.0.3p3/src/latest'
[singular-4.0.3p3] 
[singular-4.0.3p3] real	0m28.318s
[singular-4.0.3p3] user	0m13.437s
[singular-4.0.3p3] sys	0m4.493s
[singular-4.0.3p3] ************************************************************************
[singular-4.0.3p3] Error installing package singular-4.0.3p3
[singular-4.0.3p3] ************************************************************************

only caught this as I'm setting up a new laptop :-)

[dimpase]

I have aclocal-1.15, but it insists on 1.14; weird, no?

[jdemeyer]

Replying to dimpase:

aclocal is currently a dependency

This should be fixed by running the necessary autotools in spkg-src.

[leif]

Replying to jdemeyer:

Replying to dimpase:

aclocal is currently a dependency

This should be fixed by running the necessary autotools in spkg-src.

AC_REQUIRE proper timestamps.

[dimpase]

Replying to jdemeyer:

Replying to dimpase:

aclocal is currently a dependency

This should be fixed by running the necessary autotools in spkg-src.

I did make the tarball using spkg-src, as described. I guess it's an upstream bug: they don't detect presence of aclocal-1.14, even though they don't need it, and bail out, even though it's meant to be a warning.

Any, why 1.14 only? Won't they be happy with 1.14 or newer?

[dimpase]

modulo the following, ptestlong passes with the new branch

sage -t --long --warn-long 77.2 src/sage/schemes/curves/affine_curve.py
**********************************************************************
File "src/sage/schemes/curves/affine_curve.py", line 494, in sage.schemes.curves.affine_curve.AffineCurve.resolution_of_singularities
Failed example:
    C.resolution_of_singularities(extend=True)[0] # long time (2 seconds)
Expected:
    (Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*x^2*ss1^3 + 24*ss1^3 + (a0^3 - 8*a0),
     Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (6*a0^3)*s1,
     Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 8*y^2*s0^4 + (-4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y)
Got:
    (Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*x^2*ss1^3 + 24*ss1^3 + (a0^3 - 8*a0),
     Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (-6*a0^3)*s1,
     Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 8*y^2*s0^4 + (4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y)
**********************************************************************
1 item had failures:
   1 of  23 in sage.schemes.curves.affine_curve.AffineCurve.resolution_of_singularities
    [195 tests, 1 failure, 3.51 s]

I don't know enough about resolution of singularities to decide whether the latter output is correct, or not (notice some signs are changed...)

---

Last 10 new commits:

​5d8c1d9	adding a patch fixing the i386 with AVX issue
​0d27874	Fix int size bug in the interface between singular and givaro (on 32bit arch)
​162c717	preparing upgrade to linbox-1.4.2rc0 fflas-ffpack-2.2.2rc0 and givaro 4.0.2rc0
​20e86b3	update to the lastest releases
​2fb5502	* Merging and fixing conflicts.
​85ce544	Merge branch 'u/cpernet/givaro_fflasffpack_linbox' of git://trac.sagemath.org/sage into givaroupd
​b7233c9	Fix coercion bug: raise a TypeError exception whenever an ArithmeticError is caught, while trying to reduce an element mod the characteristic.
​cd30aec	Merge branch 'u/cpernet/givaro_fflasffpack_linbox' of git://trac.sagemath.org/sage into givaroupd
​0efcdec	Merge branch 'public/singular4' into givaro 4.0.2 branch from #17635
​457bd23	missing capital 'S' in Singular

[leif]

Could we please not mix commits from different upgrades?

I don't see an immediate reason to base this ticket on #17635 either; they're IMHO better first reviewed independently.

[embray]

LGTM at least in principle. I'm going to try over the next day or two to test how this works on Cygwin64.

[embray]

Also I guess the milestone for this ticket should be changed? I don't know what to though.

[dimpase]

Replying to leif:

Could we please not mix commits from different upgrades?

I don't see an immediate reason to base this ticket on #17635 either; they're IMHO better first reviewed independently.

well, they depend, albeit not too deeply. Perhaps wait for the next beta with #17635 in?

[tscrim]

IMO, we should get this in as soon as we can to get as much testing as we can before it goes into a stable release.

[embray]

Replying to tscrim:

IMO, we should get this in as soon as we can to get as much testing as we can before it goes into a stable release.

+1; I'm trying to get cygwin testing ASAP, but first I have to get cygwin builds working against the current develop (since my cygwin branch is from a couple months' old develop). Should be close to that, but I'd be fine if this is merged in the meantime, and I can open a ticket for any issue I discover later.

[jdemeyer]

#17635 is now closed, so you need to make sure that it doesn't conflict with that ticket.

[jdemeyer]

...although Volker closes tickets before fully testing them, so there is still a chance that it will be reopened.

[jpflori]

Replying to embray:

LGTM at least in principle. I'm going to try over the next day or two to test how this works on Cygwin64.

Thx, I absolutely did nothing Cygwin-related on that one.

[jpflori]

Replying to leif:

Minor thing: All error messages in spkg-install should start with "Error[:] ...".

Yes.

spkg-src could be beautified a bit as well (e.g. not unconditionally deleting singular.git, and recording what we checked out, error checking), besides that curl is non-standard... ;-)

Sure but curl was already used in the script anyway and that's only to make the tarball, not for end-users.

[jdemeyer]

I agree that using curl in spkg-src is not a big issue.

[jdemeyer]

This cannot go into Sage until there is a proper tarball.

[jdemeyer]

What's the point of this commented-out code? Just remove it.

# C++11 workaround https://trac.sagemath.org/ticket/20926
# Since https://trac.sagemath.org/ticket/17635 linbox passes a -std=gnu++11 flag, so we no longer need to add it
# if [ "$UNAME" = CYGWIN ]; then
     # Special case for Cygwin https://trac.sagemath.org/ticket/21185
#     CXXFLAGS="$CXXFLAGS -std=gnu++98"
# else
#     CXXFLAGS="$CXXFLAGS -std=c++98"
#fi

[jdemeyer]

In build/pkgs/singular/spkg-install, if you use set -e, it is pointless to do manual error checking like

    if [ $? -ne 0 ]; then
        echo >&2 "Error applying '$patch'"
        return 1
    fi

[jdemeyer]

What's the point of this commented-out code? Just remove it.

 #    Extension('sage.libs.linbox.linbox',
 #             sources = ['sage/libs/linbox/linbox.pyx']),

More generally, better remove all commented-out code unless there is a clear purpose.

[jdemeyer]

In

    cdef enum n_coeffType:
        n_unknown=0
        n_Zp=1 #/**< \F{p < 2^31} */
        n_Q=2   #/**< rational (GMP) numbers */
        [...]

I would remove the values (like =0) because that is only confusing. And please either properly format the comments there or remove them.

[jdemeyer]

This is also silly:

            if sizeof(long) > 4:

                # it seems Singular uses ints somewhere
                # internally, cf. #6051 (Sage) and #138 (Singular)

                if exponent <= 30:  ringtype = n_Z2m
                else:               ringtype = n_Znm
            else:
                if exponent <= 30: ringtype = n_Z2m
                else:              ringtype = n_Znm

You can change code of the form

if condition:
    A
else:
    A

to

A

[jdemeyer]

I think this regression is not acceptable:

             sage: P.<x,y,z> = PolynomialRing(QQ)
             sage: f= x^2 + 2*x*y + 1/2*z
             sage: f.is_squarefree()
             Traceback (most recent call last):
             ...
             NotImplementedError: is_squarefree of multivariate polynomials over rings is not implemented.

Asking whether a multivariate polynomial is squarefree seems like a very natural thing to do and not something you should just break.

[jdemeyer]

Also, I am not so happy with the regression in exponent overflow but that's a less serious issue.

[jdemeyer]

Sorry to say this, but the code is really ugly and certainly uglier than it was before. I see lots of annoying things, like unneeded comments, indentation not a multiple of 4 spaces, needlessly complicated code.

I think this requires somebody to go over all the code and clean it up.

[jdemeyer]

Another example:

    libSingularFound = False

    for extension in ["so", "dylib", "dll"]:
        lib = os.environ['SAGE_LOCAL']+"/lib/libSingular."+extension # libsingular renamed to libSingular
        if os.path.exists(lib):
            libSingularFound = True
            handle = dlopen(lib, RTLD_GLOBAL|RTLD_LAZY)
            if not handle:
                err = dlerror()
                if err:
                    print(err)
            break

    if not libSingularFound :
        raise OSError("did not find libsingular file")


    if handle == NULL:
        dlerrormsg = dlerror()
        if dlerrormsg == NULL:
            dlerrormsg = ""
        raise ImportError("failed to dlopen {} ({})".format(lib, dlerrormsg))

The handle == NULL cannot be reached since we would error out before. The old code is simpler and better:

    for extension in ["so", "dylib", "dll"]:
        lib = os.environ['SAGE_LOCAL']+"/lib/libsingular."+extension
        if os.path.exists(lib):
            handle = dlopen(lib, RTLD_GLOBAL|RTLD_LAZY)
            if not handle:
                err = dlerror()
                if err:
                    print(err)
            break

    if handle == NULL:
        raise ImportError("cannot load libSINGULAR library")

There was no reason to make the code more complicated, why was it done???

[jdemeyer]

Another example in plural.pyx:

cdef n_coeffType unknownRingtype = n_unknown

What's the point of this?

[jdemeyer]

To be clear: I am very happy with the work done on this ticket, really. I just feel that it's not ready to merge yet. I also feel that not much effort was done to re-use the existing code, that code was rewritten without reason. That makes it hard to review because the diff is huge.

[jpflori]

No worry Jeroen, that's exactly the point of making a proper review. To this point this ticket is just a heap of changes, hacks, commits by different people to get the new singular working and at least on my side I did not really look at what the other did, especially given the huge diff. I'll try to polish the points you mentioned this week.

[jdemeyer]

At least we have something that works (modulo the is_squarefree issue). That is a very important starting point.

[dimpase]

Replying to jdemeyer:

At least we have something that works (modulo the is_squarefree issue). That is a very important starting point.

surely Singular 4.0.3 still has the squarefree test, so it's matter of digging up how it is implemented:

> ring r=0,(x,y),dp;
> poly p=x^2+2*x*y+y^2;
>> ring r=0,(x,y),dp;
> poly p=x^2+2*x*y+y^2;
> sqrfree(p);
[1]:
   _[1]=1
   _[2]=x+y
[2]:
   1,2

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7e8eb3a	add pacth fixing the gcc <= 4.9.2 issue by leif.
​aab3eb1	Merge branch 'u/cpernet/givaro_fflasffpack_linbox' of trac.sagemath.org:sage into givaroupd

[dimpase]

merged last commit from givaro upgrade ticket, uploaded the tarball.

[jpflori]

oops, we pushed similar stuff at the same time...

[jpflori]

I'm working on this, so pleas refrain taking Jeroen comments into account right now :)

[jpflori]

Took some Jeroen comments into account.

---

New commits:

​95b768a	Modify error handling in Singular spkg-install.
​b4dad30	Merge remote-tracking branch 'trac/u/dimpase/singular4' into singular
​9124efc	Purify enum for singular ring types.
​c60efc2	Remove print debug statement in singular ring creation.
​34960ee	Revert simpler old code to load libsingular.
​0c7c6a9	Don't define useless variable.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​5f99779	Remove useless import.
​1d4ba4e	Simplify verbosity for singular.
​cd3cd2f	Remove print debug statement in polynomial.pyx.
​a4d5418	Remove old comments in singular ring creation.
​b307cd0	Simplify cimport from singular.
​cd00aa1	Remove print debug statement in singular lib.

[jpflori]

I'm done for at least a couple of hours. So feel free to play with the current branch. And pleas push any changes on the public/singular4 branch.

[dimpase]

with the new tarball, it complains about absent aclocal-1.13...

Something needs to be done with this. Normally speaking, you want to make tarballs using make dist, after running ./configure, and not by hand. I'll try to go this route now, and see if this works. By right, it should not require any autotools then (of course one can remove stuff from the tarball, after it's done).

Probably one just needs to run automake --add-missing --copy before making tarball.

[mkoeppe]

Replying to dimpase:

with the new tarball, it complains about absent aclocal-1.13...

Something needs to be done with this. Normally speaking, you want to make tarballs using make dist, after running ./configure, and not by hand. I'll try to go this route now, and see if this works. By right, it should not require any autotools then (of course one can remove stuff from the tarball, after it's done).

Is this using the autotools package from #21047?

[dimpase]

it should not use any autotools while installing, at all. It's clearly something is not done properly while making tarball (by spkg-src).

[jakobkroeker]

There was no reason to make the code more complicated, why was it done???

I changed the library loading code to annoy everyone ( just kidding ;-) )

Revert simpler old code to load Singular

I disagree.

Please read my arguments first and reply to the arguments (or disprove them). @Jeroen, If it was your code I changed and it hurts you. I'm sorry for that. Still, try to understand the arguments for the change and accept the change if you agree with the arguments.

The old (and current) failing message is

raise ImportError("cannot load libSINGULAR library")

In some situations this message gives no clue about the error caused a failing and even worse, is misleading:

The issue I ran into was while working on this upgrade, the loading of libSINGULAR failed, but I could for hours not figure out the cause. The library file was there, I was able to open it manually. The error message did mislead my thoughts in the wrong direction. The issue was that Singular changed the naming of 'libsingular.so' to 'libSingular.so' (did you note the difference? ('s' vs 'S') and dlopen failed because sage tried to open a non-existing file!

Also, if we reach

if handle == NULL

it is legitimate to print for _which_ file we failed to dlopen, because we try several different extensions ("so", "dylib", "dll") in a loop! Otherwise, by just looking at the error message we would have NO CLUE, for _which_ file dlopen failed. And they may exist all three of them in the same place.

[jakobkroeker]

Purify enum for singular ring types.

If the comments are not well-formatted, incomplete or incorrect, it would be better to fix them rather than removing.

(Have fun to guess by the names what they stand for)

[jakobkroeker]

Remove print debug statement in singular lib

If I recall correctly, if Singular ring creation fails (with exception), there is a fallback creating a sage ring, (sage catches the exception, and no exception is printed)

So my question is, how a developer should properly and easily detect, that Singular ring creation ended with an exception?

[jpflori]

Replying to dimpase:

it should not use any autotools while installing, at all. It's clearly something is not done properly while making tarball (by spkg-src).

spkg-src just calls autoreconf which is the usual way to generate configure and Makefile.in. Not sure why running configure or make decides to rerun autotools. Maybe some timestamps issues after patching but we don't patch a lot, especially we do nothing to configure and Makefile.in, so... And IIRC I had no trace of autotools getting rerun on my setup, except when using SAGE_DEBUG=yes (which was already strange enough).

[jpflori]

Replying to jakobkroeker:

Remove print debug statement in singular lib

If I recall correctly, if Singular ring creation fails (with exception), there is a fallback creating a sage ring, (sage catches the exception, and no exception is printed)

So my question is, how a developer should properly and easily detect, that Singular ring creation ended with an exception?

It's fine to have print statement during development, but I would say it should not be there in "production" code.

[jpflori]

Replying to jakobkroeker:

Purify enum for singular ring types.

If the comments are not well-formatted, incomplete or incorrect, it would be better to fix them rather than removing.

(Have fun to guess by the names what they stand for)

Comments are available here: ​https://github.com/Singular/Sources/blob/spielwiese/libpolys/coeffs/coeffs.h

I took the easy solution of removing them form the Cython wrapper.

[jpflori]

Replying to jakobkroeker:

There was no reason to make the code more complicated, why was it done???

I changed the library loading code to annoy everyone ( just kidding ;-) )

Revert simpler old code to load Singular

I disagree.

Please read my arguments first and reply to the arguments (or disprove them). @Jeroen, If it was your code I changed and it hurts you. I'm sorry for that. Still, try to understand the arguments for the change and accept the change if you agree with the arguments.

The old (and current) failing message is

raise ImportError("cannot load libSINGULAR library")

In some situations this message gives no clue about the error caused a failing and even worse, is misleading:

The issue I ran into was while working on this upgrade, the loading of libSINGULAR failed, but I could for hours not figure out the cause. The library file was there, I was able to open it manually. The error message did mislead my thoughts in the wrong direction. The issue was that Singular changed the naming of 'libsingular.so' to 'libSingular.so' (did you note the difference? ('s' vs 'S') and dlopen failed because sage tried to open a non-existing file!

Also, if we reach

if handle == NULL

it is legitimate to print for _which_ file we failed to dlopen, because we try several different extensions ("so", "dylib", "dll") in a loop! Otherwise, by just looking at the error message we would have NO CLUE, for _which_ file dlopen failed. And they may exist all three of them in the same place.

Ok, so maybe we should put back a more verbose error message.

[jpflori]

Replying to jpflori:

Replying to jakobkroeker:

There was no reason to make the code more complicated, why was it done???

I changed the library loading code to annoy everyone ( just kidding ;-) )

Revert simpler old code to load Singular

I disagree.

Please read my arguments first and reply to the arguments (or disprove them). @Jeroen, If it was your code I changed and it hurts you. I'm sorry for that. Still, try to understand the arguments for the change and accept the change if you agree with the arguments.

The old (and current) failing message is

raise ImportError("cannot load libSINGULAR library")

In some situations this message gives no clue about the error caused a failing and even worse, is misleading:

The issue I ran into was while working on this upgrade, the loading of libSINGULAR failed, but I could for hours not figure out the cause. The library file was there, I was able to open it manually. The error message did mislead my thoughts in the wrong direction. The issue was that Singular changed the naming of 'libsingular.so' to 'libSingular.so' (did you note the difference? ('s' vs 'S') and dlopen failed because sage tried to open a non-existing file!

Also, if we reach

if handle == NULL

it is legitimate to print for _which_ file we failed to dlopen, because we try several different extensions ("so", "dylib", "dll") in a loop! Otherwise, by just looking at the error message we would have NO CLUE, for _which_ file dlopen failed. And they may exist all three of them in the same place.

Ok, so maybe we should put back a more verbose error message.

And rather than trying three different extensions, just try the one corresponding to the underlying system.

[jakobkroeker]

It's fine to have print statement during development, but I would say it should not be there in "production" code.

Thats fine to remove the (outcommented) debug statements.

My question is just about if there is a better way to detect a failed singular ring creation than printouts?.

[jakobkroeker]

Replying to jpflori:

Replying to jakobkroeker:

Purify enum for singular ring types.

If the comments are not well-formatted, incomplete or incorrect, it would be better to fix them rather than removing.

Comments are available here: ​https://github.com/Singular/Sources/blob/spielwiese/libpolys/coeffs/coeffs.h

I took the easy solution of removing them form the Cython wrapper.

Sure, there are some comments in spielwiese/libpolys/coeffs/coeffs.h, but then the reader has to investigate (how did _you_ find the coeffs.h?)

At least I would add in decl.pxd before the line cdef enum n_coeffType: something like

# see libpolys/coeffs/coeffs.h for reference

What do you think?

Not that I did a better job, but do you agree, that several comments in coeffs.h related to the enum n_coeffType are not really helpful? (=> we should improve them?)

[dimpase]

Replying to dimpase:

it should not use any autotools while installing, at all. It's clearly something is not done properly while making tarball (by spkg-src).

I give up, I don't understand quite what's going on. There is a missing script in build-aux/ that is called on aclocal-<VERSION>, where <VERSION> is the autotools version the tarball is made with. But I don't know why it is there and how to disable it.

[jakobkroeker]

And rather than trying three different extensions, just try the one corresponding to the underlying system.

Not quite sure, since I 'm not a Mac OS user, but it might be (or might be not) a problem on Mac OS. Are you familiar with Mac OS?

If I recall correctly, on Mac OS we might have both (".so" and ".dylib") or only one of (".so" and ".dylib")? Is that true??

[jakobkroeker]

Replying to dimpase:

Replying to dimpase:

it should not use any autotools while installing, at all. It's clearly something is not done properly while making tarball (by spkg-src).

I give up, I don't understand quite what's going on. There is a missing script in build-aux/ that is called on aclocal-<VERSION>, where <VERSION> is the autotools version the tarball is made with. But I don't know why it is there and how to disable it.

We could ask Jeroen or Jean-Pierre for help to package recent Singular,

...or ask Hans Schoenemann(Singular head developer) for the Singular release timetable...

[dimpase]

the following hack appears to stop that aclocal nonsense:

--- a/build/pkgs/singular/spkg-install
+++ b/build/pkgs/singular/spkg-install
@@ -65,6 +65,9 @@ remove_old_version()
     rm -rf "$SAGE_LOCAL/share/singular"
 }
 
+touch configure.ac aclocal.m4 configure Makefile.am Makefile.in
+touch */configure.ac */aclocal.m4 */configure */Makefile.am */Makefile.in
+
 config()
 {
     # configure notes (dates from Singular 3.x, maybe outdated for 4.x):

will do more tests on it, before pushing this change...

[john_perry]

Replying to jakobkroeker:

And rather than trying three different extensions, just try the one corresponding to the underlying system.

Not quite sure, since I 'm not a Mac OS user, but it might be (or might be not) a problem on Mac OS. Are you familiar with Mac OS?

If I recall correctly, on Mac OS we might have both (".so" and ".dylib") or only one of (".so" and ".dylib")? Is that true??

I can tell you if you confirm for me where I should look. When I look in /Applications/sage-6.7/src/build/lib.macosx-10.9-x86_64-2.7/sage/libs/singular/ I find only singular.so. Do you know anywhere else we might put a library? (And is it likely anything would have changed since 6.7?

[jpflori]

I guess this one corresponds to the commit I usd:

• ​http://www.mathematik.uni-kl.de/ftp/pub/Math/Singular/SOURCES/4-0-3/singular-4.0.3p3.tar.gz

I repackaged it, but now it also fails on my system complaining about aclocal-1.15...

[jpflori]

And the upstream tarball is not autoreconfed in the xalloc directory so SAGE_DEBUG=yes won't work.

[embray]

I haven't been following this discussion closely, but if I were to test out this branch now should I expect it to work (for some value of "work"--at least build?)

[jpflori]

On my system, aclocal first gets called in resources seemingly because it thinks that files generated by libtool in ../m4/ are older than aclocal.m4.

[jdemeyer]

Replying to jakobkroeker:

Please read my arguments first and reply to the arguments (or disprove them).

If there are arguments for that in one of the 315 comments above, sorry that I missed them :-)

@Jeroen, If it was your code I changed and it hurts you. I'm sorry for that.

I don't care whose code it was. Probably not mine.

The old (and current) failing message is

raise ImportError("cannot load libSINGULAR library")

In some situations this message gives no clue about the error caused a failing and even worse, is misleading:

Improving an error message is very good, but I don't see why you needed to complicate the whole code structure for that.

Also, if we reach

if handle == NULL

It wasn't clear to me whether it was even possible to reach that line.

Anyway, this has been improved now (although the now-unused variables dlerrormsg and libSingularFound should still be removed).

[jpflori]

Note that at minute precision these files have similar timestamps, altough at second precision aclocal.m4 is actually older...

[dimpase]

on 32-bit Ubuntu 14.04 I get, with the touch hack above, the following error while building sagelib:

[193/459] Cythonizing sage/libs/singular/singular.pyx

Error compiling Cython file:
------------------------------------------------------------
...
from sage.libs.singular.decl cimport number, poly, ring, currRing
from sage.libs.singular.decl cimport rChangeCurrRing, rCopy0, rComplete, rDelete, idInit
from sage.libs.singular.decl cimport omAlloc0, omStrDup, omAlloc, omAlloc0Bin,  sip_sring_bin, rnumber_bin
from sage.libs.singular.decl cimport ringorder_dp, ringorder_Dp, ringorder_lp, ringorder_rp, ringorder_ds, ringorder_Ds, ringorder_ls, ringorder_M, ringorder_C, ringorder_wp, ringorder_Wp, ringorder_ws, ringorder_Ws, ringorder_a
from sage.libs.singular.decl cimport p_Copy, prCopyR
from sage.libs.singular.decl cimport n_unknown,  n_Zp,  n_Q,   n_R,   n_GF,  n_long_R,  n_algExt,n_transExt,n_long_C,   n_Z,   n_Zn,  n_Znm,  n_Z2m,  n_CF
^
------------------------------------------------------------

sage/libs/singular/ring.pyx:26:0: 'sage/libs/singular/decl/n_Zn.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
from sage.libs.singular.decl cimport number, poly, ring, currRing
from sage.libs.singular.decl cimport rChangeCurrRing, rCopy0, rComplete, rDelete, idInit
from sage.libs.singular.decl cimport omAlloc0, omStrDup, omAlloc, omAlloc0Bin,  sip_sring_bin, rnumber_bin
from sage.libs.singular.decl cimport ringorder_dp, ringorder_Dp, ringorder_lp, ringorder_rp, ringorder_ds, ringorder_Ds, ringorder_ls, ringorder_M, ringorder_C, ringorder_wp, ringorder_Wp, ringorder_ws, ringorder_Ws, ringorder_a
from sage.libs.singular.decl cimport p_Copy, prCopyR
...

I don't know whether it's related to touch applied, or not.

I can install the requested aclocal version and see whether without touch I get the same error...

[jdemeyer]

If I recall correctly, on Mac OS we might have both (".so" and ".dylib") or only one of (".so" and ".dylib")? Is that true??

Both exist for a different purpose. On OS X:

• .dylib = dynamic library (needed for the usual case of linking libraries at build-time).

• .so = loadable module (needed for run-time linking, for example Python extension modules).

On most other OSes, there is no difference between these.

[jpflori]

Replying to dimpase:

on 32-bit Ubuntu 14.04 I get, with the touch hack above, the following error while building sagelib:

[193/459] Cythonizing sage/libs/singular/singular.pyx

Error compiling Cython file:
------------------------------------------------------------
...
from sage.libs.singular.decl cimport number, poly, ring, currRing
from sage.libs.singular.decl cimport rChangeCurrRing, rCopy0, rComplete, rDelete, idInit
from sage.libs.singular.decl cimport omAlloc0, omStrDup, omAlloc, omAlloc0Bin,  sip_sring_bin, rnumber_bin
from sage.libs.singular.decl cimport ringorder_dp, ringorder_Dp, ringorder_lp, ringorder_rp, ringorder_ds, ringorder_Ds, ringorder_ls, ringorder_M, ringorder_C, ringorder_wp, ringorder_Wp, ringorder_ws, ringorder_Ws, ringorder_a
from sage.libs.singular.decl cimport p_Copy, prCopyR
from sage.libs.singular.decl cimport n_unknown,  n_Zp,  n_Q,   n_R,   n_GF,  n_long_R,  n_algExt,n_transExt,n_long_C,   n_Z,   n_Zn,  n_Znm,  n_Z2m,  n_CF
^
------------------------------------------------------------

sage/libs/singular/ring.pyx:26:0: 'sage/libs/singular/decl/n_Zn.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
from sage.libs.singular.decl cimport number, poly, ring, currRing
from sage.libs.singular.decl cimport rChangeCurrRing, rCopy0, rComplete, rDelete, idInit
from sage.libs.singular.decl cimport omAlloc0, omStrDup, omAlloc, omAlloc0Bin,  sip_sring_bin, rnumber_bin
from sage.libs.singular.decl cimport ringorder_dp, ringorder_Dp, ringorder_lp, ringorder_rp, ringorder_ds, ringorder_Ds, ringorder_ls, ringorder_M, ringorder_C, ringorder_wp, ringorder_Wp, ringorder_ws, ringorder_Ws, ringorder_a
from sage.libs.singular.decl cimport p_Copy, prCopyR
...

I don't know whether it's related to touch applied, or not.

Nope, I must have screwed up when removing the comments in the enumeration.

I can install the requested aclocal version and see whether without touch I get the same error...

Thats not necessary, we cannot get away with only a touch anyway because of xalloc. I'll rerun autoreconf on my side to get proper timestamps in all folders...

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​d6dee3c

Fix enum in singular ring types.

[dimpase]

Replying to git:

Branch pushed to git repo; I updated commit sha1. New commits:

​d6dee3c

Fix enum in singular ring types.

OK, this helps to get further; the latest error on 32-bit Ubuntu is

[149/444] gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wno-unused -fPIC -I/home/dima/sage/dev/sage/local/include -I/home/dima/sage/dev/sage/local/include/python2.7 -I/home/dima/sage/dev/sage/local/lib/python2.7/site-packages/numpy/core/include -I/home/dima/sage/dev/sage/src -I/home/dima/sage/dev/sage/src/sage/ext -I/home/dima/sage/dev/sage/src/build/cythonized -I/home/dima/sage/dev/sage/src/build/cythonized/sage/ext -I/home/dima/sage/dev/sage/local/include/python2.7 -c /home/dima/sage/dev/sage/src/build/cythonized/sage/libs/singular/ring.cpp -o build/temp.linux-i686-2.7/home/dima/sage/dev/sage/src/build/cythonized/sage/libs/singular/ring.o -fno-strict-aliasing -fno-tree-copyrename
error: command 'gcc' failed with exit status 1
/home/dima/sage/dev/sage/src/build/cythonized/sage/libs/singular/polynomial.cpp: In function ‘long int __pyx_f_4sage_4libs_8singular_10polynomial_singular_polynomial_deg(polyrec*, polyrec*, ip_sring*)’:
/home/dima/sage/dev/sage/src/build/cythonized/sage/libs/singular/polynomial.cpp:6028:7: warning: ‘__pyx_v_i’ may be used uninitialized in this function [-Wmaybe-uninitialized]
   int __pyx_v_i;
       ^
make[3]: *** [sage] Error 1

It might be that gcc is too old, it uses gcc version 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.3).

Should I rather build Sage's gcc?

[dimpase]

Replying to jpflori:

I can install the requested aclocal version and see whether without touch I get the same error...

Thats not necessary, we cannot get away with only a touch anyway because of xalloc. I'll rerun autoreconf on my side to get proper timestamps in all folders...

I do touch stuff in */ too, not only in top level. (probably * is an overkill, but it works). Indeed, I first tried to touch the top level only, it went a bit further, entered resources/, and complained from there instead.

[jpflori]

One also needs to touch [*/]_config.h.in in order to prevent autoheader` invokation... I'll modify the spkg-src script to:

• use the upstream tarball,
• autoreconf in xalloc,
• touch all files.

[jdemeyer]

Replying to dimpase:

OK, this helps to get further; the latest error on 32-bit Ubuntu is

What's the error message? (it's not in the piece that you quoted)

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​f8362a9

Hopefully a singular tarball not invoking autostuff.

[dimpase]

Replying to jdemeyer:

Replying to dimpase:

OK, this helps to get further; the latest error on 32-bit Ubuntu is

What's the error message? (it's not in the piece that you quoted)

oops, sorry... please see err.log in attachments to this ticket

[jpflori]

Ok reuploaded a new tarball (same address). Please test. Note that I finally did:

• use upstream 4.0.3p3
• autoreconf
• autoreconf in xalloc
• touch files

If I only autoreconfed xalloc, then libtool complained about different versions so I autoreconfed evrything then xalloc. (Yes, I know I could build my own autotools to get matching versions but thats more time consuming.)

[dimpase]

the link to the new tarball is not working

[jpflori]

Should be ok now.

[dimpase]

Replying to jpflori:

Should be ok now.

As far as aclocal & Co. problem is concerned, it's fixed now, thanks.

[jpflori]

Can you also try with SAGE_DEBUG=yes ?

[dimpase]

I now have another error, on a 64-bit machine with gcc 6.1.1:

[dochtml] /home/dima/Sage/sage/local/bin/python: cannot load libSINGULAR library; 'sage_setup.docbuild' is a package and cannot be directly executed
make[2]: *** [Makefile:1039: doc-html] Error 1

this is even after make distclean. No idea what it is and where it really comes from. make build completes, but Sage crashes on startup. (in crash report libSINGULAR is also mentioned). I'll attach it in a moment here.

[dimpase]

I bet it's one of the latest "simplifying"/"cleaning up" commits that is to blame.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​9a99f2c

Singular, not singular!

[dimpase]

Replying to git:

Branch pushed to git repo; I updated commit sha1. New commits:

​9a99f2c

Singular, not singular!

this fixes a typo in 34960ee. Now Sage starts.

[jakobkroeker]

Replying to jdemeyer:

Replying to jakobkroeker: Anyway, this (remark: loading libSINGULAR) has been improved now (although the now-unused variables dlerrormsg and libSingularFound should still be removed).

Nope, my changes were just reverted and now Dima had fun with the issue (see comment_315 ):

Dima faced the old error message ( see comment_353 )

[dochtml] /home/dima/Sage/sage/local/bin/python: cannot load libSINGULAR library; 'sage_setup.docbuild' is a package and cannot be directly executed
make[2]: *** [Makefile:1039: doc-html] Error 1

He comments the error:

 No idea what it is and where it really comes from

Finally he was able to fix it (comment_356 )

[dimpase]

still failing on 32-bit Ubuntu 14.04 with gcc 4.8.4

[dimpase]

Replying to jpflori:

Can you also try with SAGE_DEBUG=yes ?

this seems to work (the package building without aclocal-1.15 installed); whether it works is another question.

[leif]

Replying to dimpase:

still failing on 32-bit Ubuntu 14.04 with gcc 4.8.4

When you get an error during building the Sage library, please do

env SAGE_NUM_THREADS=1 ./sage -b

and post the log of that.

[leif]

I'm getting:

sage -t --long --warn-long 99.6 src/sage/schemes/curves/affine_curve.py
**********************************************************************
File "src/sage/schemes/curves/affine_curve.py", line 494, in sage.schemes.curves.affine_curve.AffineCurve.resolution_of_singularities
Failed example:
    C.resolution_of_singularities(extend=True)[0] # long time (2 seconds)
Expected:
    (Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by
    24*x^2*ss1^3 + 24*ss1^3 + (a0^3 - 8*a0),
     Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by
     24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (6*a0^3)*s1,
     Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by
     8*y^2*s0^4 + (-4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y)
Got:
    (Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*x^2*ss1^3 + 24*ss1^3 + (a0^3 - 8*a0),
     Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (-6*a0^3)*s1,
     Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 8*y^2*s0^4 + (4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y)
**********************************************************************
1 item had failures:
   1 of  23 in sage.schemes.curves.affine_curve.AffineCurve.resolution_of_singularities
    [195 tests, 1 failure, 29.15 s]
----------------------------------------------------------------------
sage -t --long --warn-long 99.6 src/sage/schemes/curves/affine_curve.py  # 1 doctest failed
----------------------------------------------------------------------

---

Just in case anybody wonders why Singular suddenly builds that fast:

 if [ "x$SAGE_DEBUG" = "xyes" ]; then
     ...
-else
-    WITH_DEBUG="--without-debug"
-    ENABLE_DEBUGOUTPUT="--disable-debugoutput"
-    
-    # By default, parts of Singular are compiled with -O2 and parts
-    # with -O3. If we do set any CFLAGS, this always overrides the
-    # default CFLAGS set by upstream. In order to be compatible, we
-    # set the optimization to -O2.
-    # Increasing the optimization level to -O3 has caused various
-    # problems in the past either with specific compilers or on specific
-    # platforms.
-    OPTIMIZATION_FLAGS="-O2"
-    
-    CFLAGS="$OPTIMIZATION_FLAGS -g $CFLAGS"
-    CXXFLAGS="$OPTIMIZATION_FLAGS -g $CXXFLAGS"
 fi

and

checking whether build environment is sane... yes
configure: WARNING: Please note that we set empty defaults for `CFLAGS' and `CXXFLAGS' (instead of `-g -O')
checking whether C compiler accepts ... yes

(One of seven instances.)

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​0ed156d

Singular 4.x (#17254): Restore setting defaults for CFLAGS and CXXFLAGS in spkg-install, minor fixes.

[leif]

Looks like Singular was currently using -O3 in many (if not all?) places though, despite the warning(s). Anyway, I've restored the old [ "$SAGE_DEBUG" != yes ] branch, for the moment at least.

[leif]

Doctest failure in src/sage/schemes/curves/affine_curve.py, line 494, more readable:

doctest/

@@ -1 +1 @@
     C.resolution_of_singularities(extend=True)[0] # long time (2 seconds)
-Expected:
+Got:
     (Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by
     24*x^2*ss1^3 + 24*ss1^3 + (a0^3 - 8*a0),
      Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by
-     24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (6*a0^3)*s1,
+     24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (-6*a0^3)*s1,
      Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by
-     8*y^2*s0^4 + (-4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y)
+     8*y^2*s0^4 + (4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y)
 

(No idea where the different line breaks in the actual output come from.)

[dimpase]

Replying to leif:

Doctest failure in src/sage/schemes/curves/affine_curve.py, line 494, more readable:

I've reported this one already (it's actually a different answer...)

[leif]

Replying to dimpase:

Replying to leif:

Doctest failure in src/sage/schemes/curves/affine_curve.py, line 494, more readable:

I've reported this one already (it's actually a different answer...)

There's been some change to that file in 7.4.beta3, haven't checked yet where / whether it's related.

---

In spkg-install, there's no error checking done in install_docs() and fix_includes(), haven't touched these though.

[leif]

Replying to leif:

Replying to dimpase:

still failing on 32-bit Ubuntu 14.04 with gcc 4.8.4

When you get an error during building the Sage library, please do

env SAGE_NUM_THREADS=1 ./sage -b

and post the log of that.

P.S.: FSF GCC 4.8.5 works for me on 64-bit Linux. (Just the doctest failure above.)

[leif]

Replying to leif:

Replying to dimpase:

Replying to leif:

Doctest failure in src/sage/schemes/curves/affine_curve.py, line 494, more readable:

I've reported this one already (it's actually a different answer...)

There's been some change to that file in 7.4.beta3, haven't checked yet where / whether it's related.

Apparently not. I'm getting the same with the branch pulled into Sage 7.4.beta3 (as opposed to fetching it, since it's based on beta2 with the Linbox upgrade manually merged in by Dima IIRC).

Presumably unrelated, with GCC 6.2:

sage -t --long src/sage/parallel/map_reduce.py
**********************************************************************
File "src/sage/parallel/map_reduce.py", line 217, in sage.parallel.map_reduce
Failed example:
    try:
        res = EX.run(timeout=0.01)
    except AbortError:
        print("Computation timeout")
    else:
        print("Computation normally finished")
        res
Expected:
    Computation timeout
Got:
    Computation normally finished
    40320*x^8 + 5040*x^7 + 720*x^6 + 120*x^5 + 24*x^4 + 6*x^3 + 2*x^2 + x + 1
**********************************************************************
1 item had failures:
   1 of  47 in sage.parallel.map_reduce
    [296 tests, 1 failure, 27.61 s]

XD

(Passes when rerun separately though.)

[dimpase]

Replying to leif:

Replying to dimpase:

still failing on 32-bit Ubuntu 14.04 with gcc 4.8.4

When you get an error during building the Sage library, please do

env SAGE_NUM_THREADS=1 ./sage -b

and post the log of that.

Right, sorry for the multithread mess. With singular built with SAGE_DEBUG=yes I get

[  1/308] gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wno-unused -fPIC -I/home/dima/sage/dev/sage/local/include -I/home/dima/sage/dev/sage/local/include/python2.7 -I/home/dima/sage/dev/sage/local/lib/python2.7/site-packages/numpy/core/include -I/home/dima/sage/dev/sage/src -I/home/dima/sage/dev/sage/src/sage/ext -I/home/dima/sage/dev/sage/src/build/cythonized -I/home/dima/sage/dev/sage/src/build/cythonized/sage/ext -I/home/dima/sage/dev/sage/local/include/python2.7 -c /home/dima/sage/dev/sage/src/build/cythonized/sage/algebras/letterplace/free_algebra_letterplace.cpp -o build/temp.linux-i686-2.7/home/dima/sage/dev/sage/src/build/cythonized/sage/algebras/letterplace/free_algebra_letterplace.o -fno-strict-aliasing -fno-tree-copyrename
In file included from /home/dima/sage/dev/sage/local/include/singular/kernel/structs.h:12:0,
                 from /home/dima/sage/dev/sage/local/include/singular/Singular/libsingular.h:8,
                 from /home/dima/sage/dev/sage/src/build/cythonized/sage/algebras/letterplace/free_algebra_letterplace.cpp:348:
/home/dima/sage/dev/sage/local/include/omalloc/omalloc.h:11:30: fatal error: omalloc/omConfig.h: No such file or directory
 #include <omalloc/omConfig.h>
                              ^

And without SAGE_DEBUG the error is

[  1/302] gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wno-unused -fPIC -I/home/dima/sage/dev/sage/local/include -I/home/di
ma/sage/dev/sage/local/include/python2.7 -I/home/dima/sage/dev/sage/local/lib/python2.7/site-packages/numpy/core/include -I/home/dima/sage/de
v/sage/src -I/home/dima/sage/dev/sage/src/sage/ext -I/home/dima/sage/dev/sage/src/build/cythonized -I/home/dima/sage/dev/sage/src/build/cytho
nized/sage/ext -I/home/dima/sage/dev/sage/local/include/python2.7 -c /home/dima/sage/dev/sage/src/build/cythonized/sage/libs/singular/singula
r.cpp -o build/temp.linux-i686-2.7/home/dima/sage/dev/sage/src/build/cythonized/sage/libs/singular/singular.o -I/home/dima/sage/dev/sage/loca
l//include -fPIC -I/home/dima/sage/dev/sage/local/include -std=gnu++11 -mmmx -mpopcnt -msse -msse2 -msse3 -msse4.1 -msse4.2 -mavx -mfpmath=ss
e -fabi-version=6 -fno-strict-aliasing -fno-tree-copyrename
/home/dima/sage/dev/sage/src/build/cythonized/sage/libs/singular/singular.cpp: In function ‘__pyx_obj_4sage_5rings_12finite_rings_14element_g
ivaro_FiniteField_givaroElement* __pyx_f_4sage_4libs_8singular_8singular_si2sa_GFqGivaro(snumber*, ip_sring*, __pyx_obj_4sage_5rings_12finite
_rings_14element_givaro_Cache_givaro*)’:
/home/dima/sage/dev/sage/src/build/cythonized/sage/libs/singular/singular.cpp:4218:104: error: call of overloaded ‘init(int&, long int)’ is a
mbiguous
     __pyx_v_c = __pyx_v_cache->objectptr->init(__pyx_v_c, ((long)p_GetCoeff(__pyx_v_z, __pyx_v_cfRing)));
                                                                                                        ^
/home/dima/sage/dev/sage/src/build/cythonized/sage/libs/singular/singular.cpp:4218:104: note: candidates are:

[..long list omitted by me - see err.log for the complete listing..] 

[dimpase]

32-bit Ubuntu 14.04 with gcc 4.8.4 error

[leif]

Replying to jdemeyer:

I think this regression is not acceptable:

             sage: P.<x,y,z> = PolynomialRing(QQ)
             sage: f= x^2 + 2*x*y + 1/2*z
             sage: f.is_squarefree()
             Traceback (most recent call last):
             ...
             NotImplementedError: is_squarefree of multivariate polynomials over rings is not implemented.

Asking whether a multivariate polynomial is squarefree seems like a very natural thing to do and not something you should just break.

How about

src/sage/rings/polynomial/multi_polynomial_libsingular.pyx

@@ -4703 +4703 @@
             sage: P.<x,y,z> = PolynomialRing(QQ)
             sage: f= x^2 + 2*x*y + 1/2*z
             sage: f.is_squarefree()
-            Traceback (most recent call last):
-            ...
-            NotImplementedError: is_squarefree of multivariate polynomials over rings is not implemented.
+            True
             sage: h = f^2
             sage: h.is_squarefree()
-            Traceback (most recent call last):
-            ...
-            NotImplementedError: is_squarefree of multivariate polynomials over rings is not implemented.
+            False
         """
-        raise NotImplementedError("is_squarefree of multivariate polynomials over rings is not implemented.")
+        return all([ e==1 for (f,e) in self.factor() ])
 
     @coerce_binop
     def quo_rem(self, MPolynomial_libsingular right):

for the moment?

We could improve it later, probably on a follow-up.

[leif]

Replying to dimpase:

Right, sorry for the multithread mess. With singular built with SAGE_CHECK=yes I get

I guess you meant SAGE_DEBUG=yes (and SAGE_NUM_THREADS=1).

[leif]

I'll try to revive a 32-bit x86 machine the next days, as we have other issues on 32-bit Ubuntu 14.04 and 16.04 (i.e., building bdists for them) as well.

[dimpase]

Replying to leif:

Replying to dimpase:

Right, sorry for the multithread mess. With singular built with SAGE_CHECK=yes I get

I guess you meant SAGE_DEBUG=yes (and SAGE_NUM_THREADS=1).

oops, yes that's what I meant.

[dimpase]

Replying to leif:

I'll try to revive a 32-bit x86 machine the next days, as we have other issues on 32-bit Ubuntu 14.04 and 16.04 (i.e., building bdists for them) as well.

I've installed gcc-6.1.1 on my 32-bit system and am rebuilding Sage with it now. Hopefully it will fix the C++ madness...

[leif]

Replying to dimpase:

Replying to leif:

I'll try to revive a 32-bit x86 machine the next days, as we have other issues on 32-bit Ubuntu 14.04 and 16.04 (i.e., building bdists for them) as well.

I've installed gcc-6.1.1 on my 32-bit system and am rebuilding Sage with it now. Hopefully it will fix the C++ madness...

Not really GCC-related I'd say, but rather 32-bit vs. 64-bit.

Does the following work for you?

src/sage/libs/singular/singular.pyx

@@ -150 +150 @@
     order = cache.objectptr.cardinality() - 1
 
     while z:
-        c = cache.objectptr.initi(c, <long>p_GetCoeff(z, cfRing))
+        c = cache.objectptr.initi(c, <int64_t>p_GetCoeff(z, cfRing))
         e = p_GetExp(z, 1, cfRing)
         if e == 0:
             ret = cache.objectptr.add(ret, c, ret)

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​fe8df87	Singular 4.x (#17254): Fix a few integer types w.r.t. width/signedness in singular.{pxd,pyx}.
​8dcddb1	Singular 4.x (#17254): Temporarily (re-)implement is_squarefree() for mpolys.

[leif]

I pushed a (hopefully sufficient) fix for the 32-bit issue and a slightly beautified version of the is_squarefree() implementation above.

(Not sure when I'll be back.)

[dimpase]

Replying to leif:

I pushed a (hopefully sufficient) fix for the 32-bit issue and a slightly beautified version of the is_squarefree() implementation above.

OK, at least now it builds on 32-bit.

[dimpase]

Replying to dimpase:

Replying to leif:

I pushed a (hopefully sufficient) fix for the 32-bit issue and a slightly beautified version of the is_squarefree() implementation above.

OK, at least now it builds on 32-bit.

OK, long tests pass too, modulo already reported here affine_curves test, and some unrelated numerical noise in gsl (see #21417, needs review).

[jpflori]

Replying to dimpase:

Replying to leif:

Replying to dimpase:

still failing on 32-bit Ubuntu 14.04 with gcc 4.8.4

When you get an error during building the Sage library, please do

env SAGE_NUM_THREADS=1 ./sage -b

and post the log of that.

Right, sorry for the multithread mess. With singular built with SAGE_DEBUG=yes I get

[  1/308] gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wno-unused -fPIC -I/home/dima/sage/dev/sage/local/include -I/home/dima/sage/dev/sage/local/include/python2.7 -I/home/dima/sage/dev/sage/local/lib/python2.7/site-packages/numpy/core/include -I/home/dima/sage/dev/sage/src -I/home/dima/sage/dev/sage/src/sage/ext -I/home/dima/sage/dev/sage/src/build/cythonized -I/home/dima/sage/dev/sage/src/build/cythonized/sage/ext -I/home/dima/sage/dev/sage/local/include/python2.7 -c /home/dima/sage/dev/sage/src/build/cythonized/sage/algebras/letterplace/free_algebra_letterplace.cpp -o build/temp.linux-i686-2.7/home/dima/sage/dev/sage/src/build/cythonized/sage/algebras/letterplace/free_algebra_letterplace.o -fno-strict-aliasing -fno-tree-copyrename
In file included from /home/dima/sage/dev/sage/local/include/singular/kernel/structs.h:12:0,
                 from /home/dima/sage/dev/sage/local/include/singular/Singular/libsingular.h:8,
                 from /home/dima/sage/dev/sage/src/build/cythonized/sage/algebras/letterplace/free_algebra_letterplace.cpp:348:
/home/dima/sage/dev/sage/local/include/omalloc/omalloc.h:11:30: fatal error: omalloc/omConfig.h: No such file or directory
 #include <omalloc/omConfig.h>
                              ^

Can you try to remove this line from omallc/omalloc.h and restart the SAge library build?

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​d69ac3d	Remove Singular omalloc stuff.
​aad26dc	Fix non-existing include in singular's xalloc.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​51502d6

Simplify CFLAGS treatment for Singular.

[jpflori]

Replying to jdemeyer:

If I recall correctly, on Mac OS we might have both (".so" and ".dylib") or only one of (".so" and ".dylib")? Is that true??

Both exist for a different purpose. On OS X:

• .dylib = dynamic library (needed for the usual case of linking libraries at build-time).

• .so = loadable module (needed for run-time linking, for example Python extension modules).

On most other OSes, there is no difference between these.

I guess we should only deal with dylib on OS X as far as Singular is concerned. Can someone confirm this?

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​536d849

Make sure to delete Singular 3.x files.

[fbissey]

Replying to jpflori:

Replying to jdemeyer:

If I recall correctly, on Mac OS we might have both (".so" and ".dylib") or only one of (".so" and ".dylib")? Is that true??

Both exist for a different purpose. On OS X:

• .dylib = dynamic library (needed for the usual case of linking libraries at build-time).

• .so = loadable module (needed for run-time linking, for example Python extension modules).

On most other OSes, there is no difference between these.

I guess we should only deal with dylib on OS X as far as Singular is concerned. Can someone confirm this?

I cannot think of another platform producing dylib so it's fair.

[jpflori]

Sorry, I was not clear: I meant that Singular produces no .so file on OS X.

[dimpase]

with SAGE_DEBUG=yes ./sage -f singular on 32-bit Linux I get

[singular-4.0.3p3]   CXX      libSingular_la-mod_lib.lo
[singular-4.0.3p3] misc_ip.cc: In function 'char* versionString()':
[singular-4.0.3p3] misc_ip.cc:783:34: error: 'SIZEOF_VOIDP' was not declared in this scope
[singular-4.0.3p3]                 SINGULAR_VERSION, SIZEOF_VOIDP*8,
[singular-4.0.3p3]                                   ^~~~~~~~~~~~
[singular-4.0.3p3] make[6]: *** [libSingular_la-misc_ip.lo] Error 1

[leif]

Replying to git:

Branch pushed to git repo; I updated commit sha1. New commits:

​51502d6

Simplify CFLAGS treatment for Singular.

You just undid common subexpression elimination... ;-)

[jpflori]

True.

I just thought it was more readable for a human being. And was in line with the if clause. But I might be wrong.

[leif]

Replying to jpflori:

And was in line with the if clause.

Well, it's more likely we add things to OPTIMIZATION_FLAGS than to / past -O0.

[jpflori]

Replying to dimpase:

with SAGE_DEBUG=yes ./sage -f singular on 32-bit Linux I get

[singular-4.0.3p3]   CXX      libSingular_la-mod_lib.lo
[singular-4.0.3p3] misc_ip.cc: In function 'char* versionString()':
[singular-4.0.3p3] misc_ip.cc:783:34: error: 'SIZEOF_VOIDP' was not declared in this scope
[singular-4.0.3p3]                 SINGULAR_VERSION, SIZEOF_VOIDP*8,
[singular-4.0.3p3]                                   ^~~~~~~~~~~~
[singular-4.0.3p3] make[6]: *** [libSingular_la-misc_ip.lo] Error 1

Got it: omConfig.h is needed at build time to define this... So the easiest solution would be not to modify xalloc/omalloc.h and to ship omConfig.h jsut as omalloc does. Can you modify xalloc/configure.ac by adding

nodist_libomalloc_include_HEADERS = omConfig.h

then autoreconf and retest?

[fbissey]

Replying to jpflori:

Sorry, I was not clear: I meant that Singular produces no .so file on OS X.

I can only see dylib around here. The only .so files are in lib/python2.7 (and they would be .bundle if we were building python as a framework).

[fbissey]

I have started working in earnest on a ebuild for sage-on-gentoo. I based on it on stuff in the maintree for 4.0.2.

Anyway I believe that by default --enable-gfanlib is on. libgfan needs cddlib to be installed to build. Since cddlib is not currently a dependency of singular, but is a standard package, we could get different singular installs depending on build order.

Since cddlib is a standard package I propose we make it a dependency of singular - unless there are reasons to have gfanlib to be completely disabled.

[leif]

Replying to fbissey:

Anyway I believe that by default --enable-gfanlib is on. libgfan needs cddlib to be installed to build. Since cddlib is not currently a dependency of singular, but is a standard package, we could get different singular installs depending on build order.

Since cddlib is a standard package I propose we make it a dependency of singular - unless there are reasons to have gfanlib to be completely disabled.

I don't think so. Go ahead...

[leif]

Replying to leif:

Replying to fbissey:

Anyway I believe that by default --enable-gfanlib is on. libgfan needs cddlib to be installed to build. Since cddlib is not currently a dependency of singular, but is a standard package, we could get different singular installs depending on build order.

Since cddlib is a standard package I propose we make it a dependency of singular - unless there are reasons to have gfanlib to be completely disabled.

I don't think so. Go ahead...

P.S.: cddlib just depends on GMP/MPIR, which is built (nearly) first anyway, so adding the dependency wouldn't slow down anything either.

[dimpase]

G(roebner)Fans are important, let's enable this.

[leif]

Replying to dimpase:

G(roebner)Fans are important, let's enable this.

AFAICS they're only accessible through the Singular binary, and/but we already also have gfan.

[fbissey]

Replying to leif:

Replying to dimpase:

G(roebner)Fans are important, let's enable this.

AFAICS they're only accessible through the Singular binary, and/but we already also have gfan.

Indeed, but there is a libgfan.so installed (linking may need improvement, I'll look into that) and could be used directly instead of going through a pexpect interface. Of course functionality and performance are also a consideration.

[jpflori]

Replying to jpflori:

Replying to dimpase:

with SAGE_DEBUG=yes ./sage -f singular on 32-bit Linux I get

[singular-4.0.3p3]   CXX      libSingular_la-mod_lib.lo
[singular-4.0.3p3] misc_ip.cc: In function 'char* versionString()':
[singular-4.0.3p3] misc_ip.cc:783:34: error: 'SIZEOF_VOIDP' was not declared in this scope
[singular-4.0.3p3]                 SINGULAR_VERSION, SIZEOF_VOIDP*8,
[singular-4.0.3p3]                                   ^~~~~~~~~~~~
[singular-4.0.3p3] make[6]: *** [libSingular_la-misc_ip.lo] Error 1

Got it: omConfig.h is needed at build time to define this... So the easiest solution would be not to modify xalloc/omalloc.h and to ship omConfig.h jsut as omalloc does. Can you modify xalloc/configure.ac by adding

nodist_libomalloc_include_HEADERS = omConfig.h

then autoreconf and retest?

This seems to work. I'll produce a patch.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​4865a48

Make sure Singular's xalloc installs omConfig.h.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​98cc68c

Only try relevant extension for OS when loading libSingular.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​399f051	Add cddlib as a dependency and make gfanlib building explicit
​797c539	Improve the cleaning

[fbissey]

And I am in. There are several other stuff I want to raise or do related to packaging: 1) drop the separate (re-)building of libfactory. Historically this was done to build maccauley2 (anyone else remember that?). My understanding is that upstream maccauley2 includes its own internal copy now. And there is no new style spkg for it currently. Any objections to ditching it? 2) singular install .pc files we should use them

[tscrim]

Replying to fbissey:

1) drop the separate (re-)building of libfactory. Historically this was done to build maccauley2 (anyone else remember that?). My understanding is that upstream maccauley2 includes its own internal copy now. And there is no new style spkg for it currently. Any objections to ditching it?

I'm guessing you're talking about dropping libfactory since Macaulay2 is an experimental spkg, so I have no objections..

[fbissey]

It is not really dropping it since it is naturally produced and installed when building singular. But in singular 3.x it was a static library only and libsingular had its own copy inside. In singular 4, libfactory is dynamic and libsingular depends on it...

In the past we were rebuilding libfactory.a with a slightly different configuration to satisfy Macauley2 and it had no impact on libsingular. This is potentially not the case anymore. I haven't checked but the rebuilding may just be costing us time with no final difference.

[fbissey]

I am almost done with my first pass at using Singular.pc. Once it is done there may be simplifications that can be done in module_list.py. But I need to clear the view and be sure that it works as it is first.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​75e383a

use Singular.pc almost everywhere

[dimpase]

Replying to fbissey:

And I am in. There are several other stuff I want to raise or do related to packaging: 1) drop the separate (re-)building of libfactory. Historically this was done to build maccauley2 (anyone else remember that?). My understanding is that upstream maccauley2 includes its own internal copy now. And there is no new style spkg for it currently. Any objections to ditching it? 2) singular install .pc files we should use them

well, AFAIK Macaulay2 uses an unpatched version of libfactory. (more precisely they apply ​this, but it's just documentation, right?)

I recently spent some time trying to resurrect M2 Sage package, so I'd prefer libfactory as it is now, a separate one.

[fbissey]

Replying to dimpase:

Replying to fbissey: I recently spent some time trying to resurrect M2 Sage package, so I'd prefer libfactory as it is now, a separate one.

I will look into this further. But rebuilding libfactory is a risk. I would be OK if libfactory was a separate package and singular and M2 would build on top of it but that's not what it looks like now.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​a0f55ea

Singular.pc is now used everywhere. plural.pyx doesn't need givaro or readline or m anymore.

[jpflori]

Replying to git:

Branch pushed to git repo; I updated commit sha1. New commits:

​399f051	Add cddlib as a dependency and make gfanlib building explicit
​797c539	Improve the cleaning

I see libomalloc's in my loca/lib but no libSingomalloc's. Is the latter one what 3.x used to install? Whatsoever we could still remove the former one as well.

[fbissey]

That's a mistake. I really don't know how I did that, looks like a misplaced cut and paste.

[jpflori]

Maybe we could also use some more wildcards in module_list.py, especially for sage/libs/singular?

[fbissey]

Yes that's why I said it was a first pass. Once you have done that, we have a better idea about potential wildcards. sage/libs/singular and sage/algebras/letterplace both look like they could be reduced to wildcards.

[fbissey]

I am done for the day. No more commits from me for the next ~10hours at least.

[leif]

Replying to dimpase:

Replying to fbissey:

And I am in. There are several other stuff I want to raise or do related to packaging: 1) drop the separate (re-)building of libfactory. Historically this was done to build maccauley2 (anyone else remember that?). My understanding is that upstream maccauley2 includes its own internal copy now. And there is no new style spkg for it currently. Any objections to ditching it? 2) singular install .pc files we should use them

well, AFAIK Macaulay2 uses an unpatched version of libfactory. (more precisely they apply ​this, but it's just documentation, right?)

I recently spent some time trying to resurrect M2 Sage package, so I'd prefer libfactory as it is now, a separate one.

Rebuilding libfactory as we do at the moment is nothing but redundant, so we can safely remove the step.

[jpflori]

Yes, I'm currently cleaning up that and the "removal" part of spkg-install. Pleas hold on.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​55447da	Do not build (a second time) libfactory when building Singular.
​e5c10ae	Cleaner removal of files.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​29077a7

Use wildcards for singular and letterplace in module_list.py.

[jpflori]

Any more work needed?

[leif]

What do we do with the failing doctest? (The patch/buildbots won't like it.)

We could tag it # known bug for the moment, for example, and open a follow-up for fixing it later.

[jpflori]

Replying to leif:

What do we do with the failing doctest? (The patch/buildbots won't like it.)

Hum, I did not think about this one. What is still failing? Only different values due to the upgrade? Or still serious stufF?

[leif]

Replying to jpflori:

Replying to leif:

What do we do with the failing doctest? (The patch/buildbots won't like it.)

Hum, I did not think about this one. What is still failing? Only different values due to the upgrade? Or still serious stufF?

Ask Dima (cf. comment 365).

[jpflori]

Yes this is the only thing I located here... I have no idea either. I'll ask on sage-devel.

[leif]

Replying to leif:

Replying to jpflori:

Replying to leif:

What do we do with the failing doctest? (The patch/buildbots won't like it.)

Hum, I did not think about this one. What is still failing? Only different values due to the upgrade? Or still serious stufF?

Ask Dima (cf. comment 365).

P.S.: The whole doctest that fails now is

sage: set_verbose(-1)
sage: K.<a> = QuadraticField(3)
sage: A.<x,y> = AffineSpace(K, 2)
sage: C = A.curve(x^4 + 2*x^2 + a*y^3 + 1)
sage: C.resolution_of_singularities(extend=True)[0] # long time (2 seconds)
(Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by
24*x^2*ss1^3 + 24*ss1^3 + (a0^3 - 8*a0),
 Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by
 24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (6*a0^3)*s1,
 Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by
 8*y^2*s0^4 + (-4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y)

[bhutz]

I don't have this ticket built, what is the doctest returning in the new version?

[dimpase]

Replying to bhutz:

I don't have this ticket built, what is the doctest returning in the new version?

see ​https://trac.sagemath.org/ticket/17254#comment:364

[leif]

Note that in both cases (old and new Singular) one gets

verbose 0 (3369: multi_polynomial_ideal.py, groebner_basis) Warning: falling back to very slow toy implementation.
verbose 0 (1081: multi_polynomial_ideal.py, dimension) Warning: falling back to very slow toy implementation.
verbose 0 (3369: multi_polynomial_ideal.py, groebner_basis) Warning: falling back to very slow toy implementation.
verbose 0 (1081: multi_polynomial_ideal.py, dimension) Warning: falling back to very slow toy implementation.
verbose 0 (2086: multi_polynomial_ideal.py, variety) Warning: falling back to very slow toy implementation.
verbose 0 (3369: multi_polynomial_ideal.py, groebner_basis) Warning: falling back to very slow toy implementation.
verbose 0 (1081: multi_polynomial_ideal.py, dimension) Warning: falling back to very slow toy implementation.
verbose 0 (3369: multi_polynomial_ideal.py, groebner_basis) Warning: falling back to very slow toy implementation.
verbose 0 (1081: multi_polynomial_ideal.py, dimension) Warning: falling back to very slow toy implementation.
verbose 0 (3369: multi_polynomial_ideal.py, groebner_basis) Warning: falling back to very slow toy implementation.
verbose 0 (1081: multi_polynomial_ideal.py, dimension) Warning: falling back to very slow toy implementation.
verbose 0 (2086: multi_polynomial_ideal.py, variety) Warning: falling back to very slow toy implementation.
verbose 0 (3369: multi_polynomial_ideal.py, groebner_basis) Warning: falling back to very slow toy implementation.
verbose 0 (1081: multi_polynomial_ideal.py, dimension) Warning: falling back to very slow toy implementation.
verbose 0 (3369: multi_polynomial_ideal.py, groebner_basis) Warning: falling back to very slow toy implementation.
verbose 0 (1081: multi_polynomial_ideal.py, dimension) Warning: falling back to very slow toy implementation.
verbose 0 (3369: multi_polynomial_ideal.py, groebner_basis) Warning: falling back to very slow toy implementation.
verbose 0 (1081: multi_polynomial_ideal.py, dimension) Warning: falling back to very slow toy implementation.
verbose 0 (2086: multi_polynomial_ideal.py, variety) Warning: falling back to very slow toy implementation.

anyway... ;-)

(Hence the set_verbose(-1) I guess.)

[jakobkroeker]

Two another minor(?) issues I could not solve due to lack of expertise in sage:

-reuse a plain polynomial ring over ZZ and

-reuse Singular polynomial ring over QQ

instead of creating a new one every time,

see

src/sage/libs/singular/singular.pyx

snip

        # if I understand nrnMapGMP/nMapFuncPtr correctly we need first
+        # a source value in ZZr
+        # create ZZr, a plain polynomial ring over ZZ with one variable.
+        #
+        # todo (later): reuse ZZr
+        _name = omStrDup(varname)
+        _ext_names = <char**>omAlloc0(sizeof(char*))
+        _ext_names[0] = omStrDup(_name)
+        _cf = nInitChar( n_Z, NULL) # integer coefficient ring
+        ZZr = rDefault (_cf ,1, _ext_names)
+        rComplete(ZZr,1)

+    # the result of nlInit2gmp() is in a plain polynomial ring over QQ (not an extension ring!),
+    # so we hace to get/create one :
+    #
+    # todo: reuse qqr/ get an existing Singular polynomial ring over Q.
+    varname = "a"
+    _name = omStrDup(varname)
+    cdef char **_ext_names
+    _ext_names = <char**>omAlloc0(sizeof(char*))
+    _ext_names[0] = omStrDup(_name)
+    qqr = rDefault( 0, 1, _ext_names);
+    rComplete(qqr,1)
+    qqr.ShortOut = 0}}}

[bhutz]

ok. I didn't see the diff on the doc test up there. It is possible it is just a coordinate change, but I'm going to have to get more information from it than the output is providing. I have it building now.

[leif]

FWIW, here's the full diff of the EXMAPLE^TM

sage: K.<a> = QuadraticField(3)
sage: A.<x,y> = AffineSpace(K, 2)
sage: C = A.curve(x^4 + a*y^3 +2*x^2 + 1)
sage: res = C.resolution_of_singularities(extend=True)
...

with Singular 3.1.7p1.p2 vs. 4.0.3p3, with all three components of the result:

.txt

@@ -24 +24 @@
 sage: res[0]
 
 (Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*x^2*ss1^3 + 24*ss1^3 + (a0^3 - 8*a0),
- Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (6*a0^3)*s1,
- Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 8*y^2*s0^4 + (-4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y)
+ Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (-6*a0^3)*s1,
+ Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 8*y^2*s0^4 + (4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y)
 sage: res[1]
 
 ((Scheme endomorphism of Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*x^2*ss1^3 + 24*ss1^3 + (a0^3 - 8*a0)
     Defn: Defined on coordinates by sending (x, ss1) to
           (x, ss1), Scheme morphism:
     From: Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*x^2*ss1^3 + 24*ss1^3 + (a0^3 - 8*a0)
-    To:   Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (6*a0^3)*s1
+    To:   Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (-6*a0^3)*s1
     Defn: Defined on coordinates by sending (x, ss1) to
-          (x*ss1 + (-1/8*a0^3)*ss1, 1/ss1), Scheme morphism:
+          (x*ss1 + (1/8*a0^3)*ss1, 1/ss1), Scheme morphism:
     From: Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*x^2*ss1^3 + 24*ss1^3 + (a0^3 - 8*a0)
-    To:   Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 8*y^2*s0^4 + (-4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y
+    To:   Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 8*y^2*s0^4 + (4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y
     Defn: Defined on coordinates by sending (x, ss1) to
-          (x^2*ss1 + ss1, 1/(x*ss1 + (-1/8*a0^3)*ss1))), (Scheme morphism:
-    From: Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (6*a0^3)*s1
+          (x^2*ss1 + ss1, 1/(x*ss1 + (1/8*a0^3)*ss1))), (Scheme morphism:
+    From: Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (-6*a0^3)*s1
     To:   Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*x^2*ss1^3 + 24*ss1^3 + (a0^3 - 8*a0)
     Defn: Defined on coordinates by sending (s1, ss0) to
-          (s1*ss0 + (1/8*a0^3), 1/ss0),
-  Scheme endomorphism of Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (6*a0^3)*s1
+          (s1*ss0 + (-1/8*a0^3), 1/ss0),
+  Scheme endomorphism of Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (-6*a0^3)*s1
     Defn: Defined on coordinates by sending (s1, ss0) to
           (s1, ss0),
   Scheme morphism:
-    From: Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (6*a0^3)*s1
-    To:   Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 8*y^2*s0^4 + (-4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y
+    From: Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (-6*a0^3)*s1
+    To:   Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 8*y^2*s0^4 + (4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y
     Defn: Defined on coordinates by sending (s1, ss0) to
-          (s1^2*ss0 + (1/4*a0^3)*s1, 1/s1)), (Scheme morphism:
-    From: Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 8*y^2*s0^4 + (-4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y
+          (s1^2*ss0 + (-1/4*a0^3)*s1, 1/s1)), (Scheme morphism:
+    From: Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 8*y^2*s0^4 + (4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y
     To:   Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*x^2*ss1^3 + 24*ss1^3 + (a0^3 - 8*a0)
     Defn: Defined on coordinates by sending (y, s0) to
-          (y*s0 + (-1/8*a0^3), 1/(y*s0^2 + (-1/4*a0^3)*s0)), Scheme morphism:
-    From: Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 8*y^2*s0^4 + (-4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y
-    To:   Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (6*a0^3)*s1
+          (y*s0 + (1/8*a0^3), 1/(y*s0^2 + (1/4*a0^3)*s0)), Scheme morphism:
+    From: Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 8*y^2*s0^4 + (4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y
+    To:   Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (-6*a0^3)*s1
     Defn: Defined on coordinates by sending (y, s0) to
-          (1/s0, y*s0^2 + (-1/4*a0^3)*s0), Scheme endomorphism of Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 8*y^2*s0^4 + (-4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y
+          (1/s0, y*s0^2 + (1/4*a0^3)*s0), Scheme endomorphism of Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 8*y^2*s0^4 + (4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y
     Defn: Defined on coordinates by sending (y, s0) to
           (y, s0)))
 sage: res[2]
@@ -68 +68 @@
    To:   Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by x^4 + (1/8*a0^3 - a0)*y^3 + 2*x^2 + 1
    Defn: Defined on coordinates by sending (x, ss1) to
          (x, x^2*ss1 + ss1), Scheme morphism:
-   From: Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (6*a0^3)*s1
+   From: Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 24*s1^2*ss0 + (a0^3 - 8*a0)*ss0^2 + (-6*a0^3)*s1
    To:   Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by x^4 + (1/8*a0^3 - a0)*y^3 + 2*x^2 + 1
    Defn: Defined on coordinates by sending (s1, ss0) to
-         (s1*ss0 + (1/8*a0^3), s1^2*ss0 + (1/4*a0^3)*s1), Scheme morphism:
-   From: Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 8*y^2*s0^4 + (-4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y
+         (s1*ss0 + (-1/8*a0^3), s1^2*ss0 + (-1/4*a0^3)*s1), Scheme morphism:
+   From: Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by 8*y^2*s0^4 + (4*a0^3)*y*s0^3 - 32*s0^2 + (a0^3 - 8*a0)*y
    To:   Affine Plane Curve over Number Field in a0 with defining polynomial y^4 - 4*y^2 + 16 defined by x^4 + (1/8*a0^3 - a0)*y^3 + 2*x^2 + 1
    Defn: Defined on coordinates by sending (y, s0) to
-         (y*s0 + (-1/8*a0^3), y))
+         (y*s0 + (1/8*a0^3), y))
 sage: 

[bhutz]

Yes, that helps, since this ticket doesn't build for me for some reason.

This looks to me like a change of coordinates (x to -x) at the first blow-up that is then carried through the rest of the stages. The connecting maps and the new curves are changed consistently so I see nothing incorrect about the new output, it is just a change of variables from the original blow-ups.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​494f8b8

Fix changed doctest because of different blow-up.

[jpflori]

Ok, I think we're good now. Some details are suboptimal, but most of them are documented in the source code (e.g. reuse singular rings).

Who wants to change the status and let the patchbots complain?

[leif]

Ok, I just randomly added reviewers such that Volker or some scripts won't complain.

And we should open (a) follow-up(s) for the mentioned further improvements.

But this really needs to get into Sage, hopefully already beta4.

[leif]

(Scrolling through hundreds of comments...)

Sorry, Dima, add yourself as author or reviewer (probably Codima as the latter ;-) ), and John Perry seems to have contributed as well, so please add anybody I missed.

[dimpase]

CoDima should go into "Viewers:" field, which is missing...

[leif]

...
2016-09-09 13:14:01 (10.8 MB/s) - ‘/tmp/tmpTW_qa8/singular-4.0.3p3.tar.bz2’ saved [12948313/12948313]

> Successfully uploaded
tar xf /tmp/tmpTW_qa8/singular-4.0.3p3.tar.bz2 -C /tmp/tmpTW_qa8
> Successfully unpacked
Sha1 of singular-4.0.3p3.tar.bz2 is c368b9dbe7e4d8abf75b49010f2bd9f426b1de11
> Correct sha1
Now comparing to previous spkg.
Unable to locate existing package singular.
http://perso.telecom-paristech.fr/~flori/singular-4.0.3p3.tar.bz2 -- 3 seconds
++++++++++ http://www.mathematik.uni-kl.de/ftp/pub/Math/Singular/SOURCES/4-0-3/singular-4.0.3p3.tar.gz ++++++++++
> name and version = singular-4.0.3p3
wget --progress=dot:mega -O /tmp/tmpj_5SgQ/singular-4.0.3p3.tar.gz http://www.mathematik.uni-kl.de/ftp/pub/Math/Singular/SOURCES/4-0-3/singular-4.0.3p3.tar.gz
--2016-09-09 13:14:02--  http://www.mathematik.uni-kl.de/ftp/pub/Math/Singular/SOURCES/4-0-3/singular-4.0.3p3.tar.gz
Resolving www.mathematik.uni-kl.de (www.mathematik.uni-kl.de)... 131.246.164.55
Connecting to www.mathematik.uni-kl.de (www.mathematik.uni-kl.de)|131.246.164.55|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12783052 (12M) [application/x-gzip]
Saving to: ‘/tmp/tmpj_5SgQ/singular-4.0.3p3.tar.gz’

     0K ........ ........ ........ ........ ........ ........ 24% 8.43M 1s
  3072K ........ ........ ........ ........ ........ ........ 49% 11.2M 1s
  6144K ........ ........ ........ ........ ........ ........ 73% 11.2M 0s
  9216K ........ ........ ........ ........ ........ ........ 98% 11.2M 0s
 12288K ...                                                  100% 11.3M=1.2s

2016-09-09 13:14:04 (10.4 MB/s) - ‘/tmp/tmpj_5SgQ/singular-4.0.3p3.tar.gz’ saved [12783052/12783052]

> Successfully uploaded
tar xf /tmp/tmpj_5SgQ/singular-4.0.3p3.tar.gz -C /tmp/tmpj_5SgQ
> Successfully unpacked
Sha1 of singular-4.0.3p3.tar.gz is de34fab01c9c58b3b6e073049a80c0c293fddded
Traceback (most recent call last):
  File "/home/sage/.local/lib/python2.7/site-packages/sage_patchbot/patchbot.py", line 1018, in test_a_ticket
    self.check_spkg(spkg)
  File "/home/sage/.local/lib/python2.7/site-packages/sage_patchbot/patchbot.py", line 1197, in check_spkg
    raise SkipTicket("spkg has incorrect sha1")
SkipTicket: spkg has incorrect sha1
...

I knew something like ​this would happen...

[vbraun]

Seems to work on all buildbots except OSX:

osx:build buildslave-sage$ ./local/bin/singular
-bash: ./local/bin/singular: Too many levels of symbolic links

Note that OSX is wonky with filename upper/lower case: Case is preserved but ignored when looking up files.

[fbissey]

Anyone knows why we deliberately create the link singular -> Singular? Is it for cygwin. Otherwise let's just drop it and correct the sage internals to use Singular. That shouldn't create any problem on OS X.

I just wish I had done a build on my OS X laptop before that came up.

[leif]

Hmmm, I didn't like

    # Lower case version is convenient (this can fail on
    # case-insensitive systems, we ignore the error).
    ln -sf Singular singular 2> /dev/null

from the beginning... ;-)

So this actually creates a symlink to itself on MacOS X. Nice.

Just skip symlinking on Darwin?

[leif]

... or remove the -s, making it a hard link, which is less dangerous.

[fbissey]

If no one can explain why we need singular on case sensitive system I think we should just drop it. There is no reason for keeping that kind of useless baggage.

[leif]

Replying to fbissey:

If no one can explain why we need singular on case sensitive system I think we should just drop it. There is no reason for keeping that kind of useless baggage.

Well, uppercase Singular is just as ugly and inconvenient as libSingular (and their dash-versions); we'd just have to make further changes like you did for the library name.

I personally don't care, but if it doesn't work on MacOS X while it's at the same time not necessary there either, why "disable" it for all other systems, having to make lots(?) of changes elsewhere?

[fbissey]

I think you are just pulling my leg because of #21430 :) In any case KISS principle dictate that we shouldn't mess up with what we install unless we need to. Someone did it in the past and the cost is either to keep carrying forward and keep paying, or just fix it once for all and pay a final bill. I also happen to have a fairly good idea of what needs to be fixed - because I used to do it in sage-on-gentoo.

My OS X laptop is ready for action before I push a fix.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​d2cab45	Do not create singular (all lower case) and only use Singular
​64b3710	clean up

[fbissey]

To my surprise there was only one spot with a call to singular instead of Singular left. Someone has done some cleaning. I also added a small clean up I had hanging up for ages. singular.py and singular.pyx where the two last files where SAGE_LOCAL is obtained through os.environ rather than sage/env.py.

I'd like to see this go through the bot again now.

[vbraun]

[sagelib-7.4.beta4] 
[sagelib-7.4.beta4] Error compiling Cython file:
[sagelib-7.4.beta4] ------------------------------------------------------------
[sagelib-7.4.beta4] ...
[sagelib-7.4.beta4]         extension = "dylib"
[sagelib-7.4.beta4]     else:
[sagelib-7.4.beta4]         extension = "so"
[sagelib-7.4.beta4] 
[sagelib-7.4.beta4]     # library name changed from libsingular to libSingular btw 3.x and 4.x
[sagelib-7.4.beta4]     lib = sage.env.SAGE_LOCAL+"/lib/libSingular."+extension
[sagelib-7.4.beta4]              ^
[sagelib-7.4.beta4] ------------------------------------------------------------
[sagelib-7.4.beta4] 
[sagelib-7.4.beta4] sage/libs/singular/singular.pyx:774:14: undeclared name not builtin: sage

[fbissey]

OK surprising but not a deal breaker.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​4d25cd8

use cleaner import syntax to get SAGE_LOCAL

[fbissey]

OK, all cleaner and I made sure it builds locally. One more time...

[leif]

Typo: form->from.

--- a/src/sage/interfaces/singular.py
+++ b/src/sage/interfaces/singular.py
@@ -2275,13 +2275,15 @@ def generate_docstring_dictionary():
sage: from sage.interfaces.singular import generate_docstring_dictionary
sage: generate_docstring_dictionary()
"""
+ form sage.env import SAGE_LOCAL
+
global nodes
global node_names
...

[leif]

I guess you meant form SAGE_LOCAL from sage.env... :-)

[leif]

Minor things:

sage.env also defines SAGE_SHARE.

We generally use os.path.join() rather than + and '/'.

[git]

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

​3740cae

Singular 4.x (#17254): Fix typo in interfaces/singular.py (syntax error).

[leif]

Replying to leif:

Typo: form->from.

--- a/src/sage/interfaces/singular.py
+++ b/src/sage/interfaces/singular.py
@@ -2275,13 +2275,15 @@ def generate_docstring_dictionary():
sage: from sage.interfaces.singular import generate_docstring_dictionary
sage: generate_docstring_dictionary()
"""
+ form sage.env import SAGE_LOCAL
+
global nodes
global node_names
...

Fixed. (Assuming the antipodeans are asleep.)

[leif]

Anyone looked into this:

checking whether to check for gfanlib... yes
checking setoper.h usability... yes
checking setoper.h presence... yes
checking for setoper.h... yes
checking cdd/setoper.h usability... no
checking cdd/setoper.h presence... no
checking for cdd/setoper.h... no
checking cddlib/setoper.h usability... no
checking cddlib/setoper.h presence... no
checking for cddlib/setoper.h... no
checking whether libcddgmp is usable... checking for dd_free_global_constants... yes
yes
no
checking whether to check for polymake interface... yes
checking for polymake-config... 0

and

checking for setoper.h... yes
checking cdd/setoper.h usability... no
checking cdd/setoper.h presence... no
checking for cdd/setoper.h... no
checking cddlib/setoper.h usability... no
checking cddlib/setoper.h presence... no
checking for cddlib/setoper.h... no
checking whether libcddgmp is usable... checking for dd_free_global_constants... yes
yes
no
checking whether to check for polymake interface... yes
checking for polymake-config... 0
checking whether non-commutative subsystem should be enabled... yes
checking whether to have ratGB... no
checking built-in modules... no
checking SI_BUILTINMODULES_ADD(add)...... unset
checking BUILTIN_LIBS...... unset

I meant the yes no (more results than checking; above you have to scroll to the right to see the third), which appears twice in cddlib context, and just noticed the polymake-config result.

(There are a few other instances where a result is shown without apparent matching checking.)

[fbissey]

Thanks Leif :) as for those messages m4/ax_gfan_check.m4 (I think it is called) is littered with AC_MSG_RESULT commands that will have that effect - I am guessing other files may be too. In sage-on-gentoo I decided to disable polymake checking until further notice. It looks like polymake may need to be built a certain way or something.

[leif]

Huh, another one:

sage -t --long --warn-long 98.3 src/sage/interfaces/tests.py
**********************************************************************
File "src/sage/interfaces/tests.py", line 42, in sage.interfaces.tests
Failed example:
    subprocess.call("echo syntax error | singular", **kwds)
Expected:
    0
Got:
    127
**********************************************************************
1 item had failures:
   1 of  18 in sage.interfaces.tests
    [17 tests, 1 failure, 6.53 s]
----------------------------------------------------------------------
sage -t --long --warn-long 98.3 src/sage/interfaces/tests.py  # 1 doctest failed
----------------------------------------------------------------------

(I actually forgot to look at the result of ptestlong which finished hours ago. 8-/ )

[git]

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

​76b7ea9

Singular 4.x (#17254): Fix last instance of lowercase singular executable usage.

[leif]

Fixed.

[fbissey]

Indeed I missed that one. I thought it was a message and of course on OS X (fake case sensitivity), where I ran tests, it passed.

[fbissey]

Darn I have one failure on OS X that doesn't exist on linux

Mirage:sage-7.2.beta5 fbissey$ ./sage -t --long --warn-long 62.6 src/sage/interfaces/singular.py
Running doctests with ID 2016-09-11-18-06-48-8b8f2392.
Git branch: singular4
Using --optional=mpir,python2,sage
Doctesting 1 file.
sage -t --long --warn-long 62.6 src/sage/interfaces/singular.py
**********************************************************************
File "src/sage/interfaces/singular.py", line 513, in sage.interfaces.singular.Singular._send_interrupt
Failed example:
    2*a
Exception raised:
    Traceback (most recent call last):
      File "/Users/fbissey/build/sage-7.2.beta5/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 498, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/fbissey/build/sage-7.2.beta5/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 861, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.interfaces.singular.Singular._send_interrupt[3]>", line 1, in <module>
        Integer(2)*a
      File "sage/rings/integer.pyx", line 1785, in sage.rings.integer.Integer.__mul__ (/Users/fbissey/build/sage-7.2.beta5/src/build/cythonized/sage/rings/integer.c:12530)
        return coercion_model.bin_op(left, right, operator.mul)
      File "sage/structure/coerce.pyx", line 1063, in sage.structure.coerce.CoercionModel_cache_maps.bin_op (/Users/fbissey/build/sage-7.2.beta5/src/build/cythonized/sage/structure/coerce.c:9153)
        raise
      File "sage/structure/coerce.pyx", line 1059, in sage.structure.coerce.CoercionModel_cache_maps.bin_op (/Users/fbissey/build/sage-7.2.beta5/src/build/cythonized/sage/structure/coerce.c:9091)
        return PyObject_CallObject(op, xy)
      File "sage/structure/element.pyx", line 1739, in sage.structure.element.RingElement.__mul__ (/Users/fbissey/build/sage-7.2.beta5/src/build/cythonized/sage/structure/element.c:15124)
        return (<RingElement>left)._mul_(right)
      File "sage/structure/element.pyx", line 1746, in sage.structure.element.RingElement._mul_ (/Users/fbissey/build/sage-7.2.beta5/src/build/cythonized/sage/structure/element.c:15309)
        cpdef _mul_(self, right):
      File "/Users/fbissey/build/sage-7.2.beta5/local/lib/python2.7/site-packages/sage/interfaces/interface.py", line 1385, in _mul_
        return self._operation('*', right)
      File "/Users/fbissey/build/sage-7.2.beta5/local/lib/python2.7/site-packages/sage/interfaces/interface.py", line 1264, in _operation
        raise TypeError(msg)
    TypeError: Singular error:
       ? `sage126` is not defined
       ? error occurred in or before STDIN line 12: `def sage135=sage127 * sage126;`
**********************************************************************
1 item had failures:
   1 of   5 in sage.interfaces.singular.Singular._send_interrupt
    [388 tests, 1 failure, 2.65 s]
----------------------------------------------------------------------
sage -t --long --warn-long 62.6 src/sage/interfaces/singular.py  # 1 doctest failed
----------------------------------------------------------------------
Total time for all tests: 2.8 seconds
    cpu time: 1.8 seconds
    cumulative wall time: 2.6 seconds

[leif]

Is it always exactly the same failure?

[fbissey]

yes.

[leif]

Ok, looking at the test (and the referenced Singular trac ticket), that's perhaps not too surprising, of course related to pexpect, and of course on MacOS X... 8-)

Can you reproduce the issue directly in Singular? (I.e., that a previously defined variable "vanishes" after pressing Ctrl-C on a following, incomplete line.)

(And when did it last pass?)

[fbissey]

I'll need instruction as I am only familiar with singular syntax for a few hours every other year :P when some debugging like this one seems necessary.

And of course I won't be able to apply it before sometime after getting up - likely after I dealt with my backlog of official work.

[leif]

I guess it's the last commit:

Branch: public/singular4
Commit: 76b7ea9ae8461b04870d329e886659b17badb000 (7.4.beta2 + 128 commits)

XD

---

You could also try running the test in verbose mode (I'm not sure if your Singular got restarted), and playing with the parameters in the test and _send_interrupt() (alarm delay, delay before sending ^C, time waiting for a prompt, and the number of times Sage tries to get the prompt, resending ;).

W.r.t. Singular syntax, compared to me you're presumably an expert. :-)

Otherwise commands have to be terminated with a ; (newline alone isn't sufficient), variable definitions look like def foo=42;, foo; evaluates the expression / prints the value of foo.

Hopefully Jeroen has a suited MacOS X box to investigate...

[vbraun]

I'm also getting this on Linux with SAGE_DEBUG=yes:

[sagetex-3.0] Done updating paths.
[sagetex-3.0] Traceback (most recent call last):
[sagetex-3.0]   File "example.sagetex.sage.py", line 4, in <module>
[sagetex-3.0]     from sage.all_cmdline import *   # import sage library
[sagetex-3.0]   File "/mnt/disk/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/all_cmdline.py", line 26, in <module>
[sagetex-3.0]     from sage.all import *
[sagetex-3.0]   File "/mnt/disk/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/all.py", line 103, in <module>
[sagetex-3.0]     from sage.rings.all      import *
[sagetex-3.0]   File "/mnt/disk/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/rings/all.py", line 54, in <module>
[sagetex-3.0]     from .number_field.all import *
[sagetex-3.0]   File "/mnt/disk/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/rings/number_field/all.py", line 9, in <module>
[sagetex-3.0]     from .totallyreal import enumerate_totallyreal_fields_prim
[sagetex-3.0]   File "sage/rings/number_field/totallyreal_data.pxd", line 12, in init sage.rings.number_field.totallyreal (build/cythonized/sage/rings/number_field/totallyreal.c:12118)
[sagetex-3.0]   File "sage/rings/number_field/totallyreal_data.pyx", line 39, in init sage.rings.number_field.totallyreal_data (build/cythonized/sage/rings/number_field/totallyreal_data.c:12869)
[sagetex-3.0]   File "/mnt/disk/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring_constructor.py", line 461, in PolynomialRing
[sagetex-3.0]     R = _single_variate(base_ring, name, sparse, implementation)
[sagetex-3.0]   File "/mnt/disk/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring_constructor.py", line 539, in _single_variate
[sagetex-3.0]     R = m.PolynomialRing_integral_domain(base_ring, name, sparse, implementation)
[sagetex-3.0]   File "/mnt/disk/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring.py", line 1582, in __init__
[sagetex-3.0]     sparse=sparse, element_class=element_class, category=category)
[sagetex-3.0]   File "/mnt/disk/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring.py", line 1453, in __init__
[sagetex-3.0]     sparse=sparse, element_class=element_class, category=category)
[sagetex-3.0]   File "/mnt/disk/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring.py", line 289, in __init__
[sagetex-3.0]     from sage.matrix.matrix_space import MatrixSpace
[sagetex-3.0]   File "/mnt/disk/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/matrix/matrix_space.py", line 58, in <module>
[sagetex-3.0]     from . import matrix_mpolynomial_dense
[sagetex-3.0]   File "sage/rings/polynomial/multi_polynomial_libsingular.pxd", line 6, in init sage.matrix.matrix_mpolynomial_dense (build/cythonized/sage/matrix/matrix_mpolynomial_dense.cpp:9160)
[sagetex-3.0] ImportError: /mnt/disk/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/rings/polynomial/multi_polynomial_libsingular.so: undefined symbol: errorreported
[sagetex-3.0] Error running Sage on example.sagetex.sage!

[fbissey]

OK, at least that one I think I can understand. A -D... is probably incorrect in Singular.pc or the whole build system, when you build it with debugging. I have fought for a long time with -DNDEBUG in singular 3 because I don't add it by default in sage-on-gentoo.

You get exactly the same kind of error (in reverse).

[fbissey]

Hum.... building from scratch I didn't seem to encounter this problem here. Access to the failed build would be needed here.

[fbissey]

OK, omalloc.pc is not produced with SAGE_DEBUG=yes and it didn't fail for me because the system one was eventually found (it is part of the require line in Singular.pc).

So this is a very real problem.

[fbissey]

The long term solution would be to have xalloc to produce its own omalloc.pc and ship it. This will have to go in spkg-src I am surprised at the fact that there is no apparent patching in spkg-src, and where is the stuff from patches/conditional applied?

[fbissey]

New tarball, updated branch to follow.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​133832d	Merge branch 'develop' into singular4
​8a99295	Add a .pc file to xalloc
​cda6a68	Remove old, unused patch folder

[fbissey]

Remain the issue on OS X if someone has a clue and instructions on how to test for someone that doesn't know how to use singular.

[jpflori]

Replying to fbissey:

The long term solution would be to have xalloc to produce its own omalloc.pc and ship it. This will have to go in spkg-src I am surprised at the fact that there is no apparent patching in spkg-src, and where is the stuff from patches/conditional applied?

Nope, I prefered to produce a patch for the autotools input and output files and apply it just before configuring. While we're at it, we could also update the xalloc patch part about omConfig.h to include upstream solution:

• ​https://github.com/Singular/Sources/commit/bebd5a0361ed499fa0769cad9424e1bb193acd7c

Can you also report the pkgconfig thing to upstream?

[jpflori]

(I've reported the omalloc.pc issue.)

[fbissey]

Replying to jpflori:

Replying to fbissey:

The long term solution would be to have xalloc to produce its own omalloc.pc and ship it. This will have to go in spkg-src I am surprised at the fact that there is no apparent patching in spkg-src, and where is the stuff from patches/conditional applied?

Nope, I prefered to produce a patch for the autotools input and output files and apply it just before configuring. While we're at it, we could also update the xalloc patch part about omConfig.h to include upstream solution:

• ​https://github.com/Singular/Sources/commit/bebd5a0361ed499fa0769cad9424e1bb193acd7c

Does that replace what I have called src/xalloc-omconfig.patch in my last series of push?

[jpflori]

Yes, exactly.

[fbissey]

And one new tarball coming then :)

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​3c8503c

adopt new upstream patch for xalloc. Make patching in spkg-src follow best practise.

[fbissey]

Tarball changed in place. I could have broken upstream patch in bits triggering a rerun of autotools and the one that don't. But I preferred to keep it as one piece.

[jpflori]

Ok, so it seems we are left with the pexpect issue on OS X.

[leif]

 506 files changed, 11695 insertions(+), 5407 deletions(-)

Thanks.

---

With GCC 4.8.5 and SAGE_DEBUG=yes, I'm getting quite a lot doctest errors (I don't get without):

----------------------------------------------------------------------
sage -t --long src/sage/algebras/free_algebra.py  # 4 doctests failed
sage -t --long src/sage/algebras/letterplace/free_algebra_element_letterplace.pyx  # 6 doctests failed
sage -t --long src/sage/algebras/letterplace/free_algebra_letterplace.pyx  # 2 doctests failed
sage -t --long src/sage/algebras/letterplace/letterplace_ideal.pyx  # 10 doctests failed
sage -t --long src/sage/rings/polynomial/plural.pyx  # Killed due to segmentation fault
sage -t --long src/sage/rings/quotient_ring.py  # 18 doctests failed
sage -t --long src/sage/rings/quotient_ring_element.py  # 1 doctest failed
sage -t --long src/sage/schemes/curves/constructor.py  # 1 doctest failed
----------------------------------------------------------------------

Will investigate later.

[leif]

P.S.: I also noticed there still seem to be left-overs:

-rwxr-xr-x 1 leif leif 1590232 2016-09-03 03:02 local/lib/p_Procs_FieldZp.so
-rwxr-xr-x 1 leif leif 2205960 2016-09-03 03:02 local/lib/p_Procs_FieldQ.so
-rwxr-xr-x 1 leif leif  167344 2016-09-03 03:02 local/lib/p_Procs_FieldIndep.so
-rwxr-xr-x 1 leif leif 4420816 2016-09-03 03:02 local/lib/p_Procs_FieldGeneral.so

[jpflori]

Replying to leif:

With GCC 4.8.5 and SAGE_DEBUG=yes, I'm getting quite a lot doctest errors (I don't get without):

----------------------------------------------------------------------
sage -t --long src/sage/algebras/free_algebra.py  # 4 doctests failed
sage -t --long src/sage/algebras/letterplace/free_algebra_element_letterplace.pyx  # 6 doctests failed
sage -t --long src/sage/algebras/letterplace/free_algebra_letterplace.pyx  # 2 doctests failed
sage -t --long src/sage/algebras/letterplace/letterplace_ideal.pyx  # 10 doctests failed
sage -t --long src/sage/rings/polynomial/plural.pyx  # Killed due to segmentation fault
sage -t --long src/sage/rings/quotient_ring.py  # 18 doctests failed
sage -t --long src/sage/rings/quotient_ring_element.py  # 1 doctest failed
sage -t --long src/sage/schemes/curves/constructor.py  # 1 doctest failed
----------------------------------------------------------------------

Hopefully these are just different printings of singular objects.

[tscrim]

Somehow, I don't think a segfault in plural.pyx is just from different printings. :p

[jakobkroeker]

I fail to build it as is:

make[2]: Entering directory '$sage-base/build/make'
sage-logger -p 'sage-spkg singular-4.0.3p3' '$sage-base/logs/pkgs/singular-4.0.3p3.log'
[singular-4.0.3p3] Found local metadata for singular-4.0.3p3
[singular-4.0.3p3] Invalid checksum for cached file $sage-base/upstream/singular-4.0.3p3.tar.bz2, deleting
[singular-4.0.3p3] Attempting to download package singular-4.0.3p3.tar.bz2 from mirrors
[singular-4.0.3p3] http://www.mirrorservice.org/sites/www.sagemath.org/spkg/upstream/singular/singular-4.0.3p3.tar.bz2
[singular-4.0.3p3] [......................................................................]
[singular-4.0.3p3] Traceback (most recent call last):
[singular-4.0.3p3]   File "$sage-base/build/bin/sage-download-file", line 28, in <module>
[singular-4.0.3p3]     run_safe()
[singular-4.0.3p3]   File "$sage-basebuild/bin/../sage_bootstrap/download/cmdline.py", line 118, in run_safe
[singular-4.0.3p3]     run()
[singular-4.0.3p3]   File "$sage-base/build/bin/../sage_bootstrap/download/cmdline.py", line 100, in run
[singular-4.0.3p3]     app.download_tarball(args.url_or_tarball, args.destination)
[singular-4.0.3p3]   File "$sage-base/build/bin/../sage_bootstrap/download/app.py", line 43, in download_tarball
[singular-4.0.3p3]     tarball.download()
[singular-4.0.3p3]   File "$sage-base/build/bin/../sage_bootstrap/tarball.py", line 163, in download
[singular-4.0.3p3]     raise ChecksumError('checksum does not match')
[singular-4.0.3p3] sage_bootstrap.tarball.ChecksumError: checksum does not match
Makefile:2761: recipe for target '$sage-base/local/var/lib/sage/installed/singular-4.0.3p3' failed

[vbraun]

Thats because of comment:485, changing the tarball in place will always lead to undesirable behavior.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​37fa8e7	Merge branch 'develop' into singular4
​d3ebc4a	additional clean up

[fbissey]

Just tidying up.

Replying to vbraun:

Thats because of comment:485, changing the tarball in place will always lead to undesirable behavior.

In this instance because the tarball that Volker uploaded on the mirror for testing doesn't match the one I linked to in the description. It is not so much the "in place" I am guessing as it is "same name" (which arguably are the same from the point of view of the computer). If it helps I'll change the tarball name at the first opportunity.

[jpflori]

That's also why I did not patch in spkg-src: you can keep the same tarball. Sure, then you have to include the autogenerated part of the patches into the patches... But all that we currently do is in upstream git so should be removed for the next upgrade.

[tscrim]

Could we try the latest snapshot from upstream to make a tarball so we don't have to worry about the patches?

[fbissey]

That would be a nice idea but I guess there may still be a need for running autoconf at some point. Actually upstream instruction on using xalloc are

README for xalloc: a debug helper and alternative to omalloc

- substitute in <topdir>/configure.ac and <topdir>/Makefile.am
   omalloc by xalloc
- in <topdir> run ./autogen.sh
- run <topdir>/configure .....   in <builddir>
- in <builddir> and <topdir>: save omalloc to omalloc.org, ln -s xalloc omalloc
- run make in <builddir>

So running autoconf before using xalloc is expected. Should we make a conditional patch to follow those instructions more closely. i.e. only apply the patch to configure{,.ac}/Makefile.{am,in} when DEBUG is enabled.

[jpflori]

Replying to fbissey:

That would be a nice idea but I guess there may still be a need for running autoconf at some point.

Sure there is.

Actually upstream instruction on using xalloc are

README for xalloc: a debug helper and alternative to omalloc

...

}}}

These instructions are buggy :) It is enough to cd to the xalloc dir, then autoreconf and if you want debug mode, then you delete the omalloc dir and mv xalloc omalloc.

To only use patch on top of the upstream 4.0.3p3 what you need to do is to autoreconf everything on your machine on a pristine tree and on a patched one and produce the patch. Then you ship the pristine tarball (or not so so pristine as we autoreconf it and indeed as you and I may use different autotools version the final autoreconfed dirs and patches may be different) and the patch to autotools input and output files. That's what I did before. And even if we use different autotools version it's not that hard to pick up the useful part of the patches produced by diff by hand and remove all the crap only due to different autotools versions.

[fbissey]

It is quite obvious to me that I shouldn't be making the next tarball. I just don't work that way. Jean-Pierre, do you mind to regenerate a new tarball and set of patches?

[mkoeppe]

I have no intentions to work on this ticket, but I am curious how people manage these sets of patches? I know Debian has a clear infrastructure and workflow for this, but we don't seem to have any in Sage. Lately, for packages scipoptsuite and gc (#21474), I found it to be most convenient to just make a fork and then do git format-patch to get a set of patches to put into patches/.

[dimpase]

Replying to mkoeppe:

I have no intentions to work on this ticket, but I am curious how people manage these sets of patches? I know Debian has a clear infrastructure and workflow for this, but we don't seem to have any in Sage. Lately, for packages scipoptsuite and gc (#21474), I found it to be most convenient to just make a fork and then do git format-patch to get a set of patches to put into patches/.

yes, that's how I (roughly) do this, too. I just use git diff rather than git format-patch. Is the latter any better?.

[mkoeppe]

It creates the patch file names from the summary lines of the git commits; and it includes a serial number.

[jpflori]

The pexpect issue seems to be:

• www.singular.uni-kl.de:8002/trac/ticket/729

Maybe we should pass the new 'r' value to 'cntrlc' to abort immediately. @Jeroen: any thought about this?

[jpflori]

It seems Singular just decides to die:

> 1+
. ^C
   ? error occurred in or before _ line 47: `1+`^M
Auf Wiedersehen.^M

[jpflori]

Running ./sage -sh -c "Singular --cntrlc=a" seems completely broken for ^C.

[jdemeyer]

Replying to jpflori:

The pexpect issue seems to be:

• www.singular.uni-kl.de:8002/trac/ticket/729

I have not been following the whole discussion (this is comment 506 after all). Which issue are you talking about?

[jpflori]

That on the OS X buildbot the following test fails:

        The following works without restarting Singular::
            sage: a = singular(1)
            sage: _ = singular._expect.sendline('1+')  # unfinished input
            sage: try:
            ....:     alarm(0.5)
            ....:     singular._expect_expr('>')  # interrupt this
            ....: except KeyboardInterrupt:
            ....:     pass
            Control-C pressed.  Interrupting Singular. Please wait a few seconds...
        We can still access a::
            sage: 2*a
            2

Logging singular output into a file, I see that when it gets interrupted, it decides to quit without further explanation (apart from auf wiedersehen).

[jpflori]

And on a machine where the tests passes:

> 1+
. ^C
;
. ;
   ? error occurred in or before STDIN line 48: `;`^M
 error at token `;`^M
   ? abort...^M
> def sage127=2;

[jpflori]

And I cannot run Singular built in debug mode on that OS X machine. It complains about not finding [_]om_Info wich I see in libomalloc.dylib in the S section.

[jpflori]

During linking some Sage files I see -lomalloc -lSingular -lpolys -lfactory -lresources which does not look that right.

[jpflori]

Anyway, that's not the issue. My executable has

bash-3.2$ nm local/bin/Singular |grep _Info
                 U _om_Info

whereas on a debug build on POWER7 it's in the B section.

[jpflori]

Wahtsoever, it seems Singular behaves better if I don't pass the -t (--no-tty) option...

[jdemeyer]

Unfortunately, OS X makes it very hard to debug your own programs when you don't have root access:

dtrace: failed to initialize dtrace: DTrace requires additional privileges

[jdemeyer]

On OS X, when changing the Signal handler to do nothing at all:

void sigint_handler(int sig)
{
  return;
}

the problem still remains.

[jpflori]

The only difference made by the -t option stems from:

latest/Singular/feOpt.cc:      case FE_OPT_NO_TTY:
latest/Singular/feOpt.cc-#if defined(HAVE_FEREAD) || defined(HAVE_READLINE)
latest/Singular/feOpt.cc:        if (feOptSpec[FE_OPT_NO_TTY].value)
latest/Singular/feOpt.cc-          fe_fgets_stdin=fe_fgets;
latest/Singular/feOpt.cc-#endif
latest/Singular/feOpt.cc-        return NULL;

Anyway, I cannot digup why this option is used. Let's try to discard it and live without it at first.

[jpflori]

Oops, it just does not work:

sage: singular(1)
<repr(<sage.interfaces.singular.SingularElement at 0x19cdc0f50>) failed: AttributeError: 'NoneType' object has no attribute 'group'>

[jdemeyer]

OK, some progress: in si_set_signal, there is this part:

  if (sig==SIGINT)
    sigemptyset (&new_action.sa_mask);
  else
    new_action.sa_flags = SA_RESTART;

Unconditionally executing the last line (i.e. removing the else) fixes the problem.

[jpflori]

To finish rambling on my side I get the following on a linux system:

[jpflori@gcc1-power7 sage.git]$ ./sage -sh -c "Singular -t --ticks-per-sec=1000 --cntrlc=a"
                     SINGULAR                                 /  Development
 A Computer Algebra System for Polynomial Computations       /   version 4.0.3
                                                           0<
 by: W. Decker, G.-M. Greuel, G. Pfister, H. Schoenemann     \   Jan 2016
FB Mathematik der Universitaet, D-67653 Kaiserslautern        \
> ^C
Auf Wiedersehen.
[jpflori@gcc1-power7 sage.git]$ ./sage -sh -c "Singular --ticks-per-sec=1000 --cntrlc=a"
                     SINGULAR                                 /  Development
 A Computer Algebra System for Polynomial Computations       /   version 4.0.3
                                                           0<
 by: W. Decker, G.-M. Greuel, G. Pfister, H. Schoenemann     \   Jan 2016
FB Mathematik der Universitaet, D-67653 Kaiserslautern        \
> ^C
;
   ? abort...
   ? error occurred in or before STDIN line 1: `;`
> 

[jdemeyer]

The man page of fgets() reports:

RETURN VALUE

       fgets() returns s on success, and NULL on error or when end of file occurs while no characters have been read.

This is the problem: an error condition (such as an interrupt) and an EOF both give a NULL result for fgets() and Singular confuses these.

[jpflori]

For the sa_flags part, it comes from this upstream commit:

• ​https://github.com/Singular/Sources/commit/72de5f54f47f1606cf29342d26eea5c7ca054155

[jpflori]

Replying to jdemeyer:

The man page of fgets() reports:

RETURN VALUE

       fgets() returns s on success, and NULL on error or when end of file occurs while no characters have been read.

This is the problem: an error condition (such as an interrupt) and an EOF both give a NULL result for fgets() and Singular confuses these.

Great!!! Thanks for finding this.

[jdemeyer]

I saw another problem with the installer: in the light of #20692, patches should be applied from src, not src/latest. Let me first fix that.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​e748300	Apply Singular patches the usual way
​f1a0dcc	Add Singular patch to fix interrupts

[jpflori]

This looks good, this was the last real issue. Let's give it a try on the bots.

There are some small issues left but I think they can be fixed on follow up tickets.

[jpflori]

I've reported the fgets issue upstream.

[jpflori]

What could be improved later:

• debug (with xalloc) build on OS X (om_Info gets undefined (U) in Singular, note that it is ok on linux (B), maybe depends on ocmpiler, linker vesions)
• failing test in debug mode,
• strange linking order (omalloc comes before singular!)
• update to a further version including all patches (everything was reported upstream).

[jdemeyer]

About interrupts: my patch is a big improvement but still not 100% robust. The problem is that fgets() is too high-level and doesn't give enough control. To do this really properly would involve the pselect() system call. I don't know to what extent upstream would accept this, since pselect() is POSIX-only.

[jakobkroeker]

Replying to jdemeyer:

About interrupts: my patch is a big improvement but still not 100% robust. The problem is that fgets() is too high-level and doesn't give enough control. To do this really properly would involve the pselect() system call. I don't know to what extent upstream would accept this, since pselect() is POSIX-only.

Let us use a robust interrupt handling in case pselect() is supported by the target OS and the flaky one only in remaining cases.

=> Open a new ticket or change to needs_work?

[jakobkroeker]

opened a follow-up ticket #21619 for comment:431

[jakobkroeker]

@all authors and reviewers

please create follow-up tickets with known remaining issues (I find that is mandatory for 'positive review')

[jdemeyer]

Replying to jakobkroeker:

=> Open a new ticket or change to needs_work?

New ticket obviously since the interrupt handling on this ticket is not worse than what we currently have in Singular-3-1-7.

[jdemeyer]

Replying to jakobkroeker:

Let us use a robust interrupt handling in case pselect() is supported by the target OS and the flaky one only in remaining cases.

Do you think that Singular upstream would be interested in this?

[jakobkroeker]

Do you think that Singular upstream would be interested in this [robust interrupt handling using pselect()]?

I don't think they care too much about this issue,

but I also think they will accept a patch/pull request if we do all the work.

[jpflori]

• For debug mode: #21624
• For linking order: #21625
• I did not open a follow up ticket for the next upgrade for obvious reasons.

[jakobkroeker]

Replying to jpflori:

Replying to fbissey:

That would be a nice idea but I guess there may still be a need for running autoconf at some point.

Sure there is.

Actually upstream instruction on using xalloc are

README for xalloc: a debug helper and alternative to omalloc

...

}}}

These instructions are buggy :) It is enough to cd to the xalloc dir, then autoreconf and if you want debug mode, then you delete the omalloc dir and mv xalloc omalloc.

To only use patch on top of the upstream 4.0.3p3 what you need to do is to autoreconf everything on your machine on a pristine tree and on a patched one and produce the patch.

Is it possible to change the upstream source in a way that we do not need an autoreconf for xalloc? For example, would it be sufficient to add

AC_CONFIG_SUBDIRS([xalloc])

to the configure.ac at upstream?

Whatever is required to get rid of repackaging/autoreconf, we should do that.

[vbraun]

There is already a different .p3 tarball on the mirrors, so we first have to propagate an updated tarball. And the master fileserver is currently down.

[jakobkroeker]

Replying to jpflori:

• For debug mode: #21624
• For linking order: #21625
• I did not open a follow up ticket for the next upgrade for obvious reasons.

There is now a metaticket #21632 to collect the remaining issues.

[jakobkroeker]

I'm sorry, but since this thread is read by some Singular interface experts, I would like to advertise the tickets #17696, #13999, #17697, #10708, #12529.

Could you have a look?

[jakobkroeker]

now xalloc folder should be autoconfigured upstream, (if I did it right); see ​https://github.com/Singular/Sources/pull/795

[jakobkroeker]

I hacked into the patchbot allowing unsafe tickets and got the following:

> Successfully unpacked
Sha1 of singular-4.0.3p3.tar.bz2 is dd68e84ef7d0e6c35994e0fce0faa8edf38a7bb1
Traceback (most recent call last):
  File "/home/jakobkroeker/.local/lib/python2.7/site-packages/sage_patchbot/patchbot.py", line 1046, in test_a_ticket
    self.check_spkg(spkg)
  File "/home/jakobkroeker/.local/lib/python2.7/site-packages/sage_patchbot/patchbot.py", line 1227, in check_spkg
    given_sha = open(path).read().splitlines()[1].split('=')[1]
IOError: [Errno 2] No such file or directory: 'build/pkgs/singular/checksums.ini'
http://www.lmona.de/files/sage/singular-4.0.3p3.tar.bz2 -- 26 seconds
Apply -- 7751 seconds
http://www.lmona.de/files/sage/singular-4.0.3p3.tar.bz2 -- 26 seconds
2016-10-04 21:51:04
7777 seconds
[2016-10-04 21:51:06] Reporting #17254 with status Spkg
fatal: Unable to read current working directory: Datei oder Verzeichnis nicht gefunden
Traceback (most recent call last):
  File "/home/jakobkroeker/.local/lib/python2.7/site-packages/sage_patchbot/patchbot.py", line 1328, in report_ticket
    tags = [describe_branch('patchbot/base', tag_only=True),
  File "/home/jakobkroeker/.local/lib/python2.7/site-packages/sage_patchbot/util.py", line 235, in describe_branch
    universal_newlines=True)
  File "/usr/lib/python2.7/subprocess.py", line 544, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command '['git', 'describe', '--tags', '--match', '[0-9].[0-9]*', 'patchbot/base']' returned non-zero exit status 128
[2016-10-04 21:51:06] #17254: Spkg
REPORT
{'base': '7.4.beta6',
 'deps': [17635],
 'machine': ['Ubuntu',
             '12.04',
             'x86_64',
             '3.2.0-70-generic',
             'x200t-ThinkPad-X200-Tablet'],
 'owner': 'unknown owner',
 'patchbot_version': '2.6.3',
 'plugins': [],
 'spkgs': [u'http://www.lmona.de/files/sage/singular-4.0.3p3.tar.bz2'],
 'status': 'Spkg',
 'time': '2016-10-04 21:51:06',
 'user': 'jakobkroeker'}
17254: Spkg
ok (report successfully posted)
[2016-10-04 21:51:15] Done reporting #17254
Traceback (most recent call last):
  File "/home/jakobkroeker/.local/lib/python2.7/site-packages/sage_patchbot/patchbot.py", line 1509, in main
    patchbot.test_a_ticket(ticket)
  File "/home/jakobkroeker/.local/lib/python2.7/site-packages/sage_patchbot/patchbot.py", line 1185, in test_a_ticket
    shutil.rmtree(maybe_temp_root)
  File "/usr/lib/python2.7/shutil.py", line 237, in rmtree
    onerror(os.listdir, path, sys.exc_info())
  File "/usr/lib/python2.7/shutil.py", line 235, in rmtree
    names = os.listdir(path)
OSError: [Errno 2] No such file or directory: '/tmp/tmp8iZiiy-sage-git-temp-17254'

[jdemeyer]

Finally! Where is the party?

[fbissey]

Replying to jdemeyer:

Finally! Where is the party?

I could invite you to my place but I gather it is a bit far for everyone else :) I'll be experimenting with 4.0.3p4 and ntl-10 once I have absorbed 7.5.beta0.

[vbraun]

It would be nice if the debug mode (#21624) could be fixed as that is tested on the buildbot...