The sloane_find command is now completely broken

[was]

The sloane_find command, which parses output of a webpage, is now completely broken due to the Sloane sequence webpage being rewritten. It silently always finds nothing.

sage: sloane_find([1,1,2,3,5,8,13,21], nresults=1)
[]

[was]

Crap, this is a dup of #10056

[tmonteil]

Hi,

the aim of #10056 was to migrate every url in sage that was dealing with Sloane's database at once (not only sloane_find()), hence this is not really a duplicate. The issue here is that oeis changed their html templates, so the sage parsers are not working anymore. I just coded a new parser, which you can see working on the combinat repository (sloane_new_website_parsers-tm.patch).

I added the more atomic functions in sage/databases/sloane.py (the documentation still needs to be fixed):

• sloane_discover() which discovers which sequences correspond to some list of integers.
• sloane_first_terms() which returns the first terms of the sequence.
• sloane_description() which returns the one line description of the sequence.

I don't really know what to do with the broken functions:

• parse_sequence() is completely dead and now useless. Should it now raise a deprecation warning ?
• sloane_sequence() and sloane_find() are broken, but maybe some users still need them, should i make them working again as a combination of the first 3 functions ?

Also, maybe all those functions should be renamed by replacing sloane by oeis ? This may be too early since i guess most people know more about Sloane than about oeis. In such a case, what is the procedure/convention to ensure backward compatibility ?

At the end, we could have a daily crontab to check when the oeis changes its html template, and adapt the parsers accordingly. Better, we could ask them to provide an online standard query system, but if i understand a discussion in the nmbrthry mailing-list, this may not be their whish.

Ciao, Thierry

[nthiery]

Salut Thierry,

Rather than having a bunch of functions to return the various pieces of a Sloane result, there could be a single object from which one could query the pieces via methods. Something like:

    sage: s = sloane_find([1,1,2,5,8,14])[0]
    sage: s.description()
    ...
    sage: s[6]
    42

Actually, Anders Claesson [1] had implemented a prototype of this feature during FPSAC'09, and his demo was quite cool. I guess he would agree to give his code away for someone else to finalize it and merge it into Sage.

But maybe that should be the topic of a separate ticket.

[1]  http://combinatorics.is/anders/

[tmonteil]

By the way, if you need to play with OEIS now, here is the current version of the patch in preparation, on sage-combinat:  http://combinat.sagemath.org/patches/file/0506836c7c66/sloane_new_website_parsers-tm.patch

The methods still parse the new website correctly, but i didn't put everything in a single class yet as Nicolas suggested.

[kini]

Oh, no particular hurry. I just happened to notice OEIS was broken, and was surprised to find that this ticket was closed, so I asked leif to reopen it. Glad to hear there is a patch in the works. If possible could you post it here once in a while? As the ticket hadn't been changed for 9 months I thought nobody was working on it. Thanks!

[slabbe]

Thierry, can you upload your patch here? Was it ready?

Sébastien

[kcrisman]

See also #13884, though it should be dealt with here.

And FYI, we would want to make sure that this also works:

sage -t --only_optional=internet "devel/sage/sage/combinat/words/paths.py"
    sage: sloane_find(_, nresults=1) #optional - internet
Expected:
    Searching Sloane's online database...
    [[1653,
      'Numbers n such that 2*n^2 - 1 is a square.',
      [1,
       5,
       29,
       169,
       985,
       5741,
       33461,
       195025,
       1136689,
       6625109,
       38613965,
       225058681,
       1311738121,
       7645370045,
       44560482149,
       259717522849,
       1513744654945,
       8822750406821,
       51422757785981,
       299713796309065,
       1746860020068409]]]
Got:
    Searching Sloane's online database...
    []
**********************************************************************

Which of course it should once this is done, but wanted to point it out since it's in a different file.

[ncohen]

Thierry, can you upload your patch here? Was it ready?

6 months old question left unanswered.. Time for a ping :-P

Nathann

[kini]

 http://combinat.sagemath.org/patches/log/tip/sloane_new_website_parsers-tm.patch

Nothing happened for two years, and the patch is still sitting in the combinat queue without being posted on trac.

[ncohen]

"The Sage-Combinat experience" :-P

So, should we set this ticket to need_work, or needs_info, or wait_for_the_combinat_queue_to_be_reviewed ? :-P

Nathann

[tmonteil]

Hi,

sorry for the delay, the version on the combinat queue is not the good one, during the Cernay combinat days in last february (less than one year!), i build a new class named oeis, that allows to get all kind of information from the oeis db (cross references, web links, ...).

I upload a temporary version here so that you can try it, but the documentation is not finished, and non backward-compatible changes may still appear.

Thierry