New interface with OEIS #10358

williamstein · 2010-12-01T03:02:31Z

The sloane_find command, which parses output of a webpage, is broken for years due to the Sloane sequence webpage being rewritten. It silently always finds nothing.

sage: sloane_find([1,1,2,3,5,8,13,21], nresults=1)
[]

The aim of this ticket is to propose a new interface with OEIS http://oeis.org/

Apply:

Depends on #15245

CC: @sagetrac-sage-combinat @williamstein @kini @jpflori @sagetrac-chrisjamesberg @VivianePons

Component: combinatorics

Keywords: Cernay2012 days49

Author: Thierry Monteil

Reviewer: Nathann Cohen

Merged: sage-5.13.beta2

Issue created by migration from https://trac.sagemath.org/ticket/10358

The text was updated successfully, but these errors were encountered:

williamstein · 2010-12-01T03:04:44Z

comment:2

Crap, this is a dup of #10056

sagetrac-tmonteil · 2011-01-25T13:43:55Z

comment:3

Hi,

the aim of #10056 was to migrate every url in sage that was dealing with Sloane's database at once (not only sloane_find()), hence this is not really a duplicate. The issue here is that oeis changed their html templates, so the sage parsers are not working anymore. I just coded a new parser, which you can see working on the combinat repository (sloane_new_website_parsers-tm.patch).

I added the more atomic functions in sage/databases/sloane.py (the documentation still needs to be fixed):

sloane_discover() which discovers which sequences correspond to some list of integers.
sloane_first_terms() which returns the first terms of the sequence.
sloane_description() which returns the one line description of the sequence.

I don't really know what to do with the broken functions:

parse_sequence() is completely dead and now useless. Should it now raise a deprecation warning ?
sloane_sequence() and sloane_find() are broken, but maybe some users still need them, should i make them working again as a combination of the first 3 functions ?

Also, maybe all those functions should be renamed by replacing sloane by oeis ? This may be too early since i guess most people know more about Sloane than about oeis. In such a case, what is the procedure/convention to ensure backward compatibility ?

At the end, we could have a daily crontab to check when the oeis changes its html template, and adapt the parsers accordingly. Better, we could ask them to provide an online standard query system, but if i understand a discussion in the nmbrthry mailing-list, this may not be their whish.

Ciao,
Thierry

nthiery · 2011-01-26T10:24:20Z

comment:6

Salut Thierry,

Rather than having a bunch of functions to return the various pieces of a Sloane result, there could be a single object from which one could query the pieces via methods. Something like:

    sage: s = sloane_find([1,1,2,5,8,14])[0]
    sage: s.description()
    ...
    sage: s[6]
    42

Actually, Anders Claesson [1] had implemented a prototype of this feature during FPSAC'09, and his demo was quite cool. I guess he would agree to give his code away for someone else to finalize it and merge it into Sage.

But maybe that should be the topic of a separate ticket.

[1] http://combinatorics.is/anders/

sagetrac-tmonteil · 2011-10-23T15:49:14Z

comment:10

By the way, if you need to play with OEIS now, here is the current version of the patch in preparation, on sage-combinat: http://combinat.sagemath.org/patches/file/0506836c7c66/sloane_new_website_parsers-tm.patch

The methods still parse the new website correctly, but i didn't put everything in a single class yet as Nicolas suggested.

kini · 2011-10-24T07:07:42Z

comment:11

Oh, no particular hurry. I just happened to notice OEIS was broken, and was surprised to find that this ticket was closed, so I asked leif to reopen it. Glad to hear there is a patch in the works. If possible could you post it here once in a while? As the ticket hadn't been changed for 9 months I thought nobody was working on it. Thanks!

sagetrac-tmonteil · 2012-02-10T02:54:40Z

Changed keywords from none to Cernay2012

seblabbe · 2012-08-01T12:57:29Z

comment:13

Thierry, can you upload your patch here? Was it ready?

Sébastien

kcrisman · 2013-01-03T18:05:34Z

comment:14

See also #13884, though it should be dealt with here.

And FYI, we would want to make sure that this also works:

sage -t --only_optional=internet "devel/sage/sage/combinat/words/paths.py"
    sage: sloane_find(_, nresults=1) #optional - internet
Expected:
    Searching Sloane's online database...
    [[1653,
      'Numbers n such that 2*n^2 - 1 is a square.',
      [1,
       5,
       29,
       169,
       985,
       5741,
       33461,
       195025,
       1136689,
       6625109,
       38613965,
       225058681,
       1311738121,
       7645370045,
       44560482149,
       259717522849,
       1513744654945,
       8822750406821,
       51422757785981,
       299713796309065,
       1746860020068409]]]
Got:
    Searching Sloane's online database...
    []
**********************************************************************

Which of course it should once this is done, but wanted to point it out since it's in a different file.

nathanncohen · 2013-01-20T10:07:35Z

comment:15

Thierry, can you upload your patch here? Was it ready?

6 months old question left unanswered.. Time for a ping :-P

Nathann

kini · 2013-01-20T10:12:26Z

comment:16

http://combinat.sagemath.org/patches/log/tip/sloane_new_website_parsers-tm.patch

Nothing happened for two years, and the patch is still sitting in the combinat queue without being posted on trac.

nathanncohen · 2013-01-20T10:15:05Z

comment:17

"The Sage-Combinat experience" :-P

So, should we set this ticket to need_work, or needs_info, or wait_for_the_combinat_queue_to_be_reviewed ? :-P

Nathann

sagetrac-tmonteil · 2013-01-21T11:32:59Z

comment:18

Hi,

sorry for the delay, the version on the combinat queue is not the good one, during the Cernay combinat days in last february (less than one year!), i build a new class named oeis, that allows to get all kind of information from the oeis db (cross references, web links, ...).

I upload a temporary version here so that you can try it, but the documentation is not finished, and non backward-compatible changes may still appear.

Thierry

fchapoton · 2013-10-15T18:07:46Z

comment:48

Patchbots are sometimes dumb. Let us try again to teach them !

apply trac_10358-oeis-tm.patch,trac_10358-oeis-review_1-tm.patch,trac_10358-oeis-review_2-tm.patch,trac_10358-oeis-review_3-nc-tm.patch

fchapoton · 2013-10-15T18:13:26Z

comment:49

Almost good ... Let me try again

apply trac_10358-oeis-tm.patch,trac_10358-oeis-review_1-tm.patch,trac_10358-oeis-review_2-tm.patch,trac_10358-oeis-review_3-nc-tm.patch

fchapoton · 2013-10-15T18:22:20Z

comment:50

pfff. Last try

apply only trac_10358-oeis-tm.patch,trac_10358-oeis-review_1-tm.patch,trac_10358-oeis-review_2-tm.patch,trac_10358-oeis-review_3-nc-tm.patch

fchapoton · 2013-10-15T18:55:30Z

comment:51

ok, it was failing because the patchbots do not like invisible characters (and neither do I). This should work:

apply trac_10358-oeis-tm.patch trac_10358-oeis-review_1-tm.patch trac_10358-oeis-review_2-tm.patch trac_10358-oeis-review_3-nc-tm.patch

sagetrac-tmonteil · 2013-10-19T12:49:13Z

Attachment: trac_10358-oeis-review_3-nc-tm.patch.gz

Tested on 5.12

jdemeyer · 2013-10-20T21:29:26Z

Dependencies: #15245

jdemeyer · 2013-10-20T21:29:26Z

comment:53

This needs to be rebased to #15245.

sagetrac-tmonteil · 2013-10-20T22:54:43Z

rebased after #15245

sagetrac-tmonteil · 2013-10-20T23:07:34Z

comment:54

Attachment: trac_10358-oeis-tm_rebase.patch.gz

The rebase is a shift in sage/matrix/matrix2.pyx

apply trac_10358-oeis-tm_rebase.patch trac_10358-oeis-review_1-tm.patch trac_10358-oeis-review_2-tm.patch trac_10358-oeis-review_3-nc-tm.patch

jdemeyer · 2013-10-21T11:58:19Z

comment:55

The patches apply now.

jdemeyer · 2013-10-31T19:15:49Z

Merged: sage-5.13.beta2

nthiery · 2013-10-31T22:52:39Z

comment:57

Joy, joy, joy is all around us!

Hail to Thierry, Hail to Nathann!

zimmermann6 · 2018-09-19T14:35:01Z

comment:58

I use this closed ticket to touch the interested people. In March 2017 I got:

  sage: oeis([1,1,2,5,14])
  0: A000108: Catalan numbers: C(n) = binomial(2n,n)/(n+1) = (2n)!/(n!(n+1)!). \
Also called Segner numbers.
  1: A120588: G.f. satisfies: 3*A(x) = 2 + x + A(x)^2, with a(0) = 1.
  2: A080937: Number of Catalan paths (nonnegative, starting and ending at 0, s\
tep +/-1) of 2*n steps with all values <= 5.

and now I get with Sage 8.3:

sage: oeis([1,1,2,5,14])

0: A000108: Catalan numbers: C(n) = binomial(2n,n)/(n+1) = (2n)!/(n!(n+1)!). Also called Segner numbers.
1: A120588: G.f. satisfies: 3*A(x) = 2 + x + A(x)^2, with a(0) = 1.
2: A000679: Number of groups of order 2^n.

I have two questions:

why did the output change?
why do we get only 3 answers (I checked with len(oeis(...)) that there are only 3 answers, and it is not only 3 answers that are printed), whereas the OEIS official site gives 209 answers?

sagetrac-tmonteil · 2018-09-19T14:54:16Z

comment:59

Replying to @zimmermann6:

I use this closed ticket to touch the interested people. In March 2017 I got:

  sage: oeis([1,1,2,5,14])
  0: A000108: Catalan numbers: C(n) = binomial(2n,n)/(n+1) = (2n)!/(n!(n+1)!). \
Also called Segner numbers.
  1: A120588: G.f. satisfies: 3*A(x) = 2 + x + A(x)^2, with a(0) = 1.
  2: A080937: Number of Catalan paths (nonnegative, starting and ending at 0, s\
tep +/-1) of 2*n steps with all values <= 5.

and now I get with Sage 8.3:

sage: oeis([1,1,2,5,14])

0: A000108: Catalan numbers: C(n) = binomial(2n,n)/(n+1) = (2n)!/(n!(n+1)!). Also called Segner numbers.
1: A120588: G.f. satisfies: 3*A(x) = 2 + x + A(x)^2, with a(0) = 1.
2: A000679: Number of groups of order 2^n.

I have two questions:

why did the output change?

The oeis module connects to an on-line database, which is subject to changes. The output depends on the ordering provided by OEIS, which depends on various criteria (i do not know the details, but if the sequence has keywords interesting or important it will appear earlier).

why do we get only 3 answers (I checked with len(oeis(...)) that there are only 3 answers, and it is not only 3 answers that are printed), whereas the OEIS official site gives 209 answers?

You can get help about oeis as follows:

sage: oeis?

You can see that there is a max_results option, which is set by default to 3 to avoid the outout being flooded. Maybe this could be extended to, say, 10.

Also, OEIS will only send the 100 first results even if you do:

sage: oeis([1,1,2,5,14], max_results=2000)

This is why there is a first_result option, to be able to access the next ones.

zimmermann6 · 2018-09-20T09:00:16Z

comment:60

Thierry, thank you for your answer.

Of course I had tried oeis?, and in Sage 8.3 I get:

   * "max_results" - (integer, default: 30) the maximum number of
     results to return, they are sorted according to their relevance.
     In any cases, the OEIS website will never provide more than 100
     results.

thus the documentation seems to differ from the code. Should I open a new ticket for that?

williamstein added this to the sage-4.6.1 milestone Dec 1, 2010

williamstein added c: combinatorics labels Dec 1, 2010

williamstein assigned sagetrac-sage-combinat Dec 1, 2010

williamstein added the t: bug label Dec 1, 2010

williamstein added r: duplicate and removed p: critical / 2 labels Dec 1, 2010

williamstein closed this as completed Dec 1, 2010

sagetrac-tmonteil mannequin assigned sagetrac-tmonteil and unassigned sagetrac-sage-combinat Jan 25, 2011

sagetrac-tmonteil mannequin added the s: needs work label Jan 25, 2011

sagetrac-tmonteil mannequin reopened this Jan 25, 2011

sagetrac-tmonteil mannequin modified the milestones: sage-4.6.1, sage-4.6.2 Jan 25, 2011

nexttime mannequin removed r: duplicate labels Oct 23, 2011

nexttime mannequin closed this as completed Oct 23, 2011

nexttime mannequin reopened this Oct 23, 2011

sagetrac-tmonteil mannequin added s: positive review and removed s: needs review labels Oct 19, 2013

jdemeyer added s: needs work and removed s: positive review labels Oct 20, 2013

This comment has been minimized.

Sign in to view

sagetrac-tmonteil mannequin added s: needs review and removed s: needs work labels Oct 20, 2013

jdemeyer added s: positive review and removed s: needs review labels Oct 21, 2013

jdemeyer removed the s: positive review label Oct 31, 2013

jdemeyer closed this as completed Oct 31, 2013

nathanncohen mannequin mentioned this issue Jun 27, 2013

optional doctest failure -- problem in species code (easy to fix) #6323

Closed

robertwb mentioned this issue Sep 6, 2015

Fix optional internet tests. #13884

Closed

nilesjohnson mannequin mentioned this issue Jul 29, 2014

OEIS database install docstring is out of date #16599

Closed

fchapoton mentioned this issue Oct 28, 2018

remove deprecated stuff in sloane.py #26459

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New interface with OEIS #10358

New interface with OEIS #10358

williamstein commented Dec 1, 2010

williamstein commented Dec 1, 2010

sagetrac-tmonteil mannequin commented Jan 25, 2011

nthiery commented Jan 26, 2011

sagetrac-tmonteil mannequin commented Oct 23, 2011

kini commented Oct 24, 2011

sagetrac-tmonteil mannequin commented Feb 10, 2012

seblabbe commented Aug 1, 2012

kcrisman commented Jan 3, 2013

nathanncohen mannequin commented Jan 20, 2013

kini commented Jan 20, 2013

nathanncohen mannequin commented Jan 20, 2013

sagetrac-tmonteil mannequin commented Jan 21, 2013

fchapoton commented Oct 15, 2013

fchapoton commented Oct 15, 2013

fchapoton commented Oct 15, 2013

fchapoton commented Oct 15, 2013

sagetrac-tmonteil mannequin commented Oct 19, 2013

jdemeyer commented Oct 20, 2013

jdemeyer commented Oct 20, 2013

sagetrac-tmonteil mannequin commented Oct 20, 2013

This comment has been minimized.

sagetrac-tmonteil mannequin commented Oct 20, 2013

jdemeyer commented Oct 21, 2013

jdemeyer commented Oct 31, 2013

nthiery commented Oct 31, 2013

zimmermann6 commented Sep 19, 2018

sagetrac-tmonteil mannequin commented Sep 19, 2018

zimmermann6 commented Sep 20, 2018

New interface with OEIS #10358

New interface with OEIS #10358

Comments

williamstein commented Dec 1, 2010

williamstein commented Dec 1, 2010

sagetrac-tmonteil mannequin commented Jan 25, 2011

nthiery commented Jan 26, 2011

sagetrac-tmonteil mannequin commented Oct 23, 2011

kini commented Oct 24, 2011

sagetrac-tmonteil mannequin commented Feb 10, 2012

seblabbe commented Aug 1, 2012

kcrisman commented Jan 3, 2013

nathanncohen mannequin commented Jan 20, 2013

kini commented Jan 20, 2013

nathanncohen mannequin commented Jan 20, 2013

sagetrac-tmonteil mannequin commented Jan 21, 2013

fchapoton commented Oct 15, 2013

fchapoton commented Oct 15, 2013

fchapoton commented Oct 15, 2013

fchapoton commented Oct 15, 2013

sagetrac-tmonteil mannequin commented Oct 19, 2013

jdemeyer commented Oct 20, 2013

jdemeyer commented Oct 20, 2013

sagetrac-tmonteil mannequin commented Oct 20, 2013

This comment has been minimized.

sagetrac-tmonteil mannequin commented Oct 20, 2013

jdemeyer commented Oct 21, 2013

jdemeyer commented Oct 31, 2013

nthiery commented Oct 31, 2013

zimmermann6 commented Sep 19, 2018

sagetrac-tmonteil mannequin commented Sep 19, 2018

zimmermann6 commented Sep 20, 2018