Skip to content

API 0.4.0

Jump to bottom Edit New page
g4jc edited this page Jul 10, 2014 · 1 revision

Tentative API for http://redmine.seeks-project.info/issues/370

API is broken into plugin dependent fragments.

All strings must be UTF-8 encoded.


Contents

websearch

plugin: websearch

resources: /info search/txt /words /cache/txt /similar/txt /cache/txt /cluster/types /cluster/auto /recent/queries

HTTP methods: GET, PUT, POST, DELETE

Unless stated otherwise, the following parameters apply to all calls:

  • page
  • engines
  • prs
  • lang


get plugin info

GET /info

  • example command

curl seeks.fr/info | python -m json.tool

  • example output 
{
    "version": "0.4",
    "code-status": "experimental",
    "url-source-code": "http: //seeks.git.sourceforge.net/git/gitweb.cgi?p=seeks/seeks;a=tree",
    "thumbs": "off",
    "content-analysis": "off",
    "clustering": "on",
    "txt-parsers": {
        "google": {
            "google": {
                "default": "true"
            }
        },
        "bing": {
            "bing": {
                "default": "true"
            }
        },
        "mediawiki": {
            "doc_mwiki": {
                "default": "false"
            },
            "wikipedia": {
                "default": "false"
            }
        },
        "dailymotion": {
            "dailymotion": {
                "default": "true"
            }
        },
        "twitter": {
            "twitter": {
                "default": "true"
            },
            "identica": {
                "default": "true"
            }
        }
    },
}
issue a query and get search results

GET /search/txt/{query}

PUT /search/txt/{query}

You need to append "?output=json" to your query.

Difference between the GET & PUT calls: GET only retrieves results.

PUT does store the query on the Seeks node. Stored queries can be later reused by the system, typically when proposing related queries.

You need to specify

  • parameters
Name Required Description
query Yes url-encoded query string
engines No selective filter of search engines, values are those specified for every feed in configuration, of the form parser:feed_name, see engines configuration
rpp No Number of requested results per page. If unspecified, the number defined by the search node is used instead.
page No Specifies the number of the requested page. Depends on rpp parameter.
prs No Applies a personalized ranking to the results. Takes value on or off. Default is on.
lang No Specifies the language of the search.
thumbs No Specifies whether a URI to a thumbails of every returned result is specified. Values are on or off.
expansion No Specifies the requested depth of the search, the higher, the more results are fetched and processed.
peers No Only effective when engine seeks is enabled (default). Allowed values, local for results by the called peer only, ring for results from the configured default peer ring of the called node. ring is default.
order No Specifies the sorting of results. Default is rank. Other values are new-date newest to old content, old-date old to newest content, new-activity newest to old recommended results, and old-activity old to newest recommended results.
swords No yes or no, specifies whether to filter similar queries with stopwords. Default is defined by the remote node.
  • example command

curl "seeks.fr/search/txt/seeks?output=json" | python -m json.tool

  • example output 
{"query": "seeks",
 "lang": "en",
 "pers": "on",
 "expansion": "1",
 "npeers": "4",
  "engines": [
   "bing",
   "blekko",
   "google",
   "seeks",
   "yahoo"],
 "date": "Wed, 27 Jul 2011 10:34:57 GMT",
 "qtime": 0.08,
 "snippets": [
   {
       "id": 1289851243,
       "title": "Seeks Project",
       "url": "<a href="/web/20130930043815/http://www.seeks-project.info/" class="external free" title="http://www.seeks-project.info/" rel="nofollow">http://www.seeks-project.info/</a>",
       "summary": "Seeks is a free and open P2P design and application for enabling social websearch. Its specific purpose is to regroup users whose queries are similar so they can share both the query results and their experience on these results. On th..."
       "seeks_meta": 4,
       "seeks_score": 0.000710273,
       "rank": 0.666667,
       "cite": "<a href="/web/20130930043815/http://www.seeks-project.info/" class="external free" title="http://www.seeks-project.info/" rel="nofollow">http://www.seeks-project.info/</a>",
       "engines": [
           "blekko",
           "google",
           "seeks"
       ],
       "words": [
           "results",
           "experience"
       ],
       "type": "webpage",
       "personalized": "yes"
       "snpeers": "3"
       "hits": "769"
   },
  ...]}
  • error values 
400 {"error":"bad parameter"}
408 {"error":"request timeout"}
500 {"error": "internal" }
retrieve a given result snippet

GET /search/txt/{query}/{snippet_id}

You need to append "?output=json" to your query.

  • parameters
Name Required Description
query Yes url-encoded query string
snippet_id Yes snippet 32bit integer id
lang No Specifies the language of the query.
  • example output 
  {
       "id": 2098355214,
       "title": "Seeks Project | An Open Decentralized Platform for ...",
       "url": "<a href="/web/20130930043815/http://www.seeks-project.info/site/" class="external free" title="http://www.seeks-project.info/site/" rel="nofollow">http://www.seeks-project.info/site/</a>",
       "summary": "Seeks is Free Software, and distributed under the AGPLv3 license...",
       "seeks_meta": 4,
       "seeks_score": 0.0005,
       "rank": 0.33,
       "cite": "<a href="/web/20130930043815/http://www.seeks-project.info/site" class="external free" title="http://www.seeks-project.info/site" rel="nofollow">http://www.seeks-project.info/site</a>",
       cached: "<a href="/web/20130930043815/http://cached.example.com/seeks-project.info" class="external free" title="http://cached.example.com/seeks-project.info" rel="nofollow">http://cached.example.com/seeks-project.info</a>",
       "engines": [
           "bing",
           "seeks",
           "yahoo"
       ],
       "words": [
           "software"
           "free"
           "distributed"
       ],
       "type": "webpage",
       "personalized": "yes",
       "snpeers": "3",
       "hits": "112"
   }
  • error values 
400 { "error":"bad parameter"}
404 { "error":"not found" }
select a result snippet: URL selection capture + optional redirect to the URL

POST /search/txt/{query}/{snippet_id}

  • parameters
Name Required Description
query Yes url-encoded query string
snippet_id Yes snippet 32bit integer id
lang No Specifies the language of the query.
redirect No redirects to the selected URL (code 302)
cpost No seeks node url to which to crosspost the snippet selection (URL-encode).
  • error values 
403 {"error":"forbidden"}
404 {"error":"not found"}

403 error occurs when trying to post a new result snippet with this call.

This call is reserved to the selection of results obtained in response to a query.

For posting a new result snippet, see 2.3

reject a result snippet

DELETE /search/txt/{query}/{snippet_id}

  • parameters
Name Required Description
query Yes url-encoded query string
snippet_id Yes snippet 32bit integer id
lang No Specifies the language of the query.
  • error output 
400 {"error":"bad parameter"}
404 {"error":"not found"}
get a list of most discriminant words for a result snippet

GET /words/{query}/{snippet_id}

  • parameters
Name Required Description
query Yes url-encoded query string
snippet_id Yes snippet 32bit integer id
lang No Specifies the language of the query.
  • example output 
{words: [
 "software",
 "free",
 "distributed"
]}
  • error values 
400 {"error":"bad parameter"}
404 {"error":"not found"}
get all discriminant words for a query

GET /words/{query}

  • parameters
Name Required Description
query Yes url-encoded query string
lang No Specifies the language of the query.
  • example output 
{"words": [
 "software",
 "free",
 "distributed"
]}
  • error values 
400 {"error":"bad parameter"}
get recently issued queries

GET /recent/queries

  • parameters
Name Required Description
nq No Specifies the maximum number of requested recent queries.
  • example output 
{"queries": [
  "seeks",
  "debian laptop"
 ]}
  • error values

none

cluster results by type

GET /cluster/types/{query}

  • parameters
Name Required Description
query Yes url-encoded query string
lang No Specifies the language of the query.
  • example output
  • error values 
400 {"error":"bad parameter"}
cluster results automatically

GET /cluster/auto/{query}

  • parameters
Name Required Description
query Yes url-encoded query string
nclusters No maximum number of requested clusters
lang No Specifies the language of the query.
  • example output 
{"query":"haskell",
 "lang":"en",
 "pers":"on",
 "expansion":"1",
 "npeers":"3",
 "engines":["seeks"],
 "date":"Thu, 04 Aug 2011 11:54:06 GMT",
 "qtime":2.21,
 "clusters":[{"label":"tutorials history",
              "snippets":[{"id":4273810721,
                           "title":"Haskell - HaskellWiki",
                           "url":"<a href="/web/20130930043815/http://haskell.org/" class="external free" title="http://haskell.org" rel="nofollow">http://haskell.org</a>",
                           "summary":"30 Nov 2010 – Repository for information on the lazy functional programming language Haskell",
                           "seeks_meta":1,
                           "seeks_score":0.000240117,
                           "rank":0,
                           "cite":"<a href="/web/20130930043815/http://haskell.org/" class="external free" title="http://haskell.org" rel="nofollow">http://haskell.org</a>",
                           "engines":["seeks"],
                           "words":[history,tutorials],
                           "type":"webpage",
                           "personalized":"yes",
                           "snpeers":"1",
                           "hits":"1"}]},
             {"label":"implementations history",
              "snippets":[{"id":1329631803,
                           "title":"Haskell (programming language) - Wikipedia","url":"<a href="/web/20130930043815/http://en.wikipedia.org/wiki/haskell_(programming_language)" class="external free" title="http://en.wikipedia.org/wiki/haskell_(programming_language)" rel="nofollow">http://en.wikipedia.org/wiki/haskell_(programming_language)</a>",
                           "summary":"Haskell 3] is a standardized, general-purpose purely functional programming language",
                           "seeks_meta":1,
                           "seeks_score":0.000141817,
                           "rank":0,
                           "cite":"<a href="/web/20130930043815/http://en.wikipedia.org/wiki/haskell_(programming_language)" class="external free" title="http://en.wikipedia.org/wiki/haskell_(programming_language)" rel="nofollow">http://en.wikipedia.org/wiki/haskell_(programming_language)</a>",
                           "engines":["seeks"],
                           "words":[history,implementations],
                           "type":"webpage",
                           "personalized":"yes"}]}]}
  • error values 
400 {"error":"bad parameter"}
rank results by similarity to a given snippet

GET /similar/txt/{query}/{snippet_id}

  • parameters
Name Required Description
query Yes url-encoded query string
snippet_id Yes snippet 32bit integer id
lang No Specifies the language of the query.
  • example output

Same as regular search but with snippets ranked in decreasing order of similarity to the snippet with the specified id.

  • error values 
400 {"error":"bad parameter"}
404 {"error":"not found"}
get a link to cached snippet content if one is known

GET /cache/txt/{query}

  • parameters
Name Required Description
query Yes url-encoded query string
url Yes URL
lang No Specifies the language of the query.
  • example output

Cached page as returned content.

  • error values 
400 {"error":"bad parameter"}
404 {"error":"not found"}

image websearch

plugin: img_websearch

resources: search/img /similar/img

HTTP methods: GET, PUT

Unless stated otherwise, the following parameters apply to all calls:

  • page
  • engines
  • prs
  • lang
issue a query and get search results

GET /search/img/{query}

PUT /search/img/{query}

Difference between the GET & PUT calls: GET only retrieves results.

PUT does store the query on the Seeks node. Stored queries can be later reused by the system, typically when proposing related queries.

  • parameters
Name Required Description
query Yes url-encoded query string
engines No selective filter of search engines, values are those specified for every feed in configuration, of the form parser:feed_name, see engines configuration
rpp No Number of requested results per page. If unspecified, the number defined by the search node is used instead.
page No Specifies the number of the requested page. Depends on rpp parameter.
prs No Applies a personalized ranking to the results. Takes value on or off. Default is on.
lang No Specifies the language of the search.
thumbs No Specifies whether a URI to a thumbails of every returned result is specified. Values are on or off.
expansion No Specifies the requested depth of the search, the higher, the more results are fetched and processed.
peers No Only effective when engine seeks is enabled (default). Allowed values, local for results by the called peer only, p2p for results from the configured default peer ring of the called node. p2p is default.
order No Specifies the sorting of results. Default is rank. Other values are new-date newest to old content, old-date old to newest content, new-activity newest to old recommended results, and old-activity old to newest recommended results.
  • example output 
{"query": "seeks",
 "lang": "en",
 "pers": "on",
 "expansion": "1",
 "npeers": "4",
  "engines": [
   "bing",
   "blekko",
   "google",
   "seeks",
   "yahoo"],
 "date": "Wed, 27 Jul 2011 10:34:57 GMT",
 "qtime": 0.08,
  "snippets": [
    {
       "id":1715241010,
       "title":"Seeks.svg",
       "url":"<a href="/web/20130930043815/http://commons.wikipedia.org/wiki/File:Seeks.svg" class="external free" title="http://commons.wikipedia.org/wiki/File:Seeks.svg" rel="nofollow">http://commons.wikipedia.org/wiki/File:Seeks.svg</a>",
       "summary":"",
       "seeks_meta":0,
       "seeks_score":1,
       "rank":0,
       "cite":"<a href="/web/20130930043815/http://commons.wikipedia.org/wiki/File:Seeks.svg" class="external free" title="http://commons.wikipedia.org/wiki/File:Seeks.svg" rel="nofollow">http://commons.wikipedia.org/wiki/File:Seeks.svg</a>",
       "cached":"//upload.wikimedia.org/wikipedia/commons/thumb/2/2c/Seeks.svg/120px-Seeks.svg.png",
       "engines":[],
       "type":"image",
       "personalized":"no"
     },
  ...]}
  • error values 
400 {"error":"bad parameter"}
408 {"error":"request timeout"}
500 {"error": "internal" }
retrieve a given result snippet

GET /search/img/{query}/{snippet_id}

  • parameters
Name Required Description
query Yes url-encoded query string
snippet_id Yes snippet 32bit integer id
lang No Specifies the language of the query.
  • example output 
 "snippets": [
    {
       "id":1715241010,
       "title":"Seeks.svg",
       "url":"<a href="/web/20130930043815/http://commons.wikipedia.org/wiki/File:Seeks.svg" class="external free" title="http://commons.wikipedia.org/wiki/File:Seeks.svg" rel="nofollow">http://commons.wikipedia.org/wiki/File:Seeks.svg</a>",
       "summary":"",
       "seeks_meta":0,
       "seeks_score":1,
       "rank":0,
       "cite":"<a href="/web/20130930043815/http://commons.wikipedia.org/wiki/File:Seeks.svg" class="external free" title="http://commons.wikipedia.org/wiki/File:Seeks.svg" rel="nofollow">http://commons.wikipedia.org/wiki/File:Seeks.svg</a>",
       "cached":"//upload.wikimedia.org/wikipedia/commons/thumb/2/2c/Seeks.svg/120px-Seeks.svg.png",
       "engines":[],
       "type":"image",
       "personalized":"no"
     }
  • error values 
400 { "error":"bad parameter"}
404 { "error":"not found" }
select a result snippet: URL selection capture + optional redirect to the URL

POST /search/img/{query}/{snippet_id}

  • parameters
Name Required Description
query Yes url-encoded query string
snippet_id Yes snippet 32bit integer id
lang No Specifies the language of the query.
redirect No redirects to the selected URL (code 302)
cpost No seeks node url to which to crosspost the snippet selection (URL-encode).
  • error values 
403 {"error":"forbidden"}
404 {"error":"not found"}

403 error occurs when trying to post a new result snippet with this call.

This call is reserved to the selection of results obtained in response to a query.

For posting a new result snippet, see 2.3

reject a result snippet

DELETE /search/img/{query}/{snippet_id}

  • parameters
Name Required Description
query Yes url-encoded query string
snippet_id Yes snippet 32bit integer id
lang No Specifies the language of the query.
  • error output 
400 {"error":"bad parameter"}
404 {"error":"not found"}

P2P collaborative filtering

plugin: cf

resources: /suggestion /recommendation

HTTP methods: GET, POST


get a set of query suggestions

GET /suggestion/{query}

  • parameters
Name Required Description
query Yes url-encoded query string
nsugg No number of requested suggestions
radius No radius of the similarity halo for query search. If not specified, left for the called node to decide.
peers No Allowed values, local for suggestions by the called peer only, ring for suggestions from the configured default peer ring of the called node. ring is default.
swords No yes or no, specifies whether to filter similar queries with stopwords. Default is defined by the remote node.
  • example output 
"suggestions":
[
 "seeks-project",
 "project seeks",
 "seeks p2p"
]
  • error values 
400 {"error":"bad parameter"}
get a set of results recommendations from the P2P collaborative filter

GET /recommendation/{query}

  • parameters
Name Required Description
query Yes url-encoded query string
nreco No number of requested result recommendations
radius No radius of the similarity halo for query search. If not specified, left for the called node to decide.
peers No Allowed values, local for recommendations by the called node only, ring for recommendations from the configured default peer ring of the called node.
lang No Specifies the language of the query. Recommended for better results.
order No Specifies the sorting of results. Default is rank. Other values are new-date newest to old content, old-date old to newest content, new-activity newest to old recommended results, and old-activity old to newest recommended results.
swords No yes or no, specifies whether to filter similar queries with stopwords. Default is defined by the remote node.
  • example output 
{"query": "seeks",
"npeers": "4",
"date": "Wed, 27 Jul 2011 10:34:57 GMT",
"qtime": 0.08,
"snippets": [
  {
      "id": 1289851243,
      "title": "Seeks Project",
      "url": "<a href="/web/20130930043815/http://www.seeks-project.info/" class="external free" title="http://www.seeks-project.info/" rel="nofollow">http://www.seeks-project.info/</a>",
      "summary": "Seeks is a free and open P2P design and application for enabling social websearch. Its specific purpose is to regroup users whose queries are similar so they can share both the query results and their experience on these results. On th..."
      "seeks_meta": 4,
      "seeks_score": 0.000710273,
      "rank": 0.666667,
      "cite": "<a href="/web/20130930043815/http://www.seeks-project.info/" class="external free" title="http://www.seeks-project.info/" rel="nofollow">http://www.seeks-project.info/</a>",
      "engines": [
          "seeks",
      ],
      "words": [
          "results",
          "experience"
      ],
      "type": "webpage",
      "personalized": "yes"
      "snpeers": "3"
      "hits": "769"
  },
 ...]}
  • error values 
400 {"error":"bad parameter"}
recommend a result to the P2P collaborative filter for a given query

POST /recommendation/{query}

  • parameters
Name Required Description
query Yes url-encoded query string that describes recommended URL.
url Yes url-encoded recommended url
title Yes / No recommended url title, if not specified, fetched from the URL. If not available, parameter error, must be specified.
radius No radius of the halo for query storage. If not specified, left for the called node to decide.
url-check No 0 or 1, whether the node should check that the url exists or can be accessed. May not be allowed by every node.
lang No Specifies the language of the query. Recommended for better results.
  • example output
  • error values 
{ 400 "error":"bad parameter" }
{ 403 "error":"forbidden" }
{ 404 "error":"recommended resource cannot be found" }

403 error can be due to API rate limit, IP control or any protective control policy implemented by the server node.

remove a result recommendation from the P2P collaborative filter

DELETE /recommendation/{query}

  • parameters
Name Required Description
query Yes url-encoded query string
url No url-encoded url to remove. If specified, only removes this URL, otherwise removes query for the default radius, or radius if specified.
lang No Specifies the language of the query.
radius No radius of the halo for query removal. If not specified, left for the called node to decide.
  • example output
  • error values 
400 {"error":"bad parameter"}
get the peer ring list of the called node

GET /peers

  • parameters

none

  • example output 
"peers":
 [
  "<a href="/web/20130930043815/http://seeks-project.info/search.php" class="external free" title="http://seeks-project.info/search.php" rel="nofollow">http://seeks-project.info/search.php</a>",
  "<a href="/web/20130930043815/http://seeks.fr/" class="external free" title="http://seeks.fr/" rel="nofollow">http://seeks.fr/</a>
 ]

database direct calls

plugin: udb-service

resources: /find/dbr /find/bqc

HTTP method: GET, POST


get a database record
  • GET /find/dbr/{plugin_name}/{urkey}
    • where plugin_name is the name of the plugin that stored the record,
    • urkey is the key (160bit) of the record.


use LSH (locality sensitive hashing) to get a record of recommended queries and URLs
  • POST /find/bqc
    • body contains a halo of hashed queries, serialized as a protobuffer. TODO: link to more doc.
    • returns a record serialized as a protobuffer. TODO: link to more doc.
      • Add a custom footer
      Add a custom sidebar
      Clone this wiki locally