Skip to content

Issues: bbcarchdev/anansi

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

33 Open 36 Closed
33 Open 36 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Handle 307 (Temporary Redirect) and 308 (Permanent Redirect) properly bug crawler triaged
#70 opened Apr 28, 2017 by nevali
1
Clustering fails with divide by 0 error from the database bug crawler triaged
#69 opened Feb 7, 2017 by CygnusAlpha
@nevali
11
Anansi sometimes fails to follow 303 redirects bug crawler libcrawl triaged
#68 opened Jan 3, 2017 by townxelliot
19
Allow partitioning of the cluster based upon newness enhancement libcrawl queue:db
#66 opened Nov 14, 2016 by nevali
URI parsing failures bug processor:rdf triaged
#62 opened Nov 4, 2016 by cgueret
4
Allow a plug-in to perform actions when a new URI is added to the queue crawler enhancement libcrawl triaged
#59 opened Sep 21, 2016 by nevali
2
crawler-add should ignore blank lines bug crawler triaged
#58 opened Sep 5, 2016 by nevali
When the crawler is terminated, the cluster_leave() is not invoked bug crawler
#57 opened Nov 6, 2015 by nevali
Rearrange libraries and executables to make purpose clearer crawler enhancement libcrawl
#55 opened Oct 13, 2015 by nevali
Add a common memory allocation API which invokes abort() on failure enhancement libcrawl
#53 opened Sep 21, 2015 by nevali
Add message IDs for anything more severe than LOG_DEBUG crawler enhancement libcrawl processor:lod processor:rdf queue:db
#51 opened Sep 21, 2015 by nevali
@nevali
1
Support storing cached resources in WARC format enhancement libcrawl
#49 opened Jun 22, 2015 by nevali
Harmonise command-line options across Anansi, Twine, Quilt bug crawler enhancement triaged
#47 opened Jun 15, 2015 by nevali
Add rel="meta" as an equivalent to rel="alternate" in HTML parsing enhancement processor:rdf
#46 opened Jun 10, 2015 by nevali
Add support for an external license look-up service enhancement processor:rdf
#43 opened May 29, 2015 by nevali
Add customisable TTLs per root enhancement libcrawl queue:db
#42 opened May 25, 2015 by nevali
Explicitly remove an instance from a cluster when it shuts down, rather than relying on registry refresh timeout bug crawler enhancement
#40 opened May 22, 2015 by nevali
Either terminate or recover after cluster heartbeat failures bug crawler enhancement
#39 opened May 22, 2015 by nevali
De-priortise fetching from sites which have a low success rate enhancement libcrawl
#37 opened May 21, 2015 by nevali
Track the number of successful retrievals versus total per crawl root (i.e., success rate) enhancement libcrawl
#35 opened May 21, 2015 by nevali
LOD: License validation does not canonicalise URL forms bug processor:lod triaged
#30 opened Mar 6, 2015 by nevali
1
Consolidate RFC822-style header parsing enhancement processor:rdf
#28 opened Feb 21, 2015 by nevali
@nevali
Crawler RDF processor Accept list should not be hard-coded bug processor:rdf
#27 opened Feb 21, 2015 by nevali
@nevali
Perform schema migration before forking crawler enhancement queue:db
#26 opened Feb 17, 2015 by nevali
@nevali
Resource TTLs and back-offs should be configurable crawler enhancement libcrawl
#24 opened Feb 17, 2015 by nevali
@nevali
ProTip! Exclude everything labeled bug with -label:bug.