Skip to content

Support of new Kafka API and other minor improvements

Compare
Choose a tag to compare
@sibiryakov sibiryakov released this 29 Nov 11:36
· 353 commits to master since this release

A long awaiting support of kafka-python 1.x.x client. Now Frontera is much more resistant to physical connectivity loss and is using new asynchronous Kafka API.
Other improvements:

  • SW consumes less CPU (because of rare state flushing),
  • requests creation api is changed in BaseCrawlingStrategy, and now it's batch oriented,
  • new article in the docs on cluster setup,
  • disable scoring log consumption option in DB worker,
  • fix of hbase drop table,
  • improved tests coverage.