Skip to content

faster/less cpu hungry!

Compare
Choose a tag to compare
@kbudde kbudde released this 17 Nov 05:17

#50 optimizations to speed up scraping and reduce cpu load

json: 30-40% cpu usage compared to the old version
bert: before it was using more cpu than json decoding (150%). Now it is just 75%.

recommended settings: (check readme for necessary rabbitmq version)
RABBIT_CAPABILITIES=no_sort,bert