feat: introduce batch read for scylla

[cisse21]

Description

• Speed up the read step and avoid network delays by introducing batch reads in scylla
• Refactored processorJobsForDest to be able to accommodate batch reads for dedup

Linear Ticket

Fixes PIPE-1651

Security

• The code changed/added as part of this pull request won't create any security issues with how the software is being used.

[codecov]

Codecov Report

Attention: Patch coverage is 84.75610% with 100 lines in your changes missing coverage. Please review.

Project coverage is 73.20%. Comparing base (da9f8c8) to head (256b4c4).

Files with missing lines	Patch %	Lines
processor/processor.go	85.04%	63 Missing and 20 partials ⚠️
services/dedup/badger/badger.go	60.00%	4 Missing and 2 partials ⚠️
services/dedup/mirrorBadger/mirrorBadger.go	57.14%	2 Missing and 1 partial ⚠️
services/dedup/mirrorScylla/mirrorScylla.go	57.14%	2 Missing and 1 partial ⚠️
services/dedup/scylla/scylla.go	95.08%	2 Missing and 1 partial ⚠️
processor/worker.go	80.00%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5212      +/-   ##
==========================================
+ Coverage   73.19%   73.20%   +0.01%     
==========================================
  Files         417      417              
  Lines       59085    59529     +444     
==========================================
+ Hits        43246    43581     +335     
- Misses      13426    13512      +86     
- Partials     2413     2436      +23     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[mihir20]

LGTM

[ktgowtham]

LGTM