Scalling aggregation service

[adhir111]

Need some details on load handling capacity of aggregation service,
we do have the guide for vertical scaling in documentation , to choose the right instance type based on the load.
Need help on how to scale aggregation service horizontally in these scenarios:

• huge single batch

• processing multiple btahces in parallel ( historical data processing use cases)

[maybellineboon]

hi @adhir111 ,

Thanks for reaching out. For future questions for Aggregation Service, we recommend you to reach out via the Aggregation Service github page so that the appropriate team can address your question properly.

1. For batches, especially big batches, we recommend checking out the sizing guide. The sizing guide will help you estimate the size of instance you require base on your number of reports and domain. Additionally, you might want to check out the batching strategies to help you batch according to shared IDs and according to frequency. It is also advised to batch by advertiser for Attribution Reporting API to keep the batch sizes manageable.

2. For processing multiple batches in parallel, each aggregation batch/job is processed by a single instance and a given batch/job cannot be split over multiple instances. If you have multiple batches running, Aggregation Service auto-scales and increases the number of instance upto the max_capacity_ec2_instances. Aggregation Service has max_capacity_ec2_instances available in the <env>.auto.tfvars terraform file which allows you to customize the maximum number of instances to be created on your AWS account.