Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document cold start scenario #542

Open
krzysztof-kukla1974 opened this issue Jul 29, 2022 · 1 comment
Open

Document cold start scenario #542

krzysztof-kukla1974 opened this issue Jul 29, 2022 · 1 comment

Comments

@krzysztof-kukla1974
Copy link
Contributor

We should have documented special considerations related to Cold Start scenario.

  1. How to tweak Collect Data Source parameters?
  2. How to avoid throttling issues?
  3. How to monitor the system and adjust?
  4. How to use "merge data batches during cold start" parameter?
  5. How to configure Trace -> Setup -> Data Retrieval -> Run Interval - to not produce too many Data Batches on start and to not kill the 10 collection jobs in parallel queue.

Example:

  1. Stop Teams connector.
  2. Configure Trace -> Setup -> Data Retrieval -> Run Interval to 1 hour - so, only 1 Data Batch (each type) will be created per 1 hour.
  3. Abandon all Data Batches which were previously run (for instance with incorrect setup) - that would cancel all underlying Collect jobs.
  4. Reconfigure Data Source.
  5. Set Start Date to XYX.
  6. Set Frequency in Minutes to 1440.
  7. Set Merge Batches During Cold Start to Yes.
  8. Set Max Number of Batches To Merge to 30 <<< this means we'll create Data Batches each 30 day long.
  9. Reset Data Source.
  10. Enable Data Source.
@krzysztof-kukla1974
Copy link
Contributor Author

@peterhallerREL Maciej Kleczar would have good insight to this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant