Skip to content

Conversation

@Mantisus
Copy link
Collaborator

@Mantisus Mantisus self-assigned this Oct 13, 2025
@Mantisus Mantisus force-pushed the flaky-tests branch 3 times, most recently from 7180ed4 to ce992b9 Compare October 13, 2025 22:43
assert telemetry_data[-1]['attributes']['code.function.name'] == 'BasicCrawler.__run_task_function'
assert telemetry_data[-1]['resource']['attributes'] == dict(resource.attributes)

last_run_fuction_data = None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this needed? Maybe setting max_requests_per_crawl=1 and ConcurrencySettings(desired_concurrency=1, max_concurrency=1) should be enough

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this because I found some strange snapshots in the latest indexes a couple of times. Unfortunately, I didn't save the examples.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to fully understand the issue; otherwise, I am worried it could be hiding some other problem.
Why are there some "strange snapshots in latest indexes" sometimes?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. I probably encountered either a very rare case or a side effect caused by something else. Since I couldn't reproduce it. I'm removing this workaround for now, let's see if anything strange happens in CI.

@Mantisus Mantisus requested a review from Pijukatel October 15, 2025 13:33
@Pijukatel Pijukatel merged commit 3158448 into apify:master Oct 16, 2025
18 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test test_crawler_instrumentor_capability is flaky Test test_gz_sitemap_with_non_gzipped is flaky

2 participants