feat: make the timeout for event observers configurable

[obycode]

Description

Adds an optional timeout_ms to the configuration for event observers, allowing users to set a unique timeout for each observer.

Checklist

• Test coverage for new or modified code paths
• Changelog is updated
• Required documentation changes (e.g., docs/rpc/openapi.yaml and rpc-endpoints.md for v2 endpoints, event-dispatcher.md for new events)
• New clarity functions have corresponding PR in clarity-benchmarking repo
• New integration test(s) added to bitcoin-tests.yml

[CharlieC3]

@obycode Would it be better to err on the side of flexibility and change the default to something larger that wouldn't likely cause issues syncing a large block to a low-powered observer, like an API with minimal resources?
Instead of setting the default to 1 second, it might be better to set it to 60 seconds by default and let others rein it in if needed. Thoughts on this?

[obycode]

@obycode Would it be better to err on the side of flexibility and change the default to something larger that wouldn't likely cause issues syncing a large block to a low-powered observer, like an API with minimal resources? Instead of setting the default to 1 second, it might be better to set it to 60 seconds by default and let others rein it in if needed. Thoughts on this?

Yeah, that could make sense. I chose 1s as the default since that what it was hard-coded to before this change, so this change would have no effect.

The other problem with defaulting to 60s is that that could be surprising, since it halts progression of a node, a 60s timeout before seeing any logs is a lot, given that blocks will be coming in on the order of seconds.

[wileyj]

for what it's worth, confirmed that this change does allow genesis sync with the API (moved past block 0)

[CharlieC3]

The other problem with defaulting to 60s is that that could be surprising, since it halts progression of a node, a 60s timeout before seeing any logs is a lot, given that blocks will be coming in on the order of seconds.

I agree, but I think that says more about the logging from when a timeout is reached rather than the default setting itself. If the log when an observer timeout is specific, like "timeout reached (60s) trying to send block 12345 to observer my.domain.com:3999", it wouldn't be very confusing to debug.

Not suggesting this is needed now, but just thinking it would pose less consternation to set a wide boundary now and come back to it later with improved logging, rather than set a narrow boundary now and cause many operators to update all of their configs since 1s is likely too low for many people, and JW has already seen payloads take close to 30 seconds.

[pc-quiknode]

@obycode timeout_ms doesn't work with 3.0.0.0.0 for some reason and I keep getting this error with Nakamoto Testnet nodes

WARN [1729752052.503256] [testnet/stacks-node/src/event_dispatcher.rs:496] [main] Event dispatcher: connection or request failed to localhost:3700 - Custom { kind: WouldBlock, error: "Timed out while receiving request" }, backoff: 3s, attempts: 18

[blockstack-devops]

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.