[MISC] Parallel patroni calls #925
Conversation
Co-authored-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
dragomirp
force-pushed
the
parallel-patroni-calls-tlsv4
branch
from
241814d to
74903f1
Compare
dragomirp
changed the title
| if result := await task: | ||
| for task in tasks: | ||
| task.cancel() | ||
| await wait(tasks) |
There was a problem hiding this comment.
Get the first result, cancel the other requests.
| ) | ||
| return cluster_status.json()["members"] | ||
| return response["members"] | ||
| raise RetryError(last_attempt=Exception("Unable to reach any units")) |
There was a problem hiding this comment.
Most existing code should handle RetryErrors instead of empty lists.
| main() | ||
| assert _urlopen.call_args_list == [ | ||
| # Iteration 1. server2 is not called | ||
| call("http://server1:8008/cluster", timeout=5, context=sentinel.sslcontext), | ||
| # Iteration 2 local unit server1 is called first | ||
| call("http://server1:8008/cluster", timeout=5, context=sentinel.sslcontext), | ||
| call("http://server3:8008/cluster", timeout=5, context=sentinel.sslcontext), | ||
| # Iteration 3 Last known member is server3 | ||
| call("https://server3:8008/cluster", timeout=5, context=sentinel.sslcontext), | ||
| ] |
There was a problem hiding this comment.
Order is no longer deterministic.
| ) | ||
| cluster_status = json.loads(resp.read()) | ||
| loop = get_running_loop() | ||
| tasks = [loop.run_in_executor(None, call_url, url, context) for url in urls] |
There was a problem hiding this comment.
No deps when executing the script, so running urllib requests in an executor.
dragomirp
requested review from
a team,
taurus-forever and
marceloneppel
and removed request for
a team
There was a problem hiding this comment.
I have run tests, the deployment of 3 units cluster is about 5-10% faster (5 minutes instead of 5-6). The defers count is also slightly better (110 VS 127 for 3 units deployment).
Also topology observer behaves properly on Primary loose (without leader re-elected).
LGTM. Good work and the step into the proper direction for sure!
| ) | ||
| return cluster_status.json()["members"] | ||
| return response["members"] | ||
| raise RetryError(last_attempt=Exception("Unable to reach any units")) |
Async parallel requests for cluster status.
Checklist