Use a cluster generator to automatically create and test clusters #494
Conversation
|
|
Python is mainly used for testing here so if you think upgrading is the right thing, go for it. |
| conf['remote_peer_connections'] = self.remote_connections | ||
| return dict(dyn_o_mite=conf) | ||
|
|
||
| def write_config(self, seeds_list): |
There was a problem hiding this comment.
We have another script for people to generate YAML files: https://github.com/Netflix/dynomite/blob/dev/scripts/dynomite/generate_yamls.py Shall we combine them together, or at least somehow leverage each other to avoid duplicate scripts?
There was a problem hiding this comment.
I guess you will do this in a separate PR?
|
I'm making the PR to upgrade to Py3 right now. |
test/func_test.py
Outdated
| d3 = DynoNode(host="127.0.0.3", ip="127.0.0.3", data_store_port=22123) | ||
| d4 = DynoNode(host="127.0.0.4", ip="127.0.0.4", data_store_port=22124) | ||
| d5 = DynoNode(host="127.0.0.5", ip="127.0.0.5", data_store_port=22125) | ||
| r = RedisNode(host="127.0.1.1", ip="127.0.1.1", port=1212) |
There was a problem hiding this comment.
Whatever we got till now is great. However, this whole section is still hardcoded. Can it understand what dynamic cluster is created by cluster generator? Ideally, I would expect DynoNode to have a start/launch api that will start that particular node.
DynoCluster consists of many DynoNodes which have start/stop APIs etc.
There was a problem hiding this comment.
I believe I've remedied this in #495. Let me know if I've achieved the desired design.
Upgrade to Python 3 + plumb cluster_generator into func_test
To accommodate a desire to be able for developers to easily test against a variety of Dynamo rings, this change adds a Python script which autogenerates clusters, launches them, and runs the existing functional tests against them.
One goal that I kept in mind is to generate clusters deterministically based on the request file. Right now the generation is fully deterministic, but I can also make it non-deterministic, but print the PRNG state to stdout. This would allow a developer to reproduce a previous test run by passing the desired PRNG state as a shell argument.
Potential follow-ups: