[New Scheduler] Etcd installation & Implements EtcdClient #5031
Conversation
KeonHee
changed the title
| } | ||
| } | ||
|
|
||
| trait KeyValueStore { |
There was a problem hiding this comment.
Will this be used as an SPI? If so should it be more specific than KeyValueStore and related to the scheduler with the function templates
There was a problem hiding this comment.
While this trait can be used to have a new store, but most of the methods and their signatures conform to ETCD.
I feel like it is highly unlikely that we can replace it with another key-value store as it should support all functionalities such as TTL with lease/keepalive, Transaction, prefix, etc.
| @@ -272,6 +272,11 @@ | |||
|
|
|||
| "CONFIG_whisk_controller_activation_pollingFromDb": "{{ controller_activation_pollingFromDb | default(true) | lower }}" | |||
|
|
|||
| "CONFIG_whisk_etcd_hosts": "{{ etcd_connect_string }}" | |||
There was a problem hiding this comment.
Since ShardingContainerPoolBalancer does not use etcd, should we selectively configure these configurations?
There was a problem hiding this comment.
In the case of configuration, it would be good to add only what is needed when etcd is used in the controller & invoker. I will exclude it from this PR.
| } | ||
| } | ||
|
|
||
| trait KeyValueStore { |
There was a problem hiding this comment.
While this trait can be used to have a new store, but most of the methods and their signatures conform to ETCD.
I feel like it is highly unlikely that we can replace it with another key-value store as it should support all functionalities such as TTL with lease/keepalive, Transaction, prefix, etc.
| import pureconfig.loadConfigOrThrow | ||
|
|
||
| @RunWith(classOf[JUnitRunner]) | ||
| class EtcdKvTests extends FlatSpec with ScalaFutures with Matchers { |
There was a problem hiding this comment.
Should we need to selectively run these tests?
| @@ -73,10 +73,11 @@ ext.testSets = [ | |||
| "org/apache/openwhisk/core/cli/test/**", | |||
| "org/apache/openwhisk/core/limits/**", | |||
| "org/apache/openwhisk/core/scheduler/queue/test/ElasticSearchDurationCheck*", | |||
| "org/apache/openwhisk/common/etcd/**", | |||
There was a problem hiding this comment.
Excluded etcd from basic unit tests.
There was a problem hiding this comment.
So we would add tests for the scheduler when all contribution is over?
There was a problem hiding this comment.
Yes. One concern is that if the test is excluded, the code is merged without going through the CI test, so I would like to see if the CI also needs a pipeline for the scheduler.
| @@ -197,6 +198,14 @@ task testUnit(type: Test) { | |||
| exclude couchDbExcludes | |||
| } | |||
|
|
|||
| task testUnitEtcd(type: Test) { | |||
There was a problem hiding this comment.
The etcd test can be run with ./gradlew tests:testUnitEtcd.
Codecov Report
@@ Coverage Diff @@
## master #5031 +/- ##
==========================================
- Coverage 82.52% 74.85% -7.67%
==========================================
Files 206 210 +4
Lines 10006 10167 +161
Branches 445 440 -5
==========================================
- Hits 8257 7611 -646
- Misses 1749 2556 +807
Continue to review full report at Codecov.
|
|
Many subsequent modules are dependent on this module. |
- The scheduler PR apache/openwhisk#5031 introduced changes that required adaption in the setup of the openwhisk environment used for the automated tests.
- The scheduler PR apache/openwhisk#5031 introduced changes that required adaption in the setup of the openwhisk environment used for the automated tests.
- The scheduler PR apache/openwhisk#5031 introduced changes that required adaption in the setup of the openwhisk environment used for the automated tests.
- The scheduler PR apache/openwhisk#5031 introduced changes that required adaption in the setup of the openwhisk environment used for the automated tests.
- The scheduler PR apache/openwhisk#5031 introduced changes that required adaption in the setup of the openwhisk environment used for the automated tests.
- The scheduler PR apache/openwhisk#5031 introduced changes that required adaption in the setup of the openwhisk environment used for the automated tests.
Description
This pr covers the etcd installation and client implementation. And the etcd will be the coordinator of cluster members and the metadata store in the new scheduler.
My changes affect the following components
Types of changes
Checklist: