Skip to content
This repository has been archived by the owner on Jun 21, 2020. It is now read-only.

Worker selection guard + Additional Unit Tests #121

Merged
merged 47 commits into from
May 6, 2019

Conversation

fredfortier
Copy link
Contributor

@fredfortier fredfortier commented Apr 17, 2019

Key features added:

  • More unit tests
  • Epoch state storage: The program can resume after shutdown and continue serving the active epoch
  • Worker selection guard: Only selected workers are allowed to request keys for the active epoch

elichai and others added 18 commits August 19, 2018 21:02
Merging new version 0.1.3 into master
# Conflicts:
#	enigma-principal/app/Cargo.lock
#	enigma-principal/app/src/epoch_u/epoch_provider.rs
#	enigma-principal/app/src/epoch_u/epoch_types.rs
…gma.specs.js`. TODOs: 1) Create a better test for the exception scenario of a failed Eth tx; 2) Unit test `get_state_keys` with mock EpochProvider that passes the worker selection.
… test runs before warming up the storage dirs. The fix is temporary. I'm currently working on improving the unit test organization. I will create a PR with a more elegant permanent fix. I did not detect this locally because the storage dirs existed. I will account for this in the future.
…hereum-types` broke some things in our code.
# Conflicts:
#	enigma-principal/app/Cargo.lock
# Conflicts:
#	enigma-core/app/Cargo.lock
#	enigma-principal/app/Cargo.lock
#	enigma-principal/app/Cargo.toml
#	enigma-tools-u/Cargo.toml
@fredfortier fredfortier changed the title Worker selection guard + Additional Unit Tests WIP: Worker selection guard + Additional Unit Tests Apr 17, 2019
@lacabra lacabra changed the title WIP: Worker selection guard + Additional Unit Tests Worker selection guard + Additional Unit Tests Apr 19, 2019
@lacabra lacabra changed the title Worker selection guard + Additional Unit Tests WIP: Worker selection guard + Additional Unit Tests Apr 19, 2019
Copy link
Contributor

@elichai elichai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please please try to make each PR do just one thing.
It's really hard to review big PRs,
every PR should implement one new feature or fix one bug, please.

enigma-core/app/Cargo.toml Outdated Show resolved Hide resolved
enigma-principal/app/src/cli/app.rs Show resolved Hide resolved
enigma-principal/app/src/esgx/epoch_keeper_u.rs Outdated Show resolved Hide resolved
enigma-principal/app/src/esgx/keys_keeper_u.rs Outdated Show resolved Hide resolved
enigma-principal/app/src/main.rs Outdated Show resolved Hide resolved
enigma-principal/enclave/Enclave.config.xml Outdated Show resolved Hide resolved
Token::Array(self.worker_params.stakes.iter().map(|s| Token::Uint(*s)).collect()),
];
encode(&tokens)
let raw_seed = H256::from(self.seed).0.to_vec();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please explain what are you doing here exactly? why don't use use the regular hash_multiple? Implementing the same thing multiple times is a bad idea.
(especially something as sensitive as this)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's fine. I wasn't aware of the hash_multiple then but I have no problem using it. That said, I don't believe that hash_multiple behave as expected so the signature won't recover. I just need to better understand how hash_multiple represents array lengths.

Copy link
Contributor

@moriaab moriaab May 3, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@elichai @fredfortier
Definitely KM node should use a general core implementation of hashing for multiple values, and not use a special function for specific values. The problem is that today in core there is no correct implementation of hashing of multiple values that may include nested arrays. Also the @fredfortier's implementation of raw_encode is not good as it may lead to the same output for different inputs. I and @elichai found an encoding for nested arrays, for which I did a formal proof that for any two different inputs, the outputs will be always different: https://github.com/enigmampc/protocol-discovery/blob/master/docs/hash_mul_nested.pdf.
So it seems to me that:

  1. The correct implementation should be added to core
  2. KM node should use this implementation
  3. The implementation of hashing in the contract for signature verification should be changed accordingly.

Meanwhile before 1 is done, I am OK to leave raw_encode as is, because signature verification works now, and we can progress with integration tests and other stuff. Once 1 is done, we will do 2 and 3 ASAP.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also the @fredfortier's implementation of raw_encode is not good as it may lead to the same output for different inputs.

@moriaab Do you mind providing an example? This isn't my implementation, I implemented exactly what I was told the algorithm was in order to match the contract.

enigma-principal/enclave/src/keys_keeper_t/mod.rs Outdated Show resolved Hide resolved
enigma-tools-m/src/keeper_types.rs Outdated Show resolved Hide resolved
@fredfortier fredfortier mentioned this pull request May 1, 2019
Copy link
Contributor Author

@fredfortier fredfortier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed each comment and made changes as appropriate.

enigma-principal/enclave/Enclave.config.xml Outdated Show resolved Hide resolved
Token::Array(self.worker_params.stakes.iter().map(|s| Token::Uint(*s)).collect()),
];
encode(&tokens)
let raw_seed = H256::from(self.seed).0.to_vec();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's fine. I wasn't aware of the hash_multiple then but I have no problem using it. That said, I don't believe that hash_multiple behave as expected so the signature won't recover. I just need to better understand how hash_multiple represents array lengths.

enigma-principal/enclave/Enclave.config.xml Outdated Show resolved Hide resolved
enigma-core/app/Cargo.toml Outdated Show resolved Hide resolved
enigma-principal/app/src/cli/app.rs Show resolved Hide resolved
enigma-principal/app/src/esgx/keys_keeper_u.rs Outdated Show resolved Hide resolved
enigma-principal/app/src/main.rs Outdated Show resolved Hide resolved
enigma-principal/enclave/Enclave.config.xml Outdated Show resolved Hide resolved
enigma-principal/enclave/src/keys_keeper_t/mod.rs Outdated Show resolved Hide resolved
enigma-tools-m/src/keeper_types.rs Outdated Show resolved Hide resolved
pub fn setup_epoch_storage() {
let mut path = storage_dir(ENCLAVE_DIR).unwrap();
path.push(EPOCH_DIR);
fs::remove_dir_all(path).unwrap_or_else(|_| println!("Epoch dir already empty"));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI you have expect(msg) for this

Copy link
Contributor Author

@fredfortier fredfortier May 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't expect panic just like unwrap? I want it do continue of the directory is already removed (possibly by another test running concurrently).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh. ok

elichai
elichai previously approved these changes May 5, 2019
# Conflicts:
#	enigma-core/app/Cargo.lock
#	enigma-core/enclave/Cargo.lock
#	enigma-principal/app/Cargo.lock
#	enigma-principal/enclave/Cargo.lock
@fredfortier fredfortier changed the base branch from develop to master May 6, 2019 17:46
@fredfortier fredfortier changed the base branch from master to develop May 6, 2019 17:46
@lacabra lacabra self-requested a review May 6, 2019 17:51
Copy link
Contributor

@lacabra lacabra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved on the basis of @elichai's previous approval. Fred and I noticed the addition of 3 very old commits, and we were trying to remove them by changing the base branch and change it back to develop. That did not change those commits, but reset the approval.

@fredfortier fredfortier merged commit 6270d2f into develop May 6, 2019
@Cashmaney Cashmaney deleted the worker-selection-guard branch December 5, 2019 13:18
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants