Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: metadata book #638

Merged
merged 3 commits into from
May 15, 2020
Merged

feat: metadata book #638

merged 3 commits into from
May 15, 2020

Conversation

vasco-santos
Copy link
Member

This PR adds the metadataBook to the PeerStore as described in #627

Closes #627

@vasco-santos vasco-santos requested a review from jacobheun May 15, 2020 08:23
protocols: []
})
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't this function just get a union of all the keys from each book and then do a this.get(peerId) on that? This would simplify data retrieval and remove the need for the strict ordering that's happening here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This got definitely cleaner! thanks :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are all maps so in theory you could simplify it further to:

const storedPeers = new Set([
  ...this.addressBook.data.keys(),
  ...this.keyBook.data.keys(),
  ...this.protoBook.data.keys(),
  ...this.metadataBook.data.keys(),  
])

Also, protobook is listed twice and keybook is missing.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's true! Looking into it

src/peer-store/metadata-book.js Outdated Show resolved Hide resolved
src/peer-store/metadata-book.js Outdated Show resolved Hide resolved
src/peer-store/metadata-book.js Outdated Show resolved Hide resolved
@@ -58,6 +65,7 @@ class PersistentPeerStore extends PeerStore {
this.on('change:protocols', this._addDirtyPeer)
this.on('change:multiaddrs', this._addDirtyPeer)
this.on('change:pubkey', this._addDirtyPeer)
this.on('change:metadata', this._addDirtyPeerMetadata)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we do the same thing as the other events and just mark the peer itself as dirty, then get the latest when we go to commit?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is one issue with that approach, that I cannot circumvent unless I add the extra dirty record.

While in the other books we know that if something changed and it is not in the book anymore, it is deleted, in the metadataBook is not that simple. So, for adding it is simple, but for deleting we would not know the metadata keys that were deleted (because they were simply gone from the book), which would result in metadata being persisted and not removed anymore. When we do the restart, that metadata would be around.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough, we could check the datastore keys and then delete the ones that we no longer have, but this might have some drawbacks to shutdown storage time (extra datastore round trips). We can punt this decision for now, as there's no user facing impact with this.

test/peer-store/metadata-book.spec.js Outdated Show resolved Hide resolved
test/peer-store/metadata-book.spec.js Outdated Show resolved Hide resolved
test/peer-store/peer-store.spec.js Outdated Show resolved Hide resolved
test/peer-store/peer-store.spec.js Outdated Show resolved Hide resolved
test/peer-store/peer-store.spec.js Outdated Show resolved Hide resolved
@vasco-santos vasco-santos force-pushed the feat/metadata-book branch 2 times, most recently from 2ec7628 to 60afd73 Compare May 15, 2020 14:40
Copy link
Contributor

@jacobheun jacobheun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one thing, otherwise this looks good to me.

protocols: []
})
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are all maps so in theory you could simplify it further to:

const storedPeers = new Set([
  ...this.addressBook.data.keys(),
  ...this.keyBook.data.keys(),
  ...this.protoBook.data.keys(),
  ...this.metadataBook.data.keys(),  
])

Also, protobook is listed twice and keybook is missing.

@vasco-santos vasco-santos requested a review from jacobheun May 15, 2020 16:59
Copy link
Contributor

@jacobheun jacobheun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚢

@jacobheun jacobheun merged commit ff9e85c into 0.28.x May 15, 2020
@jacobheun jacobheun deleted the feat/metadata-book branch May 15, 2020 17:39
jacobheun pushed a commit that referenced this pull request May 28, 2020
* feat: metadata book

* chore: address review

* chore: address review
jacobheun pushed a commit that referenced this pull request May 28, 2020
* feat: metadata book

* chore: address review

* chore: address review
@vasco-santos vasco-santos mentioned this pull request Jun 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants