Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(storev3): only select the messageHash column when using a store query with include_data: false #2637

Closed
richard-ramos opened this issue Apr 26, 2024 · 1 comment
Assignees
Labels
effort/hours Estimated to be completed in a few hours
Milestone

Comments

@richard-ramos
Copy link
Member

Consider modifying the getMessages function in line 139 of https://github.com/waku-org/nwaku/blob/4e57308905158cff62f2616732867121c92b524f/waku/waku_archive/archive.nim to also pass the includeData value, and then, in the drivers define a query that will only do a SELECT messageHash FROM messages ....... if the value from includeData is true.

We should check the query plan from sqlite/postgres, of course, but depending on how the indices were defined, this should be more efficient, due to reduced disk i/o and data transfer from disk to memory, since the query will be satisfied by only reading the values from the index, without needing to fetch additional data from the table. I imagine this would be the case as the following indices are defined in the code as far as i could see all the columns match the attributes that can be used as criteria in the store query requests:

Primary Key: `messageHash`
Index: `contentTopic`, `pubsubTopic`, `senderTimestamp`, `id`
Index: `storedAt`
@Ivansete-status
Copy link
Collaborator

Closing as per #2761

I tested it with the message-finder establishing pages of 2 and it worked fine ( I temporarily modified the message-finder app to send queries with includeData == false .)

@github-project-automation github-project-automation bot moved this from In Progress to Done in Waku Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
effort/hours Estimated to be completed in a few hours
Projects
Archived in project
Development

No branches or pull requests

4 participants