Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom cursors support #346

Open
bitgorbovsky opened this issue Mar 3, 2023 · 1 comment
Open

Custom cursors support #346

bitgorbovsky opened this issue Mar 3, 2023 · 1 comment
Assignees
Labels
3sp customer feature A new functionality

Comments

@bitgorbovsky
Copy link

bitgorbovsky commented Mar 3, 2023

There is a problem with cursor based pagination on requests by secondary indexes. Consider a simple key-value schema:

field type info
key string primary key
value string
tag string secondary index, non unique

The call crud.select('kv', {{'=', 'tag', 'tagValue'}}, {first = 10, after = ...}) makes a fullscan of secondary index from starting point (tagValue) to tuple equaling cursor. Under heavy load it leads to CPU consumption and cluster performance degradation.

To solve this problem, we have to create a special unique index paging with two fields {tag, key} and implement pagination manually, by executing such calls like

local rs = crud.select('kv', {{'==', 'paging', {'tagValue', box.NULL}}}, {first = 10})

...

local last_key = rs.ros[#rs.rows][0]
rs = crud.select('kv', {{'>', 'paging', {'tagValue', last_key}}}, {first = 10})
...

It would be nice to introduce some feature like passing name of paging index and using key from this index as cursors

@bitgorbovsky bitgorbovsky added feature A new functionality teamE labels Mar 3, 2023
@Mons Mons removed their assignment Mar 17, 2023
@kyukhin kyukhin removed the teamE label Mar 17, 2023
@Mons Mons assigned unera and unassigned artur-barsegyan Jun 13, 2023
@DifferentialOrange
Copy link
Member

We also may consider supporting new pagiation here: tarantool/tarantool#7639

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3sp customer feature A new functionality
Projects
None yet
Development

No branches or pull requests

9 participants