Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support custom sharding function #237

Closed
Totktonada opened this issue Nov 13, 2021 · 1 comment · Fixed by #239
Closed

Support custom sharding function #237

Totktonada opened this issue Nov 13, 2021 · 1 comment · Fixed by #239
Assignees
@AnaNek
Labels
feature A new functionality

Comments

@Totktonada
Copy link
Member

Let crud know about _ddl_sharding_func space (tarantool/ddl#71) and use provided function to calculate a bucket id by a sharding key.

Part of #27.
@Totktonada Totktonada added the feature A new functionality label Nov 13, 2021
@Totktonada Totktonada changed the title Use custom sharding function Support custom sharding function Nov 13, 2021
@Totktonada Totktonada mentioned this issue Nov 13, 2021
Closed
@AnaNek AnaNek mentioned this issue Nov 24, 2021
Merged
@Totktonada Totktonada added the 2sp label Dec 3, 2021
@Totktonada Totktonada mentioned this issue Dec 6, 2021
Closed
@Totktonada
Copy link
Member Author

@AnaNek Please, take a look on #247 as well.

AnaNek added a commit that referenced this issue Jan 12, 2022
@AnaNek
Replace sharding methods in new directory sharding
d6de316 
Since this patch presents the implementation of support of
DDL sharding function, the number of files associated with
sharding will increase in the `common` folder. Therefore,
a separate directory was created for files containing
methods for working with sharding.

Part of #237
AnaNek added a commit that referenced this issue Jan 12, 2022
@AnaNek
Copy the content of sharding_key.lua file to `sharding_metadata.lua…
ca650da 
…` file

PR #181 introduced support of DDL sharding keys. Implementation of
sharding keys support contains methods that are common to support
sharding keys and sharding functions. That's why a separate file
`sharding_metadata.lua` was created to contain common methods.
In this commit content of `sharding_key.lua` file is coppied to
`sharding_metadata.lua` file to simplify a reviewer's life
and display the history of changes relative to PR #181 in
the following commits.

Part of #237
AnaNek added a commit that referenced this issue Jan 12, 2022
@AnaNek
Separate common sharding metadata methods from sharding key methods
9f2090f 
It would be more efficient to get sharding keys and
sharding functions from storage in one `fetch_on_storage`
function call. So, this function is common for sharding
keys and sharding functions support. As well as functions
for fetching on router. These methods are introduced
in `sharding_metadata` module. Methods for working
with sharding key structure are introduced in
sharding key module.

Part of #237
AnaNek added a commit that referenced this issue Jan 12, 2022
@AnaNek
Fix filling sharding key cache when sharding key data is incorrect
21dc4be 
Sharding key cache contains sharding key structures
separated by space name. Before this fix if sharding key
was incorrect for some space an error was returned and
sharding keys that were after the incorrect one
did not get into the cache. The solution is to
create a variable for an error and write a
message to it if an error occurred for the space
we are working with, output a warning for other spaces.

Part of #237
AnaNek added a commit that referenced this issue Jan 12, 2022
@AnaNek
Add functions from ddl for sharding func module
676ea4c 
In DDL PR tarantool/ddl#72
methods for checking and extracting sharding function
were introduced. These methods are needed for supporting
sharding functions in CRUD as well. In this commit these
methods were coppied from DDL and covered by unit tests.

Part of #237
AnaNek added a commit that referenced this issue Jan 12, 2022
@AnaNek
Add support of custom sharding func for crud methods
1e7bf0b 
This commit introduces modifications in functions
for fetching sharding metadata on storage and router
to get sharding function. Function `sharding.key_get_bucket_id`
calculates bucket_id using DDL sharding function if
sharding function exist for specified space. Description in
documentation, integration and unit tests are added
as well.

Closes #237
AnaNek added a commit that referenced this issue Jan 27, 2022
@AnaNek
Replace sharding methods in new directory sharding
f6d00a9 
Since this patch presents the implementation of support of
DDL sharding function, the number of files associated with
sharding will increase in the `common` folder. Therefore,
a separate directory was created for files containing
methods for working with sharding.

Part of #237
AnaNek added a commit that referenced this issue Jan 27, 2022
@AnaNek
Copy the content of sharding_key.lua file to `sharding_metadata.lua…
be073b3 
…` file

PR #181 introduced support of DDL sharding keys. Implementation of
sharding keys support contains methods that are common to support
sharding keys and sharding functions. That's why a separate file
`sharding_metadata.lua` was created to contain common methods.
In this commit content of `sharding_key.lua` file is coppied to
`sharding_metadata.lua` file to simplify a reviewer's life
and display the history of changes relative to PR #181 in
the following commits.

Part of #237
AnaNek added a commit that referenced this issue Jan 27, 2022
@AnaNek
Separate common sharding metadata methods from sharding key methods
92c3fbd 
It would be more efficient to get sharding keys and
sharding functions from storage in one `fetch_on_storage`
function call. So, this function is common for sharding
keys and sharding functions support. As well as functions
for fetching on router. These methods are introduced
in `sharding_metadata` module. Methods for working
with sharding key structure are introduced in
sharding key module.

Part of #237
AnaNek added a commit that referenced this issue Jan 27, 2022
@AnaNek
Fix filling sharding key cache when sharding key data is incorrect
83413a5 
Sharding key cache contains sharding key structures
separated by space name. Before this fix if sharding key
was incorrect for some space an error was returned and
sharding keys that were after the incorrect one
did not get into the cache. The solution is to
create a variable for an error and write a
message to it if an error occurred for the space
we are working with, output a warning for other spaces.

Part of #237
AnaNek added a commit that referenced this issue Jan 27, 2022
@AnaNek
Add functions from ddl for sharding func module
10f2554 
In DDL PR tarantool/ddl#72
methods for checking and extracting sharding function
were introduced. These methods are needed for supporting
sharding functions in CRUD as well. In this commit these
methods were coppied from DDL and covered by unit tests.

Part of #237
AnaNek added a commit that referenced this issue Jan 27, 2022
@AnaNek
Add support of custom sharding func for crud methods
d922131 
This commit introduces modifications in functions
for fetching sharding metadata on storage and router
to get sharding function. Function `sharding.key_get_bucket_id`
calculates bucket_id using DDL sharding function if
sharding function exist for specified space. Description in
documentation, integration and unit tests are added
as well.

Closes #237
AnaNek added a commit that referenced this issue Jan 27, 2022
@AnaNek
Fix filling sharding key cache when sharding key data is incorrect
0d60f99 
Sharding key cache contains sharding key structures
separated by space name. Before this fix if sharding key
was incorrect for some space an error was returned and
sharding keys that were after the incorrect one
did not get into the cache. The solution is to
create a variable for an error and write a
message to it if an error occurred for the space
we are working with, output a warning for other spaces.

Part of #237
AnaNek added a commit that referenced this issue Jan 27, 2022
@AnaNek
Add functions from ddl for sharding func module
c0cfb77 
In DDL PR tarantool/ddl#72
methods for checking and extracting sharding function
were introduced. These methods are needed for supporting
sharding functions in CRUD as well. In this commit these
methods were coppied from DDL and covered by unit tests.

Part of #237
AnaNek added a commit that referenced this issue Jan 27, 2022
@AnaNek
Add support of custom sharding func for crud methods
7fec1f3 
This commit introduces modifications in functions
for fetching sharding metadata on storage and router
to get sharding function. Function `sharding.key_get_bucket_id`
calculates bucket_id using DDL sharding function if
sharding function exist for specified space. Description in
documentation, integration and unit tests are added
as well.

Closes #237
AnaNek added a commit that referenced this issue Jan 28, 2022
@AnaNek
Copy the content of sharding_key.lua file to `sharding_metadata.lua…
98d555b 
…` file

PR #181 introduced support of DDL sharding keys. Implementation of
sharding keys support contains methods that are common to support
sharding keys and sharding functions. That's why a separate file
`sharding_metadata.lua` was created to contain common methods.
In this commit content of `sharding_key.lua` file is coppied to
`sharding_metadata.lua` file to simplify a reviewer's life
and display the history of changes relative to PR #181 in
the following commits.

Part of #237
AnaNek added a commit that referenced this issue Jan 28, 2022
@AnaNek
Separate common sharding metadata methods from sharding key methods
396f282 
It would be more efficient to get sharding keys and
sharding functions from storage in one `fetch_on_storage`
function call. So, this function is common for sharding
keys and sharding functions support. As well as functions
for fetching on router. These methods are introduced
in `sharding_metadata` module. Methods for working
with sharding key structure are introduced in
sharding key module.

Part of #237
AnaNek added a commit that referenced this issue Jan 28, 2022
@AnaNek
Fix filling sharding key cache when sharding key data is incorrect
dbb314a 
Sharding key cache contains sharding key structures
separated by space name. Before this fix if sharding key
was incorrect for some space an error was returned and
sharding keys that were after the incorrect one
did not get into the cache. The solution is to
create a variable for an error and write a
message to it if an error occurred for the space
we are working with, output a warning for other spaces.

Part of #237
AnaNek added a commit that referenced this issue Jan 28, 2022
@AnaNek
Add functions from ddl for sharding func module
a4ea142 
In DDL PR tarantool/ddl#72
methods for checking and extracting sharding function
were introduced. These methods are needed for supporting
sharding functions in CRUD as well. In this commit these
methods were coppied from DDL and covered by unit tests.

Part of #237
AnaNek added a commit that referenced this issue Jan 28, 2022
@AnaNek
Add support of custom sharding func for crud methods
c63b589 
This commit introduces modifications in functions
for fetching sharding metadata on storage and router
to get sharding function. Function `sharding.key_get_bucket_id`
calculates bucket_id using DDL sharding function if
sharding function exist for specified space. Description in
documentation, integration and unit tests are added
as well.

Closes #237
AnaNek added a commit that referenced this issue Jan 28, 2022
@AnaNek
Replace sharding methods in new directory sharding
0ca4f43 
Since this patch presents the implementation of support of
DDL sharding function, the number of files associated with
sharding will increase in the `common` folder. Therefore,
a separate directory was created for files containing
methods for working with sharding.

Part of #237
AnaNek added a commit that referenced this issue Feb 1, 2022
@AnaNek
Replace sharding methods in new directory sharding
14d39f2 
Since this patch presents the implementation of support of
DDL sharding function, the number of files associated with
sharding will increase in the `common` folder. Therefore,
a separate directory was created for files containing
methods for working with sharding.

Part of #237
AnaNek added a commit that referenced this issue Feb 1, 2022
@AnaNek
Copy the content of sharding_key.lua file to `sharding_metadata.lua…
9016ed6 
…` file

PR #181 introduced support of DDL sharding keys. Implementation of
sharding keys support contains methods that are common to support
sharding keys and sharding functions. That's why a separate file
`sharding_metadata.lua` was created to contain common methods.
This commit relocates functions for fetching sharding key from
`sharding_key.lua` file to `sharding_metadata.lua` file
to simplify a reviewer's life and display the history of
changes relative to PR #181 in the following commits.

Part of #237
AnaNek added a commit that referenced this issue Feb 1, 2022
@AnaNek
Separate common sharding metadata methods from sharding key methods
22ec5c6 
It would be more efficient to get sharding keys and
sharding functions from storage in one `fetch_on_storage`
function call. So, this function is common for sharding
keys and sharding functions support. As well as functions
for fetching on router. These methods are introduced
in `sharding_metadata` module. Methods for working
with sharding key structure are introduced in
sharding key module.

Part of #237
AnaNek added a commit that referenced this issue Feb 1, 2022
@AnaNek
Fix filling sharding key cache when sharding key data is incorrect
a6c0f13 
Sharding key cache contains sharding key structures
separated by space name. Before this fix if sharding key
was incorrect for some space an error was returned and
sharding keys that were after the incorrect one
did not get into the cache. The solution is to
create a variable for an error and write a
message to it if an error occurred for the space
we are working with, output a warning for other spaces.

Part of #237
AnaNek added a commit that referenced this issue Feb 1, 2022
@AnaNek
Add functions from ddl for sharding func module
561a66a 
In DDL PR tarantool/ddl#72
methods for checking and extracting sharding function
were introduced. These methods are needed for supporting
sharding functions in CRUD as well. In this commit these
methods were coppied from DDL and covered by unit tests.

Part of #237
AnaNek added a commit that referenced this issue Feb 1, 2022
@AnaNek
Add support of custom sharding func for crud methods
2161b99 
This commit introduces modifications in functions
for fetching sharding metadata on storage and router
to get sharding function. Function `sharding.key_get_bucket_id`
calculates bucket_id using DDL sharding function if
sharding function exist for specified space. Description in
documentation, integration and unit tests are added
as well.

Closes #237
AnaNek added a commit that referenced this issue Feb 7, 2022
@AnaNek
Copy the content of sharding_key.lua file to `sharding_metadata.lua…
6e0fcc7 
…` file

PR #181 introduced support of DDL sharding keys. Implementation of
sharding keys support contains methods that are common to support
sharding keys and sharding functions. That's why a separate file
`sharding_metadata.lua` was created to contain common methods.
This commit relocates functions for fetching sharding key from
`sharding_key.lua` file to `sharding_metadata.lua` file
to simplify a reviewer's life and display the history of
changes relative to PR #181 in the following commits.

Part of #237
AnaNek added a commit that referenced this issue Feb 7, 2022
@AnaNek
Separate common sharding metadata methods from sharding key methods
891666a 
It would be more efficient to get sharding keys and
sharding functions from storage in one `fetch_on_storage`
function call. So, this function is common for sharding
keys and sharding functions support. As well as functions
for fetching on router. These methods are introduced
in `sharding_metadata` module. Methods for working
with sharding key structure are introduced in
sharding key module.

Part of #237
AnaNek added a commit that referenced this issue Feb 7, 2022
@AnaNek
Fix filling sharding key cache when sharding key data is incorrect
06fe264 
Sharding key cache contains sharding key structures
separated by space name. Before this fix if sharding key
was incorrect for some space an error was returned and
sharding keys that were after the incorrect one
did not get into the cache. The solution is to
create a variable for an error and write a
message to it if an error occurred for the space
we are working with, output a warning for other spaces.

Part of #237
AnaNek added a commit that referenced this issue Feb 7, 2022
@AnaNek
Add functions from ddl for sharding func module
ef1bc5b 
In DDL PR tarantool/ddl#72
methods for checking and extracting sharding function
were introduced. These methods are needed for supporting
sharding functions in CRUD as well. In this commit these
methods were coppied from DDL and covered by unit tests.

Part of #237
AnaNek added a commit that referenced this issue Feb 7, 2022
@AnaNek
Add support of custom sharding func for crud methods
45f9a7d 
This commit introduces modifications in functions
for fetching sharding metadata on storage and router
to get sharding function. Function `sharding.key_get_bucket_id`
calculates bucket_id using DDL sharding function if
sharding function exist for specified space. Description in
documentation, integration and unit tests are added
as well.

Closes #237
AnaNek added a commit that referenced this issue Feb 8, 2022
@AnaNek
Add functions from ddl for sharding func module
f997ae0 
In DDL PR tarantool/ddl#72
methods for checking and extracting sharding function
were introduced. These methods are needed for supporting
sharding functions in CRUD as well. In this commit these
methods were coppied from DDL and covered by unit tests.

Part of #237
AnaNek added a commit that referenced this issue Feb 8, 2022
@AnaNek
Add support of custom sharding func for crud methods
21fe7bf 
This commit introduces modifications in functions
for fetching sharding metadata on storage and router
to get sharding function. Function `sharding.key_get_bucket_id`
calculates bucket_id using DDL sharding function if
sharding function exist for specified space. Description in
documentation, integration and unit tests are added
as well.

Closes #237
AnaNek added a commit that referenced this issue Feb 21, 2022
@AnaNek
Replace sharding methods in new directory sharding
e820c9f 
Since this patch presents the implementation of support of
DDL sharding function, the number of files associated with
sharding will increase in the `common` folder. Therefore,
a separate directory was created for files containing
methods for working with sharding.

Part of #237
AnaNek added a commit that referenced this issue Feb 21, 2022
@AnaNek
Copy the content of sharding_key.lua file to `sharding_metadata.lua…
cc69113 
…` file

PR #181 introduced support of DDL sharding keys. Implementation of
sharding keys support contains methods that are common to support
sharding keys and sharding functions. That's why a separate file
`sharding_metadata.lua` was created to contain common methods.
This commit relocates functions for fetching sharding key from
`sharding_key.lua` file to `sharding_metadata.lua` file
to simplify a reviewer's life and display the history of
changes relative to PR #181 in the following commits.

Part of #237
AnaNek added a commit that referenced this issue Feb 21, 2022
@AnaNek
Separate common sharding metadata methods from sharding key methods
02e6fb1 
It would be more efficient to get sharding keys and
sharding functions from storage in one `fetch_on_storage`
function call. So, this function is common for sharding
keys and sharding functions support. As well as functions
for fetching on router. These methods are introduced
in `sharding_metadata` module. Methods for working
with sharding key structure are introduced in
sharding key module.

Part of #237
AnaNek added a commit that referenced this issue Feb 21, 2022
@AnaNek
Fix filling sharding key cache when sharding key data is incorrect
2ed4841 
Sharding key cache contains sharding key structures
separated by space name. Before this fix if sharding key
was incorrect for some space an error was returned and
sharding keys that were after the incorrect one
did not get into the cache. The solution is to
create a variable for an error and write a
message to it if an error occurred for the space
we are working with, output a warning for other spaces.

Part of #237
AnaNek added a commit that referenced this issue Feb 21, 2022
@AnaNek
Add functions from ddl for sharding func module
6b7151c 
In DDL PR tarantool/ddl#72
methods for checking and extracting sharding function
were introduced. These methods are needed for supporting
sharding functions in CRUD as well. In this commit these
methods were coppied from DDL and covered by unit tests.

Part of #237
AnaNek added a commit that referenced this issue Feb 21, 2022
@AnaNek
Add support of custom sharding func for crud methods
37e37c0 
This commit introduces modifications in functions
for fetching sharding metadata on storage and router
to get sharding function. Function `sharding.key_get_bucket_id`
calculates bucket_id using DDL sharding function if
sharding function exist for specified space. Description in
documentation, integration and unit tests are added
as well.

Closes #237
AnaNek added a commit that referenced this issue Feb 21, 2022
@AnaNek
Add support of custom sharding func for crud methods
2887e18 
This commit introduces modifications in functions
for fetching sharding metadata on storage and router
to get sharding function. Function `sharding.key_get_bucket_id`
calculates bucket_id using DDL sharding function if
sharding function exist for specified space. Description in
documentation, integration and unit tests are added
as well.

Closes #237
Totktonada pushed a commit that referenced this issue Feb 21, 2022
@AnaNek @Totktonada
Replace sharding methods in new directory sharding
300c9a9 
Since this patch presents the implementation of support of
DDL sharding function, the number of files associated with
sharding will increase in the `common` folder. Therefore,
a separate directory was created for files containing
methods for working with sharding.

Part of #237
Totktonada pushed a commit that referenced this issue Feb 21, 2022
@AnaNek @Totktonada
Copy the content of sharding_key.lua file to `sharding_metadata.lua…
c48b96f 
…` file

PR #181 introduced support of DDL sharding keys. Implementation of
sharding keys support contains methods that are common to support
sharding keys and sharding functions. That's why a separate file
`sharding_metadata.lua` was created to contain common methods.
This commit relocates functions for fetching sharding key from
`sharding_key.lua` file to `sharding_metadata.lua` file
to simplify a reviewer's life and display the history of
changes relative to PR #181 in the following commits.

Part of #237
Totktonada pushed a commit that referenced this issue Feb 21, 2022
@AnaNek @Totktonada
Separate common sharding metadata methods from sharding key methods
73b017d 
It would be more efficient to get sharding keys and
sharding functions from storage in one `fetch_on_storage`
function call. So, this function is common for sharding
keys and sharding functions support. As well as functions
for fetching on router. These methods are introduced
in `sharding_metadata` module. Methods for working
with sharding key structure are introduced in
sharding key module.

Part of #237
Totktonada pushed a commit that referenced this issue Feb 21, 2022
@AnaNek @Totktonada
Fix filling sharding key cache when sharding key data is incorrect
4c09bb2 
Sharding key cache contains sharding key structures
separated by space name. Before this fix if sharding key
was incorrect for some space an error was returned and
sharding keys that were after the incorrect one
did not get into the cache. The solution is to
create a variable for an error and write a
message to it if an error occurred for the space
we are working with, output a warning for other spaces.

Part of #237
Totktonada pushed a commit that referenced this issue Feb 21, 2022
@AnaNek @Totktonada
Add functions from ddl for sharding func module
75417d4 
In DDL PR tarantool/ddl#72
methods for checking and extracting sharding function
were introduced. These methods are needed for supporting
sharding functions in CRUD as well. In this commit these
methods were coppied from DDL and covered by unit tests.

Part of #237
Totktonada pushed a commit that referenced this issue Feb 21, 2022
@AnaNek @Totktonada
Add support of custom sharding func for crud methods
263900b 
This commit introduces modifications in functions
for fetching sharding metadata on storage and router
to get sharding function. Function `sharding.key_get_bucket_id`
calculates bucket_id using DDL sharding function if
sharding function exist for specified space. Description in
documentation, integration and unit tests are added
as well.

Closes #237
DifferentialOrange added a commit that referenced this issue Apr 8, 2022
@DifferentialOrange
changelog: fix 0.10.0 entries
73e3678 
Several "0.10.0" section changelog entries (see PRs #230 and #239) were
added after 0.10.0 release. This patch fixes the inconsistency by moving
them to "Unreleased" section.

Follows up #74, #237
@DifferentialOrange DifferentialOrange mentioned this issue Apr 8, 2022
Merged
1 task
DifferentialOrange added a commit that referenced this issue Apr 10, 2022
@DifferentialOrange
changelog: fix 0.10.0 entries
d1b1c1a 
Several "0.10.0" section changelog entries (see PRs #230 and #239) were
added after 0.10.0 release. This patch fixes the inconsistency by moving
them to "Unreleased" section.

Follows up #74, #237
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@AnaNek AnaNek

Labels
feature A new functionality
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Support custom sharding func tarantool/crud
2 participants
@Totktonada @AnaNek