Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invalidate tablets data #351

Merged
merged 5 commits into from
Nov 26, 2024

Conversation

sylwiaszunejko
Copy link
Collaborator

Add tablets invalidation in case node is removed or keyspace/table is dropped/updated.

Fixes: #350

host_source.go Outdated Show resolved Hide resolved
host_source.go Outdated Show resolved Hide resolved
host_source.go Outdated Show resolved Hide resolved
@sylwiaszunejko
Copy link
Collaborator Author

I am still thinking about how to invalidate tablets in case of lost cc in terms of dropped keyspaces/tables. Right now we don't really do whole schema fetch. We only refresh particular keyspace when we try to get the metadata and there is none for that keyspace

gocql/metadata_scylla.go

Lines 266 to 282 in 77bae76

func (s *schemaDescriber) getSchema(keyspaceName string) (*KeyspaceMetadata, error) {
s.mu.Lock()
defer s.mu.Unlock()
metadata, found := s.cache[keyspaceName]
if !found {
// refresh the cache for this keyspace
err := s.refreshSchema(keyspaceName)
if err != nil {
return nil, err
}
metadata = s.cache[keyspaceName]
}
return metadata, nil
}
. So should we clear metadata and try to refresh it for every keyspace present in old metadata in case of lost cc? That way we can check if some keyspace/table was dropped, but this is a whole new logic to add so I would like to first ask about second opinion @dkropachev

@dkropachev
Copy link
Collaborator

I am still thinking about how to invalidate tablets in case of lost cc in terms of dropped keyspaces/tables. Right now we don't really do whole schema fetch. We only refresh particular keyspace when we try to get the metadata and there is none for that keyspace

gocql/metadata_scylla.go

Lines 266 to 282 in 77bae76

func (s *schemaDescriber) getSchema(keyspaceName string) (*KeyspaceMetadata, error) {
s.mu.Lock()
defer s.mu.Unlock()
metadata, found := s.cache[keyspaceName]
if !found {
// refresh the cache for this keyspace
err := s.refreshSchema(keyspaceName)
if err != nil {
return nil, err
}
metadata = s.cache[keyspaceName]
}
return metadata, nil
}

. So should we clear metadata and try to refresh it for every keyspace present in old metadata in case of lost cc? That way we can check if some keyspace/table was dropped, but this is a whole new logic to add so I would like to first ask about second opinion @dkropachev

let's address it in a separate PR.

host_source.go Outdated Show resolved Hide resolved
host_source.go Outdated Show resolved Hide resolved
host_source.go Outdated Show resolved Hide resolved
@sylwiaszunejko sylwiaszunejko force-pushed the invalidate-tablets-data branch 3 times, most recently from 9bcfc2e to fc20bbe Compare November 25, 2024 16:02
host_source.go Outdated Show resolved Hide resolved
@dkropachev dkropachev merged commit 5cd2a6c into scylladb:master Nov 26, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make driver invalidate tablets
2 participants