Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Breakup core/keys.go file #3198

Merged
merged 3 commits into from
Oct 30, 2024

Conversation

AndrewSisley
Copy link
Contributor

Relevant issue(s)

Resolves #3197

Description

Extracts core/keys.go to multiple files.

Original file had grown way to large and was getting quite painful to find stuff in it, and I need to add new types to it shortly.

@AndrewSisley AndrewSisley added area/db-system Related to the core system related components of the DB refactor This issue specific to or requires *notable* refactoring of existing codebases and components labels Oct 29, 2024
@AndrewSisley AndrewSisley added this to the DefraDB v0.15 milestone Oct 29, 2024
@AndrewSisley AndrewSisley requested a review from a team October 29, 2024 19:10
@AndrewSisley AndrewSisley self-assigned this Oct 29, 2024
@AndrewSisley AndrewSisley changed the title refactor: Extract core/keys.go to multiple files refactor: Breakup core/keys.go file Oct 29, 2024
@AndrewSisley AndrewSisley changed the title refactor: Breakup core/keys.go file refactor: Breakup core/keys.go file Oct 29, 2024
Original file had grown way to large and was getting quite painful to find stuff in it, and I need to add new types to it shortly.
Copy link

codecov bot commented Oct 29, 2024

Codecov Report

Attention: Patch coverage is 84.92268% with 117 lines in your changes missing coverage. Please review.

Project coverage is 77.36%. Comparing base (b4b2bf2) to head (5852067).
Report is 1 commits behind head on develop.

Files with missing lines Patch % Lines
internal/keys/datastore_doc.go 88.24% 15 Missing and 5 partials ⚠️
internal/keys/systemstore_collection_root.go 69.44% 8 Missing and 3 partials ⚠️
internal/keys/datastore_view_item.go 71.43% 10 Missing ⚠️
internal/keys/headstore_doc.go 81.82% 6 Missing and 2 partials ⚠️
internal/keys/peerstore_replicator_retry.go 68.75% 4 Missing and 1 partial ⚠️
internal/keys/peerstore_replicator_retry_doc.go 77.27% 4 Missing and 1 partial ⚠️
internal/keys/systemstore_collection_schema.go 82.76% 4 Missing and 1 partial ⚠️
internal/keys/systemstore_p2p_collection.go 73.68% 4 Missing and 1 partial ⚠️
internal/keys/systemstore_schema_root.go 82.76% 4 Missing and 1 partial ⚠️
internal/keys/errors.go 33.33% 4 Missing ⚠️
... and 18 more
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #3198      +/-   ##
===========================================
- Coverage    77.52%   77.36%   -0.15%     
===========================================
  Files          357      376      +19     
  Lines        34801    34807       +6     
===========================================
- Hits         26976    26928      -48     
- Misses        6216     6254      +38     
- Partials      1609     1625      +16     
Flag Coverage Δ
all-tests 77.36% <84.92%> (-0.15%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
internal/core/crdt/base.go 66.67% <ø> (-4.76%) ⬇️
internal/core/crdt/composite.go 78.31% <100.00%> (+3.61%) ⬆️
internal/core/crdt/counter.go 66.67% <ø> (ø)
internal/core/crdt/lwwreg.go 74.65% <ø> (-4.23%) ⬇️
internal/core/data.go 91.61% <100.00%> (ø)
internal/core/encoding.go 64.60% <ø> (-13.11%) ⬇️
internal/db/collection_get.go 79.31% <ø> (-5.17%) ⬇️
internal/db/collection_id.go 79.17% <100.00%> (ø)
internal/db/db.go 66.82% <100.00%> (ø)
internal/db/description/schema.go 61.90% <100.00%> (ø)
... and 47 more

... and 7 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b4b2bf2...5852067. Read the comment docs.

Copy link
Collaborator

@fredcarle fredcarle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was about to reoganise keys within core but I think putting them in their own package makes a lot of sense. Just a todo and a suggestion before I approve.

// by the Apache License, Version 2.0, included in the file
// licenses/APL.txt.

package keys
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo: Please use the singular for package name. key

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why? Go has lots of packages with plural names (e.g. strings) and there is nothing in https://go.dev/blog/package-names against it. keys contains many different key-types, and according to the norms I have experienced, the name should be plural.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The packages in Go that use plural names are usually plural because the singular form is a reserved word or keyword (strings, types, errors, bytes).

It is generally preferred in the Go community to use singular names for packages. We've discussed this plenty of times before.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed over discord, and noted here for future reference:

When attempting to rename to key, I found this caused a large amount (+100) of variable shadowing (most places using this package want variables with the name key). Even if consensus is reached RE avoiding plural package names, we probably still want this to be called keys anyway (quite possibly the same reason why many std lib packages have plural names).

)

// DataStoreKey is a type that represents a key in the database.
type DataStoreKey struct {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Since the package name is keys (should be key), the Key suffix is less relevant. I would suggest removing it to get key.DataStore. This applies for all keys.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree, but I didn't want to bother dealing with the many niggles like this already present in the code I moved. I just want to break it up so I can add new stuff in a clean way.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've been doing this often lately — refactoring but not cleaning things up fully. It feels like a bad habit that's setting in that might hurt us in the future. This one is minor but still part of the habit so I'm pointing it out.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code is never cleaned up fully. Leaving as is.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've been doing this often lately — refactoring but not cleaning things up fully. It feels like a bad habit that's setting in that might hurt us in the future.

I am curious to know where we have done this lately? We should def have issues to track any pending cleanups if there were any.

Copy link
Collaborator

@fredcarle fredcarle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. If we come to consensus on a different consensus for package name later, we can open a PR to change it.

@AndrewSisley AndrewSisley merged commit 7c7c030 into sourcenetwork:develop Oct 30, 2024
42 of 43 checks passed
@AndrewSisley AndrewSisley deleted the 3197-brk-up-keys branch October 30, 2024 18:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/db-system Related to the core system related components of the DB refactor This issue specific to or requires *notable* refactoring of existing codebases and components
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Breakup core/keys.go
3 participants