-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CollectionDescription Source refactor #2198
Labels
area/collections
Related to the collections system
area/db-system
Related to the core system related components of the DB
refactor
This issue specific to or requires *notable* refactoring of existing codebases and components
Milestone
Comments
AndrewSisley
added
area/collections
Related to the collections system
area/db-system
Related to the core system related components of the DB
refactor
This issue specific to or requires *notable* refactoring of existing codebases and components
labels
Jan 12, 2024
AndrewSisley
changed the title
CoolectionDescription Source refactor
CollectionDescription Source refactor
Jan 12, 2024
1 task
AndrewSisley
added a commit
that referenced
this issue
Jan 19, 2024
## Relevant issue(s) Partially resolves #2198 ## Description Makes CollectionDescription.Name Option. Changes secondary indexes to be indexed by Collection ID. Moves view queries into Sources property.
shahzadlone
pushed a commit
to shahzadlone/defradb
that referenced
this issue
Jan 22, 2024
## Relevant issue(s) Partially resolves sourcenetwork#2198 ## Description Makes CollectionDescription.Name Option. Changes secondary indexes to be indexed by Collection ID. Moves view queries into Sources property.
nasdf
pushed a commit
to nasdf/defradb
that referenced
this issue
Jan 23, 2024
## Relevant issue(s) Partially resolves sourcenetwork#2198 ## Description Makes CollectionDescription.Name Option. Changes secondary indexes to be indexed by Collection ID. Moves view queries into Sources property.
5 tasks
AndrewSisley
added a commit
that referenced
this issue
Feb 12, 2024
## Relevant issue(s) Resolves #2198 ## Description Models Collection SchemaVersions and migrations on Collections, instead of in the Lens Registry. Primary motivators for the change: 1. Boost the DevEx RE Lens and schema versions. It seemed pretty painful before to find transforms, yet alone figure out where that lay in the version history, or add them to a collection. 2. Unify the Collection history/schema history with the View system, so that they both roughly work in the same way. Reducing the cognitive and maintenance overhead of having two different systems for the same thing for both us and the users. Noteworthy stuff: 1. The branching of schema is not new, it is just more visible with this change. There is nothing we can really do to block it in a distributed system. I do now see it as supported, excluding Lens, which still needs a relatively minor tweak in order to handle the migration-pathing. 2. The model change in client/descriptions.go 3. The removal of stuff from the LensRegistry. This is currently kept to a minimum, however I would very much like to completely remove LensRegistry in the near future. 4. The CLI and the Http clients used to rely on the fact that db.SetMigration and registry.SetMigration did the same thing. This is no longer the case. 5. The CLI and the Http clients followed a different, flatter structure than the Go client, flattening the lens registry into the `lens`/`schema` pathways. This was inconsistent, and only worked when (4) was as it was. This has been changed out of necessity and the 3 clients now have matching structures. 6. The CLI still paths everything under `schema`, and the http client everything under `lens`, which is strange, but out of scope. I will open a ticket to make them the same. 7. Some of the schema versions in the existing schema tests have changed, I think there was a bug in the framework where it was not caring about their accuracy. It now cares. 8. Users can now specify their transforms when patching a schema. 9. Out of the 4 collection getting funcs on `client.Store`, only `GetAllCollections` has the ability to get inactive collections. I currently very much dislike that we have 4 functions for the same thing in the Go client, especially given that this is not reflected in the Http and CLI clients. In another ticket I would very much like us to collapse these four functions into one, and have that one function support the fetching of inactive collections.
shahzadlone
pushed a commit
to shahzadlone/defradb
that referenced
this issue
Feb 23, 2024
## Relevant issue(s) Partially resolves sourcenetwork#2198 ## Description Makes CollectionDescription.Name Option. Changes secondary indexes to be indexed by Collection ID. Moves view queries into Sources property.
shahzadlone
pushed a commit
to shahzadlone/defradb
that referenced
this issue
Feb 23, 2024
…#2286) ## Relevant issue(s) Resolves sourcenetwork#2198 ## Description Models Collection SchemaVersions and migrations on Collections, instead of in the Lens Registry. Primary motivators for the change: 1. Boost the DevEx RE Lens and schema versions. It seemed pretty painful before to find transforms, yet alone figure out where that lay in the version history, or add them to a collection. 2. Unify the Collection history/schema history with the View system, so that they both roughly work in the same way. Reducing the cognitive and maintenance overhead of having two different systems for the same thing for both us and the users. Noteworthy stuff: 1. The branching of schema is not new, it is just more visible with this change. There is nothing we can really do to block it in a distributed system. I do now see it as supported, excluding Lens, which still needs a relatively minor tweak in order to handle the migration-pathing. 2. The model change in client/descriptions.go 3. The removal of stuff from the LensRegistry. This is currently kept to a minimum, however I would very much like to completely remove LensRegistry in the near future. 4. The CLI and the Http clients used to rely on the fact that db.SetMigration and registry.SetMigration did the same thing. This is no longer the case. 5. The CLI and the Http clients followed a different, flatter structure than the Go client, flattening the lens registry into the `lens`/`schema` pathways. This was inconsistent, and only worked when (4) was as it was. This has been changed out of necessity and the 3 clients now have matching structures. 6. The CLI still paths everything under `schema`, and the http client everything under `lens`, which is strange, but out of scope. I will open a ticket to make them the same. 7. Some of the schema versions in the existing schema tests have changed, I think there was a bug in the framework where it was not caring about their accuracy. It now cares. 8. Users can now specify their transforms when patching a schema. 9. Out of the 4 collection getting funcs on `client.Store`, only `GetAllCollections` has the ability to get inactive collections. I currently very much dislike that we have 4 functions for the same thing in the Go client, especially given that this is not reflected in the Http and CLI clients. In another ticket I would very much like us to collapse these four functions into one, and have that one function support the fetching of inactive collections.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
area/collections
Related to the collections system
area/db-system
Related to the core system related components of the DB
refactor
This issue specific to or requires *notable* refactoring of existing codebases and components
Long term we wish to have something like the below. This issue begins moving us to that.
The text was updated successfully, but these errors were encountered: