Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Document client interfaces in client/db.go #1305

Merged
merged 29 commits into from
Apr 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
aed4040
Document client.DB
AndrewSisley Apr 6, 2023
b6cef50
Document client.DB.Store
AndrewSisley Apr 6, 2023
d3719aa
Document client.DB.NewTxn functions
AndrewSisley Apr 6, 2023
b31415f
Document client.DB.Root
AndrewSisley Apr 6, 2023
73b25a8
Document client.DB.Blockstore
AndrewSisley Apr 6, 2023
81f633d
Document client.DB.Close
AndrewSisley Apr 6, 2023
3a62c2f
Document client.DB.Events
AndrewSisley Apr 6, 2023
c22364c
Document client.DB.MaxTxnRetries
AndrewSisley Apr 6, 2023
818ee77
Document client.DB.PrintDump
AndrewSisley Apr 6, 2023
0b95bc5
Document client.Store
AndrewSisley Apr 6, 2023
a8228f4
Document client.Store.AddSchema
AndrewSisley Apr 6, 2023
4aa0e4d
Document client.Store.CreateCollection
AndrewSisley Apr 6, 2023
e7d6139
Document client.Store.GetCollectionByName
AndrewSisley Apr 6, 2023
655650b
Document client.Store.GetCollectionBySchemaID
AndrewSisley Apr 6, 2023
edaa039
Document client.Store.GetCollectionByVersionID
AndrewSisley Apr 6, 2023
43c1fb4
Document client.Store.GetAllCollections
AndrewSisley Apr 6, 2023
c7f689b
Document client.Store.ExecRequest
AndrewSisley Apr 6, 2023
70b1419
Document client.GQLResult
AndrewSisley Apr 6, 2023
cd5b0f5
Document client.GQLResult.Errors
AndrewSisley Apr 6, 2023
05e8695
Document client.GQLResult.Data
AndrewSisley Apr 6, 2023
961ad62
Document client.RequestResult
AndrewSisley Apr 6, 2023
818e175
Document client.RequestResult.GQL
AndrewSisley Apr 6, 2023
63bc34f
Document client.RequestResult.Pub
AndrewSisley Apr 6, 2023
f44c0e2
Remove commented out code
AndrewSisley Apr 6, 2023
16ffc02
PR FIXUP - Fix spelling (programmatic)
AndrewSisley Apr 6, 2023
3001b2d
PR FIXUP - DefraDB vs Defra
AndrewSisley Apr 6, 2023
428cb45
PR FIXUP - Fix spelling (constructed)
AndrewSisley Apr 6, 2023
8fdac05
PR FIXUP - Add warning to UpdateCollection
AndrewSisley Apr 6, 2023
a52060f
PR FIXUP - Fix pluralization (event)
AndrewSisley Apr 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 84 additions & 1 deletion client/db.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,30 +19,75 @@ import (
"github.com/sourcenetwork/defradb/events"
)

// DB is the primary public programmatic access point to the local DefraDB instance.
//
// It should be constructed via the [db] package, via the [db.NewDB] function.
type DB interface {
// Store contains DefraDB functions protected by an internal, short-lived, transaction, allowing safe
// access to common database read and write operations.
Store

// NewTxn returns a new transaction on the root store that may be managed externally.
//
// It may be used with other functions in the client package. It is not threadsafe.
NewTxn(context.Context, bool) (datastore.Txn, error)

// NewConcurrentTxn returns a new transaction on the root store that may be managed externally.
//
// It may be used with other functions in the client package. It is threadsafe and mutliple threads/Go routines
// can safely operate on it concurrently.
NewConcurrentTxn(context.Context, bool) (datastore.Txn, error)

// WithTxn returns a new [client.Store] that respects the given transaction.
WithTxn(datastore.Txn) Store

// Root returns the underlying root store, within which all data managed by DefraDB is held.
Root() datastore.RootStore

// Blockstore returns the blockstore, within which all blocks (commits) managed by DefraDB are held.
//
// It sits within the rootstore returned by [Root].
Blockstore() blockstore.Blockstore

// Close closes the database instance and releases any resources held.
//
// The behaviour of other functions in this package after this function has been called is undefined
// unless explicitly stated on the function in question.
//
// It does not explicitly clear any data from persisted storage, and a new [DB] instance may typically
// be created after calling this to resume operations on the prior data - this is however dependant on
// the behaviour of the rootstore provided on database instance creation, as this function will Close
// the provided rootstore.
Comment on lines +57 to +60
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: idk if we care lol but I realize this is just American vs UK spelling diffs at this point.

Suggested change
// It does not explicitly clear any data from persisted storage, and a new [DB] instance may typically
// be created after calling this to resume operations on the prior data - this is however dependant on
// the behaviour of the rootstore provided on database instance creation, as this function will Close
// the provided rootstore.
// It does not explicitly clear any data from persisted storage, and a new [DB] instance may typically
// be created after calling this to resume operations on the prior data - this is however dependent on
// the behavior of the rootstore provided on database instance creation, as this function will Close
// the provided rootstore.

Copy link
Contributor Author

@AndrewSisley AndrewSisley Apr 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UK English always. Not sure why Americans need to try to be special with everything...

Close(context.Context)

// Events returns the database event queue.
//
// It may be used to monitor database events - a new event will be yielded for each mutation.
// Note: it does not copy the queue, just the reference to it.
Comment on lines +63 to +66
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praise: useful note

Events() events.Events

// MaxTxnRetries returns the number of retries that this DefraDB instance has been configured to
// make in the event of a transaction conflict in certain scenarios.
//
// Currently this is only used within the P2P system and will not affect operations initiated by users.
MaxTxnRetries() int

// PrintDump logs the entire contents of the rootstore (all the data managed by this DefraDB instance).
//
// It is likely unwise to call this on a large database instance.
PrintDump(ctx context.Context) error
}

// Store contains the core DefraDB read-write operations.
type Store interface {
// P2P holds the P2P related methods that must be implemented by the database.
P2P

// AddSchema takes the provided GQL schema in SDL format, and applies it to the [Store],
// creating the necessary collections, request types, etc.
//
// All schema types provided must not exist prior to calling this, and they may not reference existing
// types previously defined.
Comment on lines +89 to +90
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thought(out of scope): Reading this I realize that we should probably allow new schema to reference existing ones. For example, I may want to add an Image schema that references the User collection (images are owned by users).

AddSchema(context.Context, string) error

// PatchSchema takes the given JSON patch string and applies it to the set of CollectionDescriptions
Expand All @@ -61,6 +106,10 @@ type Store interface {
// [FieldKindStringToEnumMapping].
PatchSchema(context.Context, string) error

// CreateCollection creates a new collection using the given description.
//
// WARNING: It does not currently update the GQL types, and as such a database restart is required after
// calling this if use of the new collection via GQL is desired (for example via [ExecRequest]).
CreateCollection(context.Context, CollectionDescription) (Collection, error)
Comment on lines +109 to 113
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

though: Seeing this warning makes me think that maybe this function shouldn't be exposed publicly. Unless I'm missing something, we don't use this function anywhere internally. We only use the private version of it for the adding schemas. It feels like giving users a chance to do something they may not want to.

Copy link
Contributor Author

@AndrewSisley AndrewSisley Apr 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is still nominally useful, and my preference long term is to fix both this and UpdateCollection so that they do actually update the GQL types.

That said, I think the lack of references might be quite new - I think they were referenced until the client txn rework, but I'm not 100% sure.

I have just spotted that UpdateCollection is missing this warning and should have it too

  • Add warning to UpdateCollection

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we remove them until they do update the GQL types?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont really think so, they will still work (on restart) and leaving them in documented like this should cause no harm IMO.

Can even be fixed in 0.5.1 if we want I think, as it wouldnt quite be a breaking change IMO

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've merged, but happy to remove these if that is what we decide upon


// UpdateCollection updates the persisted collection description matching the name of the given
Expand All @@ -71,6 +120,9 @@ type Store interface {
// The collection (including the schema version ID) will only be updated if any changes have actually
// been made, if the given description matches the current persisted description then no changes will be
// applied.
//
// WARNING: It does not currently update the GQL types, and as such a database restart is required after
// calling this if use of the new collection via GQL is desired (for example via [ExecRequest]).
UpdateCollection(context.Context, CollectionDescription) (Collection, error)

// ValidateUpdateCollection validates that the given collection description is a valid update.
Expand All @@ -79,20 +131,51 @@ type Store interface {
// collection. Will return an error if it fails validation.
ValidateUpdateCollection(context.Context, CollectionDescription) (bool, error)

// GetCollectionByName attempts to retrieve a collection matching the given name.
//
// If no matching collection is found an error will be returned.
GetCollectionByName(context.Context, string) (Collection, error)

// GetCollectionBySchemaID attempts to retrieve a collection matching the given schema ID.
//
// If no matching collection is found an error will be returned.
GetCollectionBySchemaID(context.Context, string) (Collection, error)

// GetCollectionBySchemaID attempts to retrieve a collection matching the given schema version ID.
//
// If no matching collection is found an error will be returned.
GetCollectionByVersionID(context.Context, string) (Collection, error)

// GetAllCollections returns all the collections and their descriptions that currently exist within
// this [Store].
GetAllCollections(context.Context) ([]Collection, error)

// ExecRequest executes the given GQL request against the [Store].
ExecRequest(context.Context, string) *RequestResult
}

// GQLResult represents the immediate results of a GQL request.
//
// It does not handle subscription channels. This object and its children are json serializable.
type GQLResult struct {
// Errors contains any errors generated whilst attempting to execute the request.
shahzadlone marked this conversation as resolved.
Show resolved Hide resolved
//
// If there are values in this slice the request will likely not have run to completion
// and [Data] will be nil.
Errors []any `json:"errors,omitempty"`
Data any `json:"data"`

// Data contains the resultant data produced by the GQL request.
//
// It will be nil if any errors were raised during execution.
Data any `json:"data"`
}

// RequestResult represents the results of a GQL request.
type RequestResult struct {
// GQL contains the immediate results of the GQL request.
GQL GQLResult

// Pub contains a pointer to an event stream which channels any subscription results
// if the request was a GQL subscription.
Pub *events.Publisher[events.Update]
}
41 changes: 0 additions & 41 deletions client/value.go
Original file line number Diff line number Diff line change
Expand Up @@ -109,44 +109,3 @@ func newCBORValue(t CType, val any) WriteableValue {
func (v cborValue) Bytes() ([]byte, error) {
return cbor.Marshal(v.value)
}

// func ReadCBORValue()

// func (val simpleValue) GetCRDT() crdt.MerkleCRDT {
// return val.crdt
// }

// func (val *simpleValue) SetCRDT(crdt crdt.MerkleCRDT) error {
// // if val.Type() != client.CType() {

// // } else {

// // }
// val.crdt = crdt
// return nil
// }

// type merkleCRDTValue struct {
// crdt crdt.MerkleCRDT
// }

// func newMerkleCRDTValue(dt crdt.MerkleCRDT) *merkleCRDTValue {
// return &merkleCRDTValue{
// crdt: dt
// }
// }

// type listValue struct {
// vals []Value
// }

// func (l *listValue) Value() any {
// return l.vals
// }
// func (l *listValue) IsDocument() bool { return false }
// func (l *listValue) Type() client.CType { }
// func (l *listValue) IsDirty() bool
// func (l *listValue) Clean()
// func (l *listValue) IsDelete() bool //todo: Update IsDelete naming
// func (l *listValue) Delete()
// func (l *listValue) Append(v Value)
14 changes: 10 additions & 4 deletions db/txn_db.go
Original file line number Diff line number Diff line change
Expand Up @@ -209,8 +209,11 @@ func (db *explicitTxnDB) GetAllCollections(ctx context.Context) ([]client.Collec
return db.getAllCollections(ctx, db.txn)
}

// AddSchema takes the provided schema in SDL format, and applies it to the database,
// and creates the necessary collections, request types, etc.
// AddSchema takes the provided GQL schema in SDL format, and applies it to the database,
// creating the necessary collections, request types, etc.
//
// All schema types provided must not exist prior to calling this, and they may not reference existing
// types previously defined.
func (db *implicitTxnDB) AddSchema(ctx context.Context, schemaString string) error {
txn, err := db.NewTxn(ctx, false)
if err != nil {
Expand All @@ -226,8 +229,11 @@ func (db *implicitTxnDB) AddSchema(ctx context.Context, schemaString string) err
return txn.Commit(ctx)
}

// AddSchema takes the provided schema in SDL format, and applies it to the database,
// and creates the necessary collections, request types, etc.
// AddSchema takes the provided GQL schema in SDL format, and applies it to the database,
// creating the necessary collections, request types, etc.
//
// All schema types provided must not exist prior to calling this, and they may not reference existing
// types previously defined.
func (db *explicitTxnDB) AddSchema(ctx context.Context, schemaString string) error {
return db.addSchema(ctx, db.txn, schemaString)
}
Expand Down