Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

945 sstable interface #969

Merged
merged 31 commits into from
Nov 29, 2020
Merged
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
b43ebf8
SSTable interface for committed data
itaiad200 Nov 22, 2020
703c4b6
Remove unnecessary funcs from iterator
itaiad200 Nov 22, 2020
84bd269
added test for merge with zero changes
tzahij Nov 22, 2020
0a4d19b
initial commit of tree
tzahij Nov 22, 2020
d2fa10a
Change iterator to match the existing scanners interfaces
itaiad200 Nov 22, 2020
1ed15b2
Merge branch '945-sstable-tree-interfaces' of https://github.com/tree…
itaiad200 Nov 22, 2020
2bce373
initial commit of tree
tzahij Nov 22, 2020
71ac20f
initial commit of tree
tzahij Nov 22, 2020
8550a6d
initial commit for rocks catalog interface
guy-har Nov 23, 2020
d32c46c
sstable initial implementation without repo seperation
itaiad200 Nov 23, 2020
3b721d0
Merge remote-tracking branch 'origin/master' into 945-sstable-tree-in…
itaiad200 Nov 23, 2020
8931903
initial catalog interface
guy-har Nov 24, 2020
99bdab8
Merge branch 'master' into feature/rocks-catalog-interface
guy-har Nov 24, 2020
acca7d9
initial catalog interface
guy-har Nov 24, 2020
f711b30
Merge branch 'master' into feature/rocks-catalog-interface
guy-har Nov 24, 2020
9feb2fd
initial catalog interface
guy-har Nov 25, 2020
9505288
Update catalog/rocks/catalog.go
guy-har Nov 25, 2020
8eec272
iterator interface
nopcoder Nov 25, 2020
864c5c8
diff type comment
nopcoder Nov 25, 2020
19bfe19
Merge branch 'feature/rocks-catalog-interface' of github.com:treevers…
nopcoder Nov 25, 2020
0834c13
continue talking about StagingManager
nopcoder Nov 25, 2020
a5cad17
initial commit of tree
tzahij Nov 25, 2020
238838a
Merge branch 'master' into 945-sstable-tree-interfaces
tzahij Nov 26, 2020
3162279
Merge branch 'feature/rocks-catalog-interface' into 945-sstable-tree-…
tzahij Nov 26, 2020
6da61db
just save changes - initial
tzahij Nov 26, 2020
36df291
working on apply
tzahij Nov 27, 2020
9768b34
working on apply
tzahij Nov 28, 2020
d30d1d5
working on apply
tzahij Nov 28, 2020
8ff22f1
Merge remote-tracking branch 'origin/master' into 945-sstable-interface
itaiad200 Nov 29, 2020
e486488
sstable interface
itaiad200 Nov 29, 2020
ab24d5b
changes
itaiad200 Nov 29, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions forest/sstable/sstable.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
package sstable

import (
"github.com/treeverse/lakefs/catalog/rocks"
)

// SSTableID is an identifier for an SSTable
type SSTableID string

type Manager interface {
// GetEntry returns the entry matching the path in the SSTable referenced by the id.
// If path not found, (nil, ErrPathNotFound) is returned.
GetEntry(path rocks.Path, tid SSTableID) (*rocks.Entry, error)

// NewSSTableIterator takes a given SSTable and returns an EntryIterator seeked to >= "from" path
NewSSTableIterator(tid SSTableID, from rocks.Path) (rocks.EntryIterator, error)

// GetWriter returns a new SSTable writer instance
GetWriter() (Writer, error)
}

// WriteResult is the result of a completed write of an SSTable
type WriteResult struct {
// SSTableID is the identifier for the written SSTable.
// Calculated by an hash function to all paths and entries.
SSTableID SSTableID

// First is the Path of the first entry in the SSTable.
First rocks.Path

// Last is the Path of the last entry in the SSTable.
Last rocks.Path

// Count is the number of entries in the SSTable.
Count int
}

// Writer is an abstraction for writing SSTables.
// Written entries must be sorted by path.
type Writer interface {
// WriteEntry appends the given entry to the SSTable
WriteEntry(path rocks.Path, entry rocks.Entry) error

// Close flushes all entries to the disk and returns the WriteResult.
Close() (*WriteResult, error)
}