Use SQLite instead of bbolt #779
Conversation
adombeck
force-pushed
the
UDENG-4890-use-sqlite
branch
from
763b8ed to
99bfaab
Compare
| // TODO: I don't see why we should fail here instead of just fixing the permissions. | ||
| fileInfo, err := os.Stat(dbPath) | ||
| if err != nil { | ||
| return nil, fmt.Errorf("can't stat database file: %v", err) | ||
| } | ||
| perm := fileInfo.Mode().Perm() | ||
| if perm != 0600 { | ||
| return nil, fmt.Errorf("wrong file permission for %s: %o", dbPath, perm) | ||
| } |
There was a problem hiding this comment.
To the reviewer: I copied this from the old code but added this TODO comment:
// TODO: I don't see why we should fail here instead of just fixing the permissions.
Is there a good reason for failing or should we just fix the permissions?
adombeck
force-pushed
the
UDENG-4890-use-sqlite
branch
from
99bfaab to
79ce337
Compare
adombeck
force-pushed
the
UDENG-4890-use-sqlite
branch
from
79ce337 to
7f0d869
Compare
Changes done by running: git rm "internal/*/testdata/golden/*" TESTS_UPDATE_GOLDEN=1 go test ./internal/... git add -A
We don't use our database as a cache anymore. It's time to reflect this in the package name.
We can't use the "AAAAATIME" placeholders anymore when using SQLite, because the schema only allows actual timestamps.
We don't store the current UID in golden files anymore since 3f81332.
These files were forgotten in 5555836. Also simplify the testdata for the "pam_unix_non_existent" test case, because we don't need multiple users and groups for that test case.
adombeck
force-pushed
the
UDENG-4890-use-sqlite
branch
from
7f0d869 to
143ef36
Compare
Our database requirements are better met by a relational database than a key value store, so we now use SQLite instead of bbolt.
Some cases were not tested.
And inline the `userRow` struct into `UserDB`, because it now doesn't have any additional fields anymore.
adombeck
force-pushed
the
UDENG-4890-use-sqlite
branch
from
143ef36 to
1c7b268
Compare
|
|
||
| func allUsersInternal(db queryable) ([]userRow, error) { | ||
| query := fmt.Sprintf(`SELECT %s FROM users`, allUserColumns) | ||
| rows, err := db.Query(query) |
There was a problem hiding this comment.
As per what I've been reading, these calls shouldn't be happen concurrently, so maybe we should have a wrapper struct that manages the db locking via a mutex? So that we don't risk of accessing to it when it's locked? Or are we using another way to prevent this to happen?
There was a problem hiding this comment.
That's taken care of by the use of transactions. All functions which write to the database use transactions, which lock the database.
| log.Debugf(context.Background(), "Inserting into %s: %s", table, vals) | ||
|
|
||
| //nolint:gosec // We don't care about SQL injection in our tests. | ||
| query := fmt.Sprintf("INSERT INTO %s (%s) VALUES (%s)", table, columns, values) |
There was a problem hiding this comment.
Is there any potential string escaping that we may miss here?
There was a problem hiding this comment.
None that I know of. Is it worth spending time trying to find one? I don't see why we should care about the SQL injection warning here.
internal/users/db/testutils.go
Outdated
| for _, record := range records { | ||
| columns := "" | ||
| values := "" | ||
| var vals []interface{} |
internal/users/db/usergroups.go
Outdated
| defer func() { | ||
| if err := rows.Close(); err != nil { | ||
| log.Warningf(context.Background(), "failed to close rows: %v", err) | ||
| } | ||
| }() |
There was a problem hiding this comment.
We've enough of these calls that maybe they can be in a separated function to be just called with defer closeRows(rows)?
There was a problem hiding this comment.
On:
Migrate all testdata to SQLite
It would have been nice if the json-like to yaml format for the db files would have happened before of switching to sqlite, so that it's easier to track if something changed.
Is it possible (e.g. to use the newer syntax for the golden files before changing the db to sqlite)?
There was a problem hiding this comment.
No, that's not possible. It's not the format which has changed (it was YAML and still is YAML), it's the structure of the database which has changed. Before, we had 7 different buckets, now we have 3 tables.
Our database requirements are better met by a relational database than a key value store, so we now use SQLite instead of bbolt.
UDENG-4890