Exporter: detect & handle deleted workspace objects (notebooks/files/directories) when running in incremental mode #3225
Conversation
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #3225 +/- ##
==========================================
+ Coverage 83.60% 83.77% +0.17%
==========================================
Files 169 170 +1
Lines 15094 15301 +207
==========================================
+ Hits 12619 12819 +200
+ Misses 1735 1731 -4
- Partials 740 751 +11
|
alexott
changed the title
exporter/context.go
Outdated
| @@ -499,6 +529,110 @@ func (ic *importContext) Run() error { | |||
| return nil | |||
| } | |||
|
|
|||
| func isSupportedWsObject(obj workspace.ObjectStatus) bool { | |||
| return obj.ObjectType == workspace.Directory || obj.ObjectType == workspace.Notebook || obj.ObjectType == workspace.File | |||
There was a problem hiding this comment.
Having a list of supported objects and then checking in that would be better, we could extend it to other objects in future.
There was a problem hiding this comment.
Reimplemented as switch, but really we can detect changes only in specific resources...
| case workspace.Notebook: | ||
| rtype = "databricks_notebook" | ||
| default: | ||
| log.Printf("[WARN] Unsupported WS object type: %s in obj %v", obj.ObjectType, obj) |
There was a problem hiding this comment.
it's too late to fail - it's better to don't some missing code that could be added manually, than fail multi-hour work. Although, this block is just defensive piece - because we have filters earlier we don't reach it.
| p := ic.oldWorkspaceObjectMapping[object.ObjectID] | ||
| if p == "" || p == object.Path { | ||
| log.Printf("[DEBUG] skipping '%s' that was modified at %d (last active=%d)", | ||
| object.Path, modifiedAt, updatedSinceMs) |
There was a problem hiding this comment.
is last active same as last updated?
There was a problem hiding this comment.
No. Last Active - when the exporter ran last time...
|
Most of the tests in |
Changes
The initial version of code for detection & handling of removed workspace objects (notebooks, workspace files, and directories) and associated permissions... Renames are handled as (delete/create new)...
Tests
make testrun locallydocs/folderinternal/acceptance