sdk/rust: Fix overlay lookup using wrong delta parent inode#300
Merged
Conversation
When looking up an entry whose parent is in the Base layer, the overlay walks the delta to find the corresponding delta parent directory. If any component is missing in the delta, the walk broke early and used whatever intermediate inode it had reached (often the delta root). This caused delta.lookup() to search under the wrong parent, potentially returning an unrelated entry that shadows the correct base directory. This manifested as ENOTDIR when cargo tried to resolve path dependencies like ../sdk/rust/Cargo.toml — if an unrelated file with a matching name existed under the delta root, lookup returned it as a regular file instead of the base directory, causing the kernel to reject further path traversal. Fix both lookup() and ensure_parent_dirs(): - lookup(): skip the delta lookup entirely when the path walk doesn't resolve all components (use Option<i64> instead of bare i64). - ensure_parent_dirs(): track base and delta parent inodes separately so the base lookup uses the correct base inode, not a delta inode.
broady
added a commit
to broady/agentfs
that referenced
this pull request
Feb 9, 2026
…directories
When a base-layer file is unlinked or renamed inside a directory that
has been promoted from Base to Delta (via ensure_parent_dirs), the old
code walked the base HostFS to decide whether to create a whiteout.
For promoted directories the base inode mapping was already removed by
promote_to_delta(), so the walk silently returned early — skipping
whiteout creation and leaving the base file visible through the overlay.
This caused dpkg to fail with EEXIST during package installs: unlink
of /var/lib/dpkg/status-old succeeded in delta but never created a
whiteout, so the subsequent link("status", "status-old") found the
base-layer file and returned EEXIST.
Fix by:
1. Extracting resolve_delta_parent() that walks the delta layer by
path to find the parent inode, replacing 5 inline walk patterns
across lookup, unlink, rmdir, and rename. The walk validates that
each path component is a directory to avoid shadowing by
non-directory entries (same class of bug fixed in tursodatabase#300).
2. Changing unlink/rmdir to remove from delta first, then use
self.lookup() to detect whether the file is still visible — if so,
it must be from the base layer and needs a whiteout.
3. Moving delta parent resolution in rename() after copy_up() and
ensure_parent_dirs() so the delta directories exist before we try
to resolve them.
Add 8 unit tests and a shell integration test reproducing the dpkg
EEXIST scenario.
broady
added a commit
to broady/agentfs
that referenced
this pull request
Feb 9, 2026
When a base-layer file is unlinked or renamed inside a directory that
has been promoted from Base to Delta (via ensure_parent_dirs), the
old code walked the base HostFS to decide whether to create a
whiteout. For promoted directories the base inode mapping was already
removed by promote_to_delta(), so the walk silently returned early
-- skipping whiteout creation and leaving the base file visible.
This caused dpkg to fail with EEXIST during package installs:
unlink of /var/lib/dpkg/status-old succeeded in delta but never
created a whiteout, so the subsequent link("status", "status-old")
found the base-layer file and returned EEXIST.
Fix by:
1. Extracting resolve_delta_parent() that walks the delta layer
by path to find the parent inode, replacing 5 inline walk
patterns across lookup, unlink, rmdir, and rename. The walk
validates that each component is a directory to avoid
shadowing by non-directory entries (same class as tursodatabase#300).
2. Changing unlink/rmdir to remove from delta first, then use
self.lookup() to detect whether the file is still visible
-- if so, it must be from base and needs a whiteout.
3. Moving delta parent resolution in rename() after copy_up()
and ensure_parent_dirs() so the delta directories exist
before we try to resolve them.
Add 8 unit tests and a shell integration test reproducing the
dpkg EEXIST scenario.
broady
added a commit
to broady/agentfs
that referenced
this pull request
Feb 9, 2026
When a base-layer file is unlinked or renamed inside a directory
that has been promoted from Base to Delta (via ensure_parent_dirs),
the old code walked the base HostFS to decide whether to create a
whiteout. For promoted directories the base inode mapping was
already removed by promote_to_delta(), so the walk silently
returned early -- skipping whiteout creation and leaving the base
file visible through the overlay.
This caused dpkg to fail with EEXIST during package installs:
unlink of /var/lib/dpkg/status-old succeeded in delta but never
created a whiteout, so the subsequent link("status", "status-old")
found the base-layer file and returned EEXIST.
Fix by:
1. Extracting resolve_delta_parent() that walks the delta layer
by path to find the parent inode, replacing 5 inline walk
patterns across lookup, unlink, rmdir, and rename. The walk
validates that each component is a directory to avoid
shadowing by non-directory entries (same class as tursodatabase#300).
2. Changing unlink/rmdir to remove from delta first, then use
self.lookup() to detect whether the file is still visible
-- if so, it must be from base and needs a whiteout.
3. Moving delta parent resolution in rename() after copy_up()
and ensure_parent_dirs() so the delta directories exist
before we try to resolve them.
Add 8 unit tests and extend the existing shell integration test
with base-file whiteout coverage.
broady
added a commit
to broady/agentfs
that referenced
this pull request
Feb 9, 2026
When a base-layer file is unlinked or renamed inside a directory
that has been promoted from Base to Delta (via ensure_parent_dirs),
the old code walked the base HostFS to decide whether to create a
whiteout. For promoted directories the base inode mapping was
already removed by promote_to_delta(), so the walk silently
returned early -- skipping whiteout creation and leaving the base
file visible through the overlay.
This caused dpkg to fail with EEXIST during package installs:
unlink of /var/lib/dpkg/status-old succeeded in delta but never
created a whiteout, so the subsequent link("status", "status-old")
found the base-layer file and returned EEXIST.
Fix by:
1. Extracting resolve_delta_parent() that walks the delta layer
by path to find the parent inode, replacing 5 inline walk
patterns across lookup, unlink, rmdir, and rename. The walk
validates that each component is a directory to avoid
shadowing by non-directory entries (same class as tursodatabase#300).
2. Changing unlink/rmdir to remove from delta first, then use
self.lookup() to detect whether the file is still visible
-- if so, it must be from base and needs a whiteout.
3. Moving delta parent resolution in rename() after copy_up()
and ensure_parent_dirs() so the delta directories exist
before we try to resolve them.
Add 8 unit tests and extend the existing shell integration test
with base-file whiteout coverage.
Amp-Thread-ID: https://ampcode.com/threads/T-019c404d-2623-70b7-b5d8-2dfa38f4d21f
Co-authored-by: Amp <amp@ampcode.com>
broady
added a commit
to broady/agentfs
that referenced
this pull request
Feb 9, 2026
When a base-layer file is unlinked or renamed inside a directory
that has been promoted from Base to Delta (via ensure_parent_dirs),
the old code walked the base HostFS to decide whether to create a
whiteout. For promoted directories the base inode mapping was
already removed by promote_to_delta(), so the walk silently
returned early -- skipping whiteout creation and leaving the base
file visible through the overlay.
This caused dpkg to fail with EEXIST during package installs:
unlink of /var/lib/dpkg/status-old succeeded in delta but never
created a whiteout, so the subsequent link("status", "status-old")
found the base-layer file and returned EEXIST.
Fix by:
1. Extracting resolve_delta_parent() that walks the delta layer
by path to find the parent inode, replacing 9 inline walk
patterns across lookup, mkdir, create_file, mknod, symlink,
unlink, rmdir, link, and rename. The walk validates that each
component is a directory to avoid shadowing by non-directory
entries (same class as tursodatabase#300).
2. Changing unlink/rmdir to remove from delta first, then use
self.lookup() to detect whether the file is still visible
-- if so, it must be from base and needs a whiteout.
3. Moving delta parent resolution in rename() after copy_up()
and ensure_parent_dirs() so the delta directories exist
before we try to resolve them.
Add 8 unit tests and extend the existing shell integration test
with base-file whiteout coverage.
Amp-Thread-ID: https://ampcode.com/threads/T-019c404d-2623-70b7-b5d8-2dfa38f4d21f
Co-authored-by: Amp <amp@ampcode.com>
broady
added a commit
to broady/agentfs
that referenced
this pull request
Feb 9, 2026
When a base-layer file is unlinked or renamed inside a directory
that has been promoted from Base to Delta (via ensure_parent_dirs),
the old code walked the base HostFS to decide whether to create a
whiteout. For promoted directories the base inode mapping was
already removed by promote_to_delta(), so the walk silently
returned early -- skipping whiteout creation and leaving the base
file visible through the overlay.
This caused dpkg to fail with EEXIST during package installs:
unlink of /var/lib/dpkg/status-old succeeded in delta but never
created a whiteout, so the subsequent link("status", "status-old")
found the base-layer file and returned EEXIST.
Fix by:
1. Extracting resolve_delta_parent() that walks the delta layer
by path to find the parent inode, replacing 9 inline walk
patterns across lookup, mkdir, create_file, mknod, symlink,
unlink, rmdir, link, and rename. The walk validates that each
component is a directory to avoid shadowing by non-directory
entries (same class as tursodatabase#300).
2. Changing unlink/rmdir to remove from delta first, then use
self.lookup() to detect whether the file is still visible
-- if so, it must be from base and needs a whiteout.
3. Moving delta parent resolution in rename() after copy_up()
and ensure_parent_dirs() so the delta directories exist
before we try to resolve them.
Add 8 unit tests and extend the existing shell integration test
with base-file whiteout coverage.
broady
added a commit
to broady/agentfs
that referenced
this pull request
Feb 9, 2026
When a base-layer file is unlinked or renamed inside a directory
that has been promoted from Base to Delta (via ensure_parent_dirs),
the old code walked the base HostFS to decide whether to create a
whiteout. For promoted directories the base inode mapping was
already removed by promote_to_delta(), so the walk silently
returned early -- skipping whiteout creation and leaving the base
file visible through the overlay.
This caused dpkg to fail with EEXIST during package installs:
unlink of /var/lib/dpkg/status-old succeeded in delta but never
created a whiteout, so the subsequent link("status", "status-old")
found the base-layer file and returned EEXIST.
Fix by:
1. Extracting resolve_delta_parent() that walks the delta layer
by path to find the parent inode, replacing 9 inline walk
patterns across lookup, mkdir, create_file, mknod, symlink,
unlink, rmdir, link, and rename. The walk validates that each
component is a directory to avoid shadowing by non-directory
entries (same class as tursodatabase#300).
2. Changing unlink/rmdir to remove from delta first, then use
self.lookup() to detect whether the file is still visible
-- if so, it must be from base and needs a whiteout.
3. Moving delta parent resolution in rename() after copy_up()
and ensure_parent_dirs() so the delta directories exist
before we try to resolve them.
Add 8 unit tests and extend the existing shell integration test
with base-file whiteout coverage.
broady
added a commit
to broady/agentfs
that referenced
this pull request
Feb 9, 2026
When a base-layer file is unlinked or renamed inside a directory
that has been promoted from Base to Delta (via ensure_parent_dirs),
the old code walked the base HostFS to decide whether to create a
whiteout. For promoted directories the base inode mapping was
already removed by promote_to_delta(), so the walk silently
returned early -- skipping whiteout creation and leaving the base
file visible through the overlay.
This caused dpkg to fail with EEXIST during package installs:
unlink of /var/lib/dpkg/status-old succeeded in delta but never
created a whiteout, so the subsequent link("status", "status-old")
found the base-layer file and returned EEXIST.
Fix by:
1. Extracting resolve_delta_parent() that walks the delta layer
by path to find the parent inode, replacing 9 inline walk
patterns across lookup, mkdir, create_file, mknod, symlink,
unlink, rmdir, link, and rename. The walk validates that each
component is a directory to avoid shadowing by non-directory
entries (same class as tursodatabase#300).
2. Changing unlink/rmdir to remove from delta first, then use
self.lookup() to detect whether the file is still visible
-- if so, it must be from base and needs a whiteout.
3. Moving delta parent resolution in rename() after copy_up()
and ensure_parent_dirs() so the delta directories exist
before we try to resolve them.
Add 8 unit tests and extend the existing shell integration test
with base-file whiteout coverage.
broady
added a commit
to broady/agentfs
that referenced
this pull request
Feb 13, 2026
When a base-layer file is unlinked or renamed inside a directory
that has been promoted from Base to Delta (via ensure_parent_dirs),
the old code walked the base HostFS to decide whether to create a
whiteout. For promoted directories the base inode mapping was
already removed by promote_to_delta(), so the walk silently
returned early -- skipping whiteout creation and leaving the base
file visible through the overlay.
This caused dpkg to fail with EEXIST during package installs:
unlink of /var/lib/dpkg/status-old succeeded in delta but never
created a whiteout, so the subsequent link("status", "status-old")
found the base-layer file and returned EEXIST.
Fix by:
1. Extracting resolve_delta_parent() that walks the delta layer
by path to find the parent inode, replacing 9 inline walk
patterns across lookup, mkdir, create_file, mknod, symlink,
unlink, rmdir, link, and rename. The walk validates that each
component is a directory to avoid shadowing by non-directory
entries (same class as tursodatabase#300).
2. Changing unlink/rmdir to remove from delta first, then use
self.lookup() to detect whether the file is still visible
-- if so, it must be from base and needs a whiteout.
3. Moving delta parent resolution in rename() after copy_up()
and ensure_parent_dirs() so the delta directories exist
before we try to resolve them.
Add 8 unit tests and extend the existing shell integration test
with base-file whiteout coverage.
broady
added a commit
to broady/agentfs
that referenced
this pull request
Feb 14, 2026
When a base-layer file is unlinked or renamed inside a directory
that has been promoted from Base to Delta (via ensure_parent_dirs),
the old code walked the base HostFS to decide whether to create a
whiteout. For promoted directories the base inode mapping was
already removed by promote_to_delta(), so the walk silently
returned early -- skipping whiteout creation and leaving the base
file visible through the overlay.
This caused dpkg to fail with EEXIST during package installs:
unlink of /var/lib/dpkg/status-old succeeded in delta but never
created a whiteout, so the subsequent link("status", "status-old")
found the base-layer file and returned EEXIST.
Fix by:
1. Extracting resolve_delta_parent() that walks the delta layer
by path to find the parent inode, replacing 9 inline walk
patterns across lookup, mkdir, create_file, mknod, symlink,
unlink, rmdir, link, and rename. The walk validates that each
component is a directory to avoid shadowing by non-directory
entries (same class as tursodatabase#300).
2. Changing unlink/rmdir to remove from delta first, then use
self.lookup() to detect whether the file is still visible
-- if so, it must be from base and needs a whiteout.
3. Moving delta parent resolution in rename() after copy_up()
and ensure_parent_dirs() so the delta directories exist
before we try to resolve them.
Add 8 unit tests and extend the existing shell integration test
with base-file whiteout coverage.
broady
added a commit
to broady/agentfs
that referenced
this pull request
Feb 15, 2026
When a base-layer file is unlinked or renamed inside a directory
that has been promoted from Base to Delta (via ensure_parent_dirs),
the old code walked the base HostFS to decide whether to create a
whiteout. For promoted directories the base inode mapping was
already removed by promote_to_delta(), so the walk silently
returned early -- skipping whiteout creation and leaving the base
file visible through the overlay.
This caused dpkg to fail with EEXIST during package installs:
unlink of /var/lib/dpkg/status-old succeeded in delta but never
created a whiteout, so the subsequent link("status", "status-old")
found the base-layer file and returned EEXIST.
Fix by:
1. Extracting resolve_delta_parent() that walks the delta layer
by path to find the parent inode, replacing 9 inline walk
patterns across lookup, mkdir, create_file, mknod, symlink,
unlink, rmdir, link, and rename. The walk validates that each
component is a directory to avoid shadowing by non-directory
entries (same class as tursodatabase#300).
2. Changing unlink/rmdir to remove from delta first, then use
self.lookup() to detect whether the file is still visible
-- if so, it must be from base and needs a whiteout.
3. Moving delta parent resolution in rename() after copy_up()
and ensure_parent_dirs() so the delta directories exist
before we try to resolve them.
Add 8 unit tests and extend the existing shell integration test
with base-file whiteout coverage.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When looking up an entry whose parent is in the Base layer, the overlay walks the delta to find the corresponding delta parent directory. If any component is missing in the delta, the walk broke early and used whatever intermediate inode it had reached (often the delta root). This caused delta.lookup() to search under the wrong parent, potentially returning an unrelated entry that shadows the correct base directory.
This manifested as ENOTDIR when cargo tried to resolve path dependencies like ../sdk/rust/Cargo.toml — if an unrelated file with a matching name existed under the delta root, lookup returned it as a regular file instead of the base directory, causing the kernel to reject further path traversal.
Fix both lookup() and ensure_parent_dirs():
lookup(): skip the delta lookup entirely when the path walk doesn't resolve all components (use Option instead of bare i64).
ensure_parent_dirs(): track base and delta parent inodes separately so the base lookup uses the correct base inode, not a delta inode.