Tracking Issue for File lock API

[cberner]

Feature gate: #![feature(file_lock)]

This is a tracking issue for rust-lang/libs-team#412

This feature exposes advisory file locks on File. They allow a file handle to acquire an exclusive or shared file lock, which blocks other file handles to the same file from acquiring a conflicting lock. Some semantics are platform dependent, and these are documented in the API documentation.

Public API

impl File {
    fn lock(&self) -> io::Result<()>;
    fn lock_shared(&self) -> io::Result<()>;
    fn try_lock(&self) -> io::Result<bool>;
    fn try_lock_shared(&self) -> io::Result<bool>;
    fn unlock(&self) -> io::Result<()>;
}

Steps / History

• Implementation: Implement file_lock feature #130999
• Final comment period (FCP)¹
• Signature changed due to feedback: Change signature of File::try_lock and File::try_lock_shared #139343
• Signature changed again due to feedback: Change File::try_lock() and try_lock_shared() to return io::Result<()> #140718
• Simpler error propagation: Add From<TryLockError> for io::Error #141312
• Stabilization PR: Stabilize "file_lock" feature #142125

Unresolved Questions

• None yet.

Footnotes

1. https://std-dev-guide.rust-lang.org/feature-lifecycle/stabilization.html ↩

[workingjubilee]

apparently not supported on all tier 2 OS: #132921

[eric-seppanen]

Note that this triggers the unstable_name_collisions lint for code using the fs2/fs3/fs4 crates, all of which have a FileExt trait with methods called lock_shared, try_lock_shared, and unlock.

This lint fires on the 1.84 beta release, so there might be a number of people who discover this once 1.84 stable goes out.

[NilsIrl]

It would also be useful to atomically create and lock a file. This is possible on MacOS using the O_SHLOCK and O_EXLOCK flags on open and on Linux using O_TMPFILE.

[joshtriplett]

While there are more features we may want to add to this in the future, the current state of this seems useful, and works. Stabilizing it would let people who encounter the warnings about a future conflict switch to the new API.

Shall we stabilize the current File locking APIs?

@rfcbot merge

[rfcbot]

Team member @joshtriplett has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Amanieu
• @BurntSushi
• @dtolnay
• @joshtriplett
• @m-ou-se

Concerns:

• api-is-unergonomic resolved by Tracking Issue for File lock API #130994 (comment)

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[m-ou-se]

I'd like to see some documentation on the lock methods that make it clear that you don't need to manually unlock / that the lock is automatically unlocked when the File is dropped. Right now, that's only documented on the unlock method.

Other than that, the documentation could be made a lot less confusing by adding 'by another process' and 'by the same process' in a few places. Right now, the try methods say "Returns false if the file is locked.", but then go on to say it might deadlock if it's already locked. I assume the former should be "locked by another process" and the latter should be "locked by this process".

[m-ou-se]

The unlock method should document whether it's okay to call it if no locks are held.

If calling unlock() is only acceptable when a lock is actually held, this should probably be a Guard style of API. If it's always okay to call unlock(), the current design makes sense to me.

[m-ou-se]

I'm checking my box with the assumption that these are just small docs changes that we'll do during/before stabilization.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[joshtriplett]

I've submitted #136288 which should address all the documentation requests.

@m-ou-se wrote:

I'd like to see some documentation on the lock methods that make it clear that you don't need to manually unlock / that the lock is automatically unlocked when the File is dropped. Right now, that's only documented on the unlock method.

Done.

Other than that, the documentation could be made a lot less confusing by adding 'by another process' and 'by the same process' in a few places. Right now, the try methods say "Returns false if the file is locked.", but then go on to say it might deadlock if it's already locked. I assume the former should be "locked by another process" and the latter should be "locked by this process".

"process" isn't the granularity here, but I've added clear distinctions about locks acquired via the same handle/descriptor (may deadlock) vs locks acquired via a different handle/descriptor (will block the blocking methods or make the try_ methods return Ok(false)). I've also clarified the return value documentation to avoid that ambiguity.

The unlock method should document whether it's okay to call it if no locks are held.

It's always safe to call (in the Rust sense). I've documented that it'll either return an error or return without doing anything. It'll never explode.

If calling unlock() is only acceptable when a lock is actually held, this should probably be a Guard style of API. If it's always okay to call unlock(), the current design makes sense to me.

It's always OK (unlike a mutex or similar). It'll never explode.

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.

[joshtriplett]

@BurntSushi I no longer think those cases are worth distinguishing myself, as the error case largely can't happen. For instance, you can't normally get EBADF because the file is known valid.

[BurntSushi]

I'm not sure that changes my opinion here unless there is a guarantee that other errors can't happen. For me personally, if I were using this API, I would want to treat the two cases separately, if only to use different error messages.

[joshtriplett]

@BurntSushi No objection, if you feel that you would actually use that.

[cberner]

I have a slight preference for keeping TryLockError::WouldBlock, but I don't have a strong opinion either way. I'll put together a PR to add the From impl for the current API, but lemme know if you all decide you'd rather go with returning io::Error

Here's an example of how I use the current API, and you can see that I already handle multiple special error cases:

• lock not acquired because it's already held by another process
• lock not acquired because filesystem does not support locking
• lock not acquired due to some other I/O error

[Amanieu]

I personally have a very slight preference towards keeping TryLockError but would be happy with either choice. Given the shape of the discussion in the last few comments, I think we should just merge #141312 and close #140718.

[joshtriplett]

@rfcbot resolved api-is-unergonomic

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[fishface60]

Having banged my head against the Java file lock APIs recently I'm glad Rust hasn't decided to use the fcntl F_SETLK interfaces just because they include file range locking. It's been a lot of work to work around the limitations caused by that.

I like that this implementation documents what the platform-specific behaviours are by platform. It was irritating that the Java implementation documented that "some platforms may convert a request for a shared lock into an exclusive lock" and then I could find no trace of which one.

To its credit however, it gave recommendations for how to use the interfaces in a way that is safe cross-platform. I'm not sure how obvious it is from the Rust documentation that you should treat all locks as advisory and open files for write if you take an exclusive lock or read for a shared lock.

I think it may be important to document the platform-specific lock inheritance behaviour. It wasn't hugely relevant for Java since all implementations' locks were per-process, but with flock the lock will be inherited if the file descriptor is and with LockFileEx subprocesses don't inherit the lock.

[cberner]

@fishface60

I'm not sure how obvious it is from the Rust documentation that you should treat all locks as advisory

I tried to make this clear in this paragraph of the docs, but feel free to suggest improvements!:

    /// This lock may be advisory or mandatory. This lock is meant to interact with [`lock`],
    /// [`try_lock`], [`lock_shared`], [`try_lock_shared`], and [`unlock`]. Its interactions with
    /// other methods, such as [`read`] and [`write`] are platform specific, and it may or may not
    /// cause non-lockholders to block.

open files for write if you take an exclusive lock or read for a shared lock

As far as I can tell that is not required. The tests pass on Linux and Windows even if I change them to use just read permission when acquiring an exclusive lock, and the Windows documentation says: The handle must have been created with either the GENERIC_READ or GENERIC_WRITE access right.

[cberner]

@joshtriplett it has been 10 days since the FCP started, but it seems like @rfcbot is stuck. Maybe due to this issue

Alright if I submit the stabilization PR?

[cberner]

@m-ou-se it looks like you fixed this rfcbot issue the last time it happened. This FCP should have completed a week ago. Could you help get the FCP unstuck?

[ChrisDenton]

I'm not rustbot but I can do my best impression...

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.