Tracking Issue for File lock API

[cberner]

Feature gate: #![feature(file_lock)]

This is a tracking issue for rust-lang/libs-team#412

This feature exposes advisory file locks on File. They allow a file handle to acquire an exclusive or shared file lock, which blocks other file handles to the same file from acquiring a conflicting lock. Some semantics are platform dependent, and these are documented in the API documentation.

Public API

impl File {
    fn lock(&self) -> io::Result<()>;
    fn lock_shared(&self) -> io::Result<()>;
    fn try_lock(&self) -> io::Result<bool>;
    fn try_lock_shared(&self) -> io::Result<bool>;
    fn unlock(&self) -> io::Result<()>;
}

Steps / History

• Implementation: Implement file_lock feature #130999
• Final comment period (FCP)¹
• Stabilization PR
  • Stabilize file_lock #136794

Unresolved Questions

• None yet.

Footnotes

1. https://std-dev-guide.rust-lang.org/feature-lifecycle/stabilization.html ↩

[workingjubilee]

apparently not supported on all tier 2 OS: #132921

[eric-seppanen]

Note that this triggers the unstable_name_collisions lint for code using the fs2/fs3/fs4 crates, all of which have a FileExt trait with methods called lock_shared, try_lock_shared, and unlock.

This lint fires on the 1.84 beta release, so there might be a number of people who discover this once 1.84 stable goes out.

[NilsIrl]

It would also be useful to atomically create and lock a file. This is possible on MacOS using the O_SHLOCK and O_EXLOCK flags on open and on Linux using O_TMPFILE.

[joshtriplett]

While there are more features we may want to add to this in the future, the current state of this seems useful, and works. Stabilizing it would let people who encounter the warnings about a future conflict switch to the new API.

Shall we stabilize the current File locking APIs?

@rfcbot merge

[rfcbot]

Team member @joshtriplett has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Amanieu
• @BurntSushi
• @dtolnay
• @joshtriplett
• @m-ou-se

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[m-ou-se]

I'd like to see some documentation on the lock methods that make it clear that you don't need to manually unlock / that the lock is automatically unlocked when the File is dropped. Right now, that's only documented on the unlock method.

Other than that, the documentation could be made a lot less confusing by adding 'by another process' and 'by the same process' in a few places. Right now, the try methods say "Returns false if the file is locked.", but then go on to say it might deadlock if it's already locked. I assume the former should be "locked by another process" and the latter should be "locked by this process".

[m-ou-se]

The unlock method should document whether it's okay to call it if no locks are held.

If calling unlock() is only acceptable when a lock is actually held, this should probably be a Guard style of API. If it's always okay to call unlock(), the current design makes sense to me.

[m-ou-se]

I'm checking my box with the assumption that these are just small docs changes that we'll do during/before stabilization.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[joshtriplett]

I've submitted #136288 which should address all the documentation requests.

@m-ou-se wrote:

I'd like to see some documentation on the lock methods that make it clear that you don't need to manually unlock / that the lock is automatically unlocked when the File is dropped. Right now, that's only documented on the unlock method.

Done.

Other than that, the documentation could be made a lot less confusing by adding 'by another process' and 'by the same process' in a few places. Right now, the try methods say "Returns false if the file is locked.", but then go on to say it might deadlock if it's already locked. I assume the former should be "locked by another process" and the latter should be "locked by this process".

"process" isn't the granularity here, but I've added clear distinctions about locks acquired via the same handle/descriptor (may deadlock) vs locks acquired via a different handle/descriptor (will block the blocking methods or make the try_ methods return Ok(false)). I've also clarified the return value documentation to avoid that ambiguity.

The unlock method should document whether it's okay to call it if no locks are held.

It's always safe to call (in the Rust sense). I've documented that it'll either return an error or return without doing anything. It'll never explode.

If calling unlock() is only acceptable when a lock is actually held, this should probably be a Guard style of API. If it's always okay to call unlock(), the current design makes sense to me.

It's always OK (unlike a mutex or similar). It'll never explode.

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.

[dpc]

Hello. I have not immersed myself myself in the topic so far, but just landed here after debugging session caused by Result<bool> being an error prone API.

cargo-bins/cargo-binstall#2091

This was in fs4 library, but I've noticed this that there's is actually unstable locking API in std and was surprised, it has the same signature.

If you look at the issues we're 3rd project reporting a bug w.r.t to this API - 2 were already opened, I didn't bother opening one for us and just commented on the existing one. It might be magnified by the fact that previously this call returned Result<()> so people more mindlessly might have swapped it for _, but still a sign that this API will definitely cause mistakes.

If the goal of the API is to get a lock, not getting a lock should be an Err, even if we're just "trying".

• https://doc.rust-lang.org/std/sync/struct.Mutex.html#method.try_lock
• https://docs.rs/tokio/latest/tokio/sync/struct.Mutex.html#method.try_lock
• https://doc.rust-lang.org/std/sync/mpmc/struct.Sender.html#method.try_send

None of these returns a bool flag, so I think we're kind of conditioned not to use bools to denote failure. Especially try_send seems comparable, because the goal of the try_ is to ... avoid blocking.

This could be either Result<(), TryLockError> or even a nested Result. I don't really care about ergonomics of a call that is so rarely used it wasn't in stdlib for 10 years, but I do care about locking not accidentally being incorrect in my own projects and whole Rust ecosystems.

[joshtriplett]

Making it just a Result would make it difficult to distinguish between failure to obtain the lock and another OS error. Code using try_lock may want to print a message and then try again blocking, for a failure to acquire the lock, but for an OS error they'll want to error out and exit.

Nested Result makes that case easy. Result of bool makes that case easy, though potentially error-prone until we have a way to mark the bool as must_use.

[BurntSushi]

I think Result<(), TryLockError> is probably the right call here too.

A nested Result is weird IMO. I'm not above having used them before, but never in public APIs (I believe).

Having to do a bit of matching on the error type to distinguish the error cases for something like this seems very okay to me. Especially given that this API is probably rarely used, and so I think it doesn't need to be that ergonomic to use. But it shouldn't be bug-prone.

[moxian]

Since we are not sure the API is appropriate, it was suggested on zullip to revert the change the stabilization in the short term before it hits stable while we figure this out. I've made #138822

[jieyouxu]

Reopening because stabilization was reverted to reconsider the signature.

[cberner]

@BurntSushi Result<(), TryLockError> seems good to me too. Process-wise, anything I can do to help here? I can send a PR to make that change, but am not sure if I should open a new ACP first, since the current one says to use the Result<bool> style.

[BurntSushi]

I still think that's the best option, yes. I'm not sure if we have consensus yet though: https://rust-lang.zulipchat.com/#narrow/channel/219381-t-libs/topic/File.3A.3Atry_lock.20API.3A.20Result.3Cbool.3E

I don't think an ACP is needed for this sort of thing. ACPs are more like, "is the general direction of this API something that libs-api is receptive to." I think we have consensus that we want an API here, but it sounds like we don't yet have consensus on its form. So the right place for iterating on that is this issue, Zulip and libs-api meetings.

[Amanieu]

We briefly discussed this in the @rust-lang/libs-api meeting today. Overall we felt that the best solution was indeed something along the lines of Result<(), TryLockError> where TryLockError is an enum with 2 variants.

[cberner]

@Amanieu sounds good. I will put together a PR to make that change. One other option came to mind though. Did you discuss having the signature be Result<(), io::Error> and returning ErrorKind::WouldBlock to indicate that lock acquisition failed?

On the one hand, it's nice because then try_lock()? can be used in a function that returns io::Result, but on the other hand it's less obvious from the signature how acquisition failure is handled.

[juntyr]

We could also add a From impl to convert the new error type to an io::Error

[cberner]

@juntyr oh ya, that gives the best of both worlds. I didn't realize that would automatically work with ?. Thanks! I'll update the PR

[BurntSushi]

@juntyr oh ya, that gives the best of both worlds. I didn't realize that would automatically work with ?. Thanks! I'll update the PR

Please keep that separate for now. It isn't clear to me that is a good idea, and it is an impl that can be added later after we gain experience with the API.

[cberner]

Sounds good. I reverted that part of the PR

[Amanieu]

@Amanieu sounds good. I will put together a PR to make that change. One other option came to mind though. Did you discuss having the signature be Result<(), io::Error> and returning ErrorKind::WouldBlock to indicate that lock acquisition failed?

On the one hand, it's nice because then try_lock()? can be used in a function that returns io::Result, but on the other hand it's less obvious from the signature how acquisition failure is handled.

This was discussed during the meeting and there was a slightly preference towards a custom error type since that makes it very obvious if the failure was due to a lock already existing or some other I/O error. Also note that currently Windows doesn't turn ERROR_IO_PENDING into ErrorKind::WouldBlock so that would have to be added.