Is there a method like .contains_key() that returns true if the entry is present, or is currently being loaded?

[ns476]

From my experiments, .contains_key() returns false when a key isn't currently present but is in the process of being loaded. This can be seen with a test like the one below:

#[tokio::test]
async fn test_contains_key() {
    use moka::future::CacheBuilder;

    let cache = CacheBuilder::new(1024).build();

    let _ = tokio::join!(
        cache.get_with("hello", async {
            tokio::time::sleep(Duration::from_secs(1)).await;
            "world"
        }),
        async {
            tokio::time::sleep(Duration::from_millis(500)).await;
            assert!(!cache.contains_key("hello"));
            tokio::time::sleep(Duration::from_secs(1)).await;
            assert!(cache.contains_key("hello"));
        }
    );
}

I want something like .contains_key(), but that also returns true if the key is currently being loaded, so the following would work:

#[tokio::test]
async fn test_contains_or_is_loading_key() {
    use moka::future::CacheBuilder;

    let cache = CacheBuilder::new(1024).build();

    let _ = tokio::join!(
        cache.get_with("hello", async {
            tokio::time::sleep(Duration::from_secs(1)).await;
            "world"
        }),
        async {
            tokio::time::sleep(Duration::from_millis(500)).await;
            // True, as the key is currently being loaded
            assert!(cache.contains_or_is_loading_key("hello"));
            tokio::time::sleep(Duration::from_secs(1)).await;
            assert!(cache.contains_or_is_loading_key("hello"));
        }
    );
}

This can be emulated with a separate set, but seems like a generally useful primitive.

Our use case: we have a Moka cache in front of our service, which load sheds. We don't want to load shed requests that will hit the cache, but to detect this we need something like contains_or_is_loading_key.

[tatsuya6502]

Hi. Thank you for the suggestion, and sorry for the late response.

From my experiments, .contains_key() returns false when a key isn't currently present but is in the process of being loaded.

Yes, that is correct.

This can be emulated with a separate set, but seems like a generally useful primitive.

Right, it can be emulated with a separate set.

I almost agree with you that it is a generally useful primitive. But I am not sure if we can provide a good and consistent API for this to meet all users' needs. contains_or_is_loading_key would be useful for everybody who wants to know if the entry is being loaded by get_with. But what if you want to know the entry is being loaded by similar methods like optionally_get_with or try_get_with, which may not load the entry? I think it depends on the use case.

The cache provides other methods like entry(key).and_try_compute_with(...) (doc), which may load, update, remove or do nothing to the entry. The cache can know if and_try_compute_with is resolving a given Future for a key, but it cannot know whether the entry for the key is being loaded or not until the Future is resolved.

So, I think:

• contains_or_is_loading_key will be useful for get_with (and entry(key).or_insert_with(...)).
• It may or may not be useful for optionally_get_with and try_get_with. It depends on use cases.
• It does not work for and_try_compute_with and and_compute_with without knowing what the Future is doing.

Perhaps, it would be straightforward to emulate this with a separate set at the application side? Maybe we should provide example code snippets to show how to implement it?

Also, there is a technical issue if we want contains_or_is_loading_key to support try_get_with. In order to check if the entry is being loaded by try_get_with, the contains_or_is_loading_key needs to know the concrete error type of the generic parameter E of the init Future given to try_get_with. This is because calls for try_get_with on the same key will be coalesced only if the error type is the same.

• https://github.com/moka-rs/moka/blob/v0.12.8/src/future/value_initializer.rs#L521
• https://github.com/moka-rs/moka/blob/v0.12.8/src/future/value_initializer.rs#L462-L477

An alternative way is to add another set in the cache to store keys that are being loaded by try_get_with and all other methods. But this will increase the memory usage of the cache and you will not want it if you do not use try_get_with. So again, it may be straightforward to emulate this at the application side?

I think contains_or_is_loading_key is generic enough to be useful for get_with. But I am not sure if it is useful for other methods. It might be better to provide example code snippets to show how to implement it at the application side? I am open to suggestions though.