Report number of registrations required/supported by domain

[shefty]

Add mr_cnt field to domain_attr, to indicate how many MRs are supported by this domain. This is a small change to the domain attribute, but would need to be fit in prior to the 1.5 release, since it updates the API.

[jswaro]

I like this idea. Is this something reported back to the application during the fi_getinfo inquiry by the application?

[shefty]

Yes - I was thinking of just adding a new domain attribute field. The change is fairly small, but should be done before 1.5 is released to save another api churn.

[jswaro]

Is it going to be something that is only returned from providers, or will it be a 'in' parameter as part of hints?

If it is part of the hints, I imagine providers would need to return -ENODATA if they cannot support the number of registrations requested in the hints.

[shefty]

I believe it should work as an in parameter as well. Maybe the provider can optimize if it knew how many regions the app needed?

[jswaro]

I believe it should work as an in parameter as well. Maybe the provider can optimize if it knew how many regions the app needed?

I think in some cases that might be true. I know that the GNI provider has limitations on the number of registrations depending on the memory registration mode, which could be affected by this change (really only affects what number gets reported). I think if providers use a user-space level cache, it can help to reduce memory consumption if much of the space used would be wasted without the knowledge of how many registrations an application needed.

It does pose an interesting question. I don't know that this applies to other providers, but the number of available keys is dependent on the memory model being used, and for the GNI provider, that is associated with the domain (but really tied to the [hardware]protection key/[libfabric]authorization key). Does it make sense to report the number of memory registrations that can be associated with a particular authorization key? I realize that the GNI provider may one of a few (or the only) provider(s) that tie memory registration to the hardware level protection constructs through the libfabric authorization key.

[shefty]

See PR #3105 -- all providers will need to be updated. That will be a separate set of PRs.