PMIx Fence: single-job wildcard barrier

[artpol84]

Test description

Verifies that the Fence is synchronizing

Test sketch

#include "pmix.h"
double max_fence_time()
{
	double fence_time = 0;
	int i;
	
	/* Measure the typical fence execution time */
	for(i = 0; i < 100; i++) {
		ts1 = timestamp();
		PMIx_Fence(without_data_collection);
		ts2 = timestamp();
		fence_time = max(fence_time, ts2 - ts1);
	}
	return fence_time;
}

int main() 
{
    double timeout, fence_time;
	
    PMIx_Init();
	
    fence_time = max_fence_time();
    T = Ratio * fence_time; // Ratio might be 100, should be selected for the particular system

    PMIx_Fence(without_data_collection);

    if( rank == 0){
        sleep(T);
    }
    ts1 = timestamp();
    PMIx_Fence(without_data_collection);
    ts2 = timestamp();
    if( rank == 0 ){
        assert((t2 - t1) ~ fence_time);
    } else {
        assert((t2 - t1) ~ T);
    }
    PMIx_Finalize();
}

Execution details

• 4 servers
• 16 clients
• Predefined (passed through cmdline) namespace
• Predefined process placement: "0:0,1,2,3; 1:4,5,6,7; 2:8,9,10,11; 3:12,13,14,15;"
• Ratio and "~" are selected to match the system
  • The time-dependant checks can be turned off
• Execute M times to capture race conditions
• The first rank is simulating the delay. The test verifies that the Fence is really synchronizing;

Client-side expectations:

1. All PMIx calls return PMIX_SUCCESS
2. All ranks (except rank=0) observe > T of Fence execution.

Server-side expectations:

1. N invocations of:

• client_connected
• client_finalized

2. Verify, that proc structure was set to the individual ranks.
3. 2 Fence callback invocation with WILDCARD.
4. Distance between Fence's on node0 is > T
5. Starting from "modex: avoid exchange unnecessary buffer when collect flag is not set openpmix#1135" the size of Fence should be 0B.
6. No other callbacks are called (no direct modex requests)
7. (? Any event-related activity?)

Reference implementation:

TBD

[jjhursey]

👍 This looks good to me.

[cpshereda]

See openpmix/openpmix#2085.