Enhance the test by test multiple communicators. This catches the
current ofi huge path's incorrect context_id handling.

Since we always call MPIDI_OFI_recv_event at completion of huge message,
we do not need the indirect function pointer. Remove for cleaner code.

Make MPIDI_OFI_send_control_t internally a union to reflect the general
control semantics.

Replace MPIDI_OFI_do_control_send with MPIDI_NM_am_send_hdr.

It is not critical to inline functions related to huge messages as they
are bandwidth dominated. Move them into ofi_huge.c for better context.

The counter was not incremented for the correct nic.

Add FIXME to note potential issues with multiple simultaneous fi_read.

Rather than check and fallback in MPIDI_OFI_recv_huge_event, check
whether it's a huge message in event loop and dispatch accordingly.

This partially fixes test/mpi/errors/pt2pt/truncmsg1 when set
MPIR_CVAR_CH4_OFI_EAGER_MAX_MSG_SIZE=16384. We still need fix
the case when huge message sent but small buffer is posted.

The progress of huge message will depend on receiving the ctrl message.
Send it first to promote the likelihood of ctrl header not arriving too
much behind the message body.

The debug messages are temporary and should be removed after debugging.

When probing huge messages and control is missing, we should handle
probe and mprobe differently. With probe, we can simply return not
found. With mprobe, we can enqueue the rreq since the entry is
guaranteed not to be double matched.

Store huge_send_mrs in the sreq so we don't need the extra global map.

Store the recv_elem pointer with rreq, so we don't need extra
huge_recv_counters map.

Clean up the data structure to be more specific.

Differentiate probe and mprobe. The former can be discarded when the
control isn't ready. If we don't discard unsuccessful probe and put it in
a queue, it can cause issues when another probe or recv come to
interfere. Mprobe, on the other hand, is guaranteed to match once, so
there is no issue.

Persist the remote info with the original rreq. This avoids the use of
separate hash maps to look up. It is also cleaner to track.

Split the huge recv completion into static function.

We need handle this case the the sender still receives the ack message.

Add safety assertions to ensure consistentcy.

The huge recv modifies the data_sz. Thus we need set
MPIDI_OFI_REQUEST(rreq, util.iov.iov_len) earlier to prevent setting the
wrong size.

The comm_id in huge message path is, in fact, context_id of the
communicator. We need use MPIR_Context_id_t as type.

Direct applying mask to wc->tag won't get the corresponding context_id
due to missing shift. Fix it by directly using
rreq->comm->recvcontext_id.