providers/mana: RC QPs #1461
providers/mana: RC QPs #1461
Conversation
TaranovK
force-pushed
the
for-next/cq_upstream
branch
5 times, most recently
from
354b2a1
to
4b0e953
Compare
|
Please rebase your branch to latest rdma-core and prepare right "Update kernel headers" commit. Thanks |
Thanks. We are performing more extensive internal tests. So I will rebase with the commit ID, once I get a green flag from testers. |
providers/mana/wr.c
Outdated
| } | ||
|
|
||
| mana_ib_update_shared_mem_right_offset(qp, unmasked_queue_offset); | ||
| gdma_ring_send_doorbell(&qp->rc_qp.queues[USER_RC_SEND_QUEUE_REQUESTER]); |
There was a problem hiding this comment.
Is it necessary to ring the doorbell once per wr within the loop body, or can the doorbell ring be moved outside the loop to batch multiple submissions?
There was a problem hiding this comment.
No, it is not necessary. The code is not final, and some final optimizations will be made last.
The initial reason for this implementation was debugging of HW/SW.
Any update? |
|
@rleon It seems to take longer than I expected to get an internal signoff on the code. I marked the PR as draft. Sorry for noise. |
So what will do with UAPI kernel part? |
Re-organize code to have cq.c and wr.c with CQ and WR code, respectively. Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Mana HW expects buffers to be a power of 2 and be at least 4KB. The existing align_hw_size returns 1 when input is 0, providing incorrect buffer size for empty queues. Change the order of align and pow to get 4096 for the input 0, which is expected by the HW. Use uint32_t in the get_wqe_size helper. Fixes: 443f196 ("mana: Microsoft Azure Network Adapter (MANA) RDMA provider") Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Implement creation and destruction of RNIC cqs. Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Implement writing to doorbells for different queue types. Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Define struct for fields related to raw QPs. Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Implement shadow queues to store pending WQEs. Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Introduce helpers to work with PSNs. Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
TaranovK
force-pushed
the
for-next/cq_upstream
branch
from
4b0e953
to
58eedfb
Compare
Then in this case, I update the branch to have the code that was agreed with my colleagues. It will be ready tomorrow. Sorry again. |
TaranovK
force-pushed
the
for-next/cq_upstream
branch
4 times, most recently
from
c4589a6
to
6ceae2a
Compare
|
@rleon Sorry again for the delay. The code is ready. |
| free(context->qp_table[i].table); | ||
| } | ||
| pthread_mutex_unlock(&context->qp_table_mutex); | ||
| pthread_mutex_destroy(&context->qp_table_mutex); |
There was a problem hiding this comment.
This line says to me that previous lock/unlock is useless.
There was a problem hiding this comment.
So I guess you are implying that it is user's problem if it called "close device" while other threads access context.
I will remove it in a bit.
There was a problem hiding this comment.
I'm saying something different. If you have lock destroy, it means that this lock is not used in this path otherwise your concurrent thread which used this lock will crash.
Implement a table for finding QPs using receive queue ids. Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Implement creation and destruction of RC QPs. Add helpers to work with rollback memory regions. Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Implement modify QP for RC QPs. Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Implement work posting and define HW structures. Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Implement polling of CQs and arming CQs. Add processing of CQEs for RC QPs. Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
TaranovK
force-pushed
the
for-next/cq_upstream
branch
from
6ceae2a
to
1ef7c56
Compare
This patch series aims to provide RC QP functionality to mana.