README

This README is for userspace RDMA cm library.


Building
========
To make this directory, run:
./autogen.sh && ./configure && make && make install

Typically the autogen and configure steps only need be done the first
time unless configure.in or Makefile.am changes.

Libraries are installed by default at /usr/local/lib.


Device files
============
The userspace CMA uses a single device file regardless of the number
of adapters or ports present.

To create the appropriate character device file automatically with
udev, a rule like

    KERNEL="rdma_cm", NAME="infiniband/%k", MODE="0666"

can be used.  This will create the device node named

    /dev/infiniband/rdma_cm

or you can create it manually

  mknod /dev/infiniband/rdma_cm c 231 255


Common issues
=============

Using multiple interfaces
	The librdmacm does support multiple interfaces.  To make use
	of multiple interfaces, however, you need to instruct linux
	to only send ARP reples on the interface targetted in the ARP
	request.  This can be done using a command similar to the
	following:

	sysctl -w net.ipv4.conf.all.arp_ignore=2

	Without this change, it's possible for linux to resopnd to ARP
	requests on a different interface (IP address) than the IP
	address carried in the ARP request.  This causes the RDMA stack
	to incorrectly map the remote IP address to the wrong RDMA
	device.

Using loopback
	The librdmacm relies on ARP to resolve IP address to RDMA
	addresses.  To support loopback connections between different
	ports on the same system, ARP must be enabled for local
	resolution:

	sysctl net.ipv4.conf.all.accept_local=1

	Without this setting, loopback connections may timeout
	during address resolution.