IB/core: Add cross-channel support (8a06ce59) · Commits · 戴 / test

The cross-channel feature allows to execute WQEs that involve synchronization of I/O operations’ on different QPs. This capability enables to program complex flows with a single function call, hereby significantly reducing overhead associated with I/O processing. Cross-channel operations support is indicated by HCA capability information. The queue pairs can be configured to work as a “sync master queue” or “sync slave queues”. The added flags are: 1. Device capability flag IB_DEVICE_CROSS_CHANNEL for the devices that can perform cross-channel operations. 2. CQ property flag IB_CQ_FLAGS_IGNORE_OVERRUN to disable CQ overrun check. This check is useless in cross-channel scenario. 3. QP property flags to indicate if queues are slave or master: * IB_QP_CREATE_MANAGED_SEND indicates that posted send work requests will not be executed immediately and requires enabling. * IB_QP_CREATE_MANAGED_RECV indicates that posted receive work requests will not be executed immediately and requires enabling. * IB_QP_CREATE_CROSS_CHANNEL declares the QP to work in cross-channel mode. If IB_QP_CREATE_MANAGED_SEND and IB_QP_CREATE_MANAGED_RECV are not provided, this QP will be sync master queue, else it will be sync slave. Reviewed-by:

Sagi Grimberg <sagig@mellanox.com> Signed-off-by:

Leon Romanovsky <leonro@mellanox.com> Signed-off-by:

Doug Ledford <dledford@redhat.com>

drivers/infiniband/core/uverbs_cmd.c

+4 −1

Original line number	Diff line number	Diff line
		@@ -1824,7 +1824,10 @@ static int create_qp(struct ib_uverbs_file *file,
		sizeof(cmd->create_flags))
		attr.create_flags = cmd->create_flags;

		if (attr.create_flags & ~IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK) {
		if (attr.create_flags & ~(IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK \|
		IB_QP_CREATE_CROSS_CHANNEL \|
		IB_QP_CREATE_MANAGED_SEND \|
		IB_QP_CREATE_MANAGED_RECV)) {
		ret = -EINVAL;
		goto err_put;
		}

include/rdma/ib_verbs.h

+11 −0

Original line number	Diff line number	Diff line
		@@ -201,6 +201,13 @@ enum ib_device_cap_flags {
		IB_DEVICE_MEM_WINDOW_TYPE_2B = (1 << 24),
		IB_DEVICE_RC_IP_CSUM = (1 << 25),
		IB_DEVICE_RAW_IP_CSUM = (1 << 26),
		/*
		* Devices should set IB_DEVICE_CROSS_CHANNEL if they
		* support execution of WQEs that involve synchronization
		* of I/O operations with single completion queue managed
		* by hardware.
		*/
		IB_DEVICE_CROSS_CHANNEL = (1 << 27),
		IB_DEVICE_MANAGED_FLOW_STEERING = (1 << 29),
		IB_DEVICE_SIGNATURE_HANDOVER = (1 << 30),
		IB_DEVICE_ON_DEMAND_PAGING = (1 << 31),
		@@ -246,6 +253,7 @@ struct ib_odp_caps {

		enum ib_cq_creation_flags {
		IB_CQ_FLAGS_TIMESTAMP_COMPLETION = 1 << 0,
		IB_CQ_FLAGS_IGNORE_OVERRUN = 1 << 1,
		};

		struct ib_cq_init_attr {
		@@ -950,6 +958,9 @@ enum ib_qp_type {
		enum ib_qp_create_flags {
		IB_QP_CREATE_IPOIB_UD_LSO = 1 << 0,
		IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK = 1 << 1,
		IB_QP_CREATE_CROSS_CHANNEL = 1 << 2,
		IB_QP_CREATE_MANAGED_SEND = 1 << 3,
		IB_QP_CREATE_MANAGED_RECV = 1 << 4,
		IB_QP_CREATE_NETIF_QP = 1 << 5,
		IB_QP_CREATE_SIGNATURE_EN = 1 << 6,
		IB_QP_CREATE_USE_GFP_NOIO = 1 << 7,

Admin message