fabrics: define admin sqsize min default, per spec (f994d9dc) · Commits · 戴 / test

Upon admin queue connect(), the rdma qp was being set based on NVMF_AQ_DEPTH. However, the fabrics layer was using the sqsize field value set for I/O queues for the admin queue, which threw the nvme layer and rdma layer off-whack: root@fedora23-fabrics-host1 nvmf]# dmesg [ 3507.798642] nvme_fabrics: nvmf_connect_admin_queue():admin sqsize being sent is: 128 [ 3507.798858] nvme nvme0: creating 16 I/O queues. [ 3507.896407] nvme nvme0: new ctrl: NQN "nullside-nqn", addr 192.168.1.3:4420 Thus, to have a different admin queue value, we use NVMF_AQ_DEPTH for connect() and RDMA private data as the minimum depth specified in the NVMe-over-Fabrics 1.0 spec (and in that RDMA private data we treat hrqsize as 1's-based value, per current understanding of the fabrics spec). Reported-by:

Daniel Verkamp <daniel.verkamp@intel.com> Signed-off-by:

Jay Freyensee <james_p_freyensee@linux.intel.com> Reviewed-by:

Daniel Verkamp <daniel.verkamp@intel.com> Signed-off-by:

Sagi Grimberg <sagi@grimberg.me>

drivers/nvme/host/fabrics.c

+8 −1

Original line number	Diff line number	Diff line
		@@ -363,7 +363,14 @@ int nvmf_connect_admin_queue(struct nvme_ctrl *ctrl)
		cmd.connect.opcode = nvme_fabrics_command;
		cmd.connect.fctype = nvme_fabrics_type_connect;
		cmd.connect.qid = 0;
		cmd.connect.sqsize = cpu_to_le16(ctrl->sqsize);

		/*
		* fabrics spec sets a minimum of depth 32 for admin queue,
		* so set the queue with this depth always until
		* justification otherwise.
		*/
		cmd.connect.sqsize = cpu_to_le16(NVMF_AQ_DEPTH - 1);

		/*
		* Set keep-alive timeout in seconds granularity (ms * 1000)
		* and add a grace period for controller kato enforcement

drivers/nvme/host/rdma.c

+11 −2

Original line number	Diff line number	Diff line
		@@ -1278,8 +1278,17 @@ static int nvme_rdma_route_resolved(struct nvme_rdma_queue *queue)

		priv.recfmt = cpu_to_le16(NVME_RDMA_CM_FMT_1_0);
		priv.qid = cpu_to_le16(nvme_rdma_queue_idx(queue));
		/*
		* set the admin queue depth to the minimum size
		* specified by the Fabrics standard.
		*/
		if (priv.qid == 0) {
		priv.hrqsize = cpu_to_le16(NVMF_AQ_DEPTH);
		priv.hsqsize = cpu_to_le16(NVMF_AQ_DEPTH - 1);
		} else {
		priv.hrqsize = cpu_to_le16(queue->queue_size);
		priv.hsqsize = cpu_to_le16(queue->queue_size);
		}

		ret = rdma_connect(queue->cm_id, &param);
		if (ret) {

Admin message