blk-mq: allow non-softirq completions (95f09684) · Commits · 戴 / test

Right now we export two ways of completing a request: 1) blk_mq_complete_request(). This uses an IPI (if needed) and completes through q->softirq_done_fn(). It also works with timeouts. 2) blk_mq_end_io(). This completes inline, and ignores any timeout state of the request. Let blk_mq_complete_request() handle non-softirq_done_fn completions as well, by just completing inline. If a driver has enough completion ports to place completions correctly, it need not define a mq_ops->complete() and we can avoid an indirect function call by doing the completion inline. Signed-off-by:

Jens Axboe <axboe@fb.com>

block/blk-mq.c

+9 −3

Original line number	Diff line number	Diff line
		@@ -434,10 +434,16 @@ void __blk_mq_complete_request(struct request *rq)
		**/
		void blk_mq_complete_request(struct request *rq)
		{
		if (unlikely(blk_should_fake_timeout(rq->q)))
		struct request_queue *q = rq->q;

		if (unlikely(blk_should_fake_timeout(q)))
		return;
		if (!blk_mark_rq_complete(rq))
		if (!blk_mark_rq_complete(rq)) {
		if (q->softirq_done_fn)
		__blk_mq_complete_request(rq);
		else
		blk_mq_end_io(rq, rq->errors);
		}
		}
		EXPORT_SYMBOL(blk_mq_complete_request);

include/linux/blk-mq.h

+4 −0

Original line number	Diff line number	Diff line
		@@ -173,6 +173,10 @@ void __blk_mq_end_io(struct request *rq, int error);

		void blk_mq_requeue_request(struct request *rq);

		/*
		* Complete request through potential IPI for right placement. Driver must
		* have defined a mq_ops->complete() hook for this.
		*/
		void blk_mq_complete_request(struct request *rq);

		void blk_mq_stop_hw_queue(struct blk_mq_hw_ctx *hctx);

Admin message