Commit 89448c47 authored by Pavel Begunkov's avatar Pavel Begunkov Committed by Jens Axboe
Browse files

io_uring: limit {io|sq}poll submit locking scope 



We don't need to take uring_lock for SQPOLL|IOPOLL to do
io_cqring_overflow_flush() when cq_overflow_list is empty, remove it
from the hot path.

Signed-off-by: default avatarPavel Begunkov <asml.silence@gmail.com>
Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
parent 09e88404

fs/io_uring.c

+6 −3
Original line number Diff line number Diff line
@@ -9154,10 +9154,13 @@ SYSCALL_DEFINE6(io_uring_enter, unsigned int, fd, u32, to_submit, 
	 */

	ret = 0;

	if (ctx->flags & IORING_SETUP_SQPOLL) {

		io_ring_submit_lock(ctx, (ctx->flags & IORING_SETUP_IOPOLL));

		if (!list_empty_careful(&ctx->cq_overflow_list))

		if (!list_empty_careful(&ctx->cq_overflow_list)) {

			bool needs_lock = ctx->flags & IORING_SETUP_IOPOLL;



			io_ring_submit_lock(ctx, needs_lock);

			io_cqring_overflow_flush(ctx, false, NULL, NULL);

		io_ring_submit_unlock(ctx, (ctx->flags & IORING_SETUP_IOPOLL));

			io_ring_submit_unlock(ctx, needs_lock);

		}

		if (flags & IORING_ENTER_SQ_WAKEUP)

			wake_up(&ctx->sq_data->wait);

		if (flags & IORING_ENTER_SQ_WAIT)

CRA Git | Maintained and supported by SUSTech CRA and CCSE