block: relax jiffies rounding for timeouts (9054650f) · Commits · 戴 / test

In doing high IOPS testing, blk-mq is generally pretty well optimized. There are a few things that stuck out as using more CPU than what is really warranted, and one thing is the round_jiffies_up() that we do twice for each request. That accounts for about 0.8% of the CPU in my testing. We can make this cheaper by avoiding an integer division, by just adding a rough HZ mask that we can AND with instead. The timeouts are only on a second granularity already, we don't have to be that accurate here and this patch barely changes that. All we care about is nice grouping. Signed-off-by:

Jens Axboe <axboe@kernel.dk>

block/blk-timeout.c

+20 −2

Original line number	Diff line number	Diff line
		@@ -88,11 +88,29 @@ void blk_abort_request(struct request *req)
		}
		EXPORT_SYMBOL_GPL(blk_abort_request);

		static unsigned long blk_timeout_mask __read_mostly;

		int __init blk_timeout_init(void)
		{
		blk_timeout_mask = roundup_pow_of_two(HZ) - 1;
		return 0;
		}

		late_initcall(blk_timeout_init);

		/*
		* Just a rough estimate, we don't care about specific values for timeouts.
		*/
		static inline unsigned long blk_round_jiffies(unsigned long j)
		{
		return (j + blk_timeout_mask) + 1;
		}

		unsigned long blk_rq_timeout(unsigned long timeout)
		{
		unsigned long maxt;

		maxt = round_jiffies_up(jiffies + BLK_MAX_TIMEOUT);
		maxt = blk_round_jiffies(jiffies + BLK_MAX_TIMEOUT);
		if (time_after(timeout, maxt))
		timeout = maxt;

		@@ -129,7 +147,7 @@ void blk_add_timer(struct request *req)
		* than an existing one, modify the timer. Round up to next nearest
		* second.
		*/
		expiry = blk_rq_timeout(round_jiffies_up(expiry));
		expiry = blk_rq_timeout(blk_round_jiffies(expiry));

		if (!timer_pending(&q->timeout) \|\|
		time_before(expiry, q->timeout.expires)) {

Admin message