Commit 6ccdc84b authored Apr 24, 2014 by Peter Zijlstra Committed by Ingo Molnar May 07, 2014

sched: Skip double execution of pick_next_task_fair()



Tim wrote:

 "The current code will call pick_next_task_fair a second time in the
  slow path if we did not pull any task in our first try.  This is
  really unnecessary as we already know no task can be pulled and it
  doubles the delay for the cpu to enter idle.

  We instrumented some network workloads and that saw that
  pick_next_task_fair is frequently called twice before a cpu enters
  idle.  The call to pick_next_task_fair can add non trivial latency as
  it calls load_balance which runs find_busiest_group on an hierarchy of
  sched domains spanning the cpus for a large system.  For some 4 socket
  systems, we saw almost 0.25 msec spent per call of pick_next_task_fair
  before a cpu can be idled."

Optimize the second call away for the common case and document the
dependency.

Reported-by: Tim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Len Brown <len.brown@intel.com>
Link: http://lkml.kernel.org/r/20140424100047.GP11096@twins.programming.kicks-ass.net


Signed-off-by: Ingo Molnar <mingo@kernel.org>

parent 6227cb00

kernel/sched/core.c

+8 −2

Original line number	Diff line number	Diff line
		@@ -2592,7 +2592,13 @@ pick_next_task(struct rq rq, struct task_struct prev)
		if (likely(prev->sched_class == class &&
		rq->nr_running == rq->cfs.h_nr_running)) {
		p = fair_sched_class.pick_next_task(rq, prev);
		if (likely(p && p != RETRY_TASK))
		if (unlikely(p == RETRY_TASK))
		goto again;

		/* assumes fair_sched_class->next == idle_sched_class */
		if (unlikely(!p))
		p = idle_sched_class.pick_next_task(rq, prev);

		return p;
		}

Admin message