Commit 93824900 authored Sep 13, 2017 by Uladzislau Rezki Committed by Ingo Molnar Oct 10, 2017

sched/fair: Search a task from the tail of the queue



As a first step this patch makes cfs_tasks list as MRU one.
It means, that when a next task is picked to run on physical
CPU it is moved to the front of the list.

Therefore, the cfs_tasks list is more or less sorted (except
woken tasks) starting from recently given CPU time tasks toward
tasks with max wait time in a run-queue, i.e. MRU list.

Second, as part of the load balance operation, this approach
starts detach_tasks()/detach_one_task() from the tail of the
queue instead of the head, giving some advantages:

 - tends to pick a task with highest wait time;

 - tasks located in the tail are less likely cache-hot,
   therefore the can_migrate_task() decision is higher.

hackbench illustrates slightly better performance. For example
doing 1000 samples and 40 groups on i5-3320M CPU, it shows below
figures:

 default: 0.657 avg
 patched: 0.646 avg

Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Kirill Tkhai <tkhai@yandex.ru>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Mike Galbraith <umgwanakikbuti@gmail.com>
Cc: Nicolas Pitre <nicolas.pitre@linaro.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Oleksiy Avramchenko <oleksiy.avramchenko@sonymobile.com>
Cc: Paul Turner <pjt@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Tim Chen <tim.c.chen@linux.intel.com>
Link: http://lkml.kernel.org/r/20170913102430.8985-2-urezki@gmail.com


Signed-off-by: Ingo Molnar <mingo@kernel.org>

parent 051f3ca0

kernel/sched/fair.c

+16 −8

Original line number	Diff line number	Diff line
		@@ -6628,10 +6628,7 @@ again:
		set_next_entity(cfs_rq, se);
		}

		if (hrtick_enabled(rq))
		hrtick_start_fair(rq, p);

		return p;
		goto done;
		simple:
		#endif

		@@ -6645,6 +6642,16 @@ simple:

		p = task_of(se);

		done: __maybe_unused
		#ifdef CONFIG_SMP
		/*
		* Move the next running task to the front of
		* the list, so our cfs_tasks list becomes MRU
		* one.
		*/
		list_move(&p->se.group_node, &rq->cfs_tasks);
		#endif

		if (hrtick_enabled(rq))
		hrtick_start_fair(rq, p);

		@@ -7080,11 +7087,12 @@ static void detach_task(struct task_struct p, struct lb_env env)
		*/
		static struct task_struct detach_one_task(struct lb_env env)
		{
		struct task_struct p, n;
		struct task_struct *p;

		lockdep_assert_held(&env->src_rq->lock);

		list_for_each_entry_safe(p, n, &env->src_rq->cfs_tasks, se.group_node) {
		list_for_each_entry_reverse(p,
		&env->src_rq->cfs_tasks, se.group_node) {
		if (!can_migrate_task(p, env))
		continue;

		@@ -7130,7 +7138,7 @@ static int detach_tasks(struct lb_env *env)
		if (env->idle != CPU_NOT_IDLE && env->src_rq->nr_running <= 1)
		break;

		p = list_first_entry(tasks, struct task_struct, se.group_node);
		p = list_last_entry(tasks, struct task_struct, se.group_node);

		env->loop++;
		/* We've more or less seen every task there is, call it quits */
		@@ -7180,7 +7188,7 @@ static int detach_tasks(struct lb_env *env)

		continue;
		next:
		list_move_tail(&p->se.group_node, tasks);
		list_move(&p->se.group_node, tasks);
		}

		/*

Admin message