Commit 17950c5b authored by Jeremy Fitzhardinge's avatar Jeremy Fitzhardinge
Browse files

x86-64: move clts into batch cpu state updates when preloading fpu 



When a task is likely to be using the fpu, we preload its state during
the context switch, rather than waiting for it to run an fpu instruction.
Make sure the clts() happens while we're doing batched fpu state updates
to optimise paravirtualized context switches.

[ Impact: optimise paravirtual FPU context switch ]

Signed-off-by: default avatarJeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Cc: Alok Kataria <akataria@vmware.com>
Cc: Rusty Russell <rusty@rustcorp.com.au>
parent 16d9dbf0

arch/x86/kernel/process_64.c

+18 −9
Original line number Diff line number Diff line
@@ -386,9 +386,17 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p) 
	int cpu = smp_processor_id();

	struct tss_struct *tss = &per_cpu(init_tss, cpu);

	unsigned fsindex, gsindex;

	bool preload_fpu;



	/*

	 * If the task has used fpu the last 5 timeslices, just do a full

	 * restore of the math state immediately to avoid the trap; the

	 * chances of needing FPU soon are obviously high now

	 */

	preload_fpu = tsk_used_math(next_p) && next_p->fpu_counter > 5;



	/* we're going to use this soon, after a few expensive things */

	if (next_p->fpu_counter > 5)

	if (preload_fpu)

		prefetch(next->xstate);



	/*
@@ -422,6 +430,10 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p) 
	/* Must be after DS reload */

	unlazy_fpu(prev_p);



	/* Make sure cpu is ready for new context */

	if (preload_fpu)

		clts();



	/*

	 * Leave lazy mode, flushing any hypercalls made here.

	 * This must be done before restoring TLS segments so
@@ -480,15 +492,12 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p) 
		     task_thread_info(prev_p)->flags & _TIF_WORK_CTXSW_PREV))

		__switch_to_xtra(prev_p, next_p, tss);



	/* If the task has used fpu the last 5 timeslices, just do a full

	 * restore of the math state immediately to avoid the trap; the

	 * chances of needing FPU soon are obviously high now

	 *

	 * tsk_used_math() checks prevent calling math_state_restore(),

	 * which can sleep in the case of !tsk_used_math()

	/*

	 * Preload the FPU context, now that we've determined that the

	 * task is likely to be using it. 

	 */

	if (tsk_used_math(next_p) && next_p->fpu_counter > 5)

		math_state_restore();

	if (preload_fpu)

		__math_state_restore();

	return prev_p;

}

CRA Git | Maintained and supported by SUSTech CRA and CCSE