x86/process/64: Use FSBSBASE in switch_to() if available (67390349) · Commits · 戴 / test

With the new FSGSBASE instructions, FS and GSABSE can be efficiently read and writen in __switch_to(). Use that capability to preserve the full state. This will enable user code to do whatever it wants with the new instructions without any kernel-induced gotchas. (There can still be architectural gotchas: movl %gs,%eax; movl %eax,%gs may change GSBASE if WRGSBASE was used, but users are expected to read the CPU manual before doing things like that.) This is a considerable speedup. It seems to save about 100 cycles per context switch compared to the baseline 4.6-rc1 behavior on a Skylake laptop. This is mostly due to avoiding the WRMSR operation. [ chang: 5~10% performance improvements were seen with a context switch benchmark that ran threads with different FS/GSBASE values (to the baseline 4.16). Minor edit on the changelog. ] [ tglx: Masaage changelog ] Signed-off-by:

Andy Lutomirski <luto@kernel.org> Signed-off-by:

Chang S. Bae <chang.seok.bae@intel.com> Signed-off-by:

Thomas Gleixner <tglx@linutronix.de> Signed-off-by:

Sasha Levin <sashal@kernel.org> Signed-off-by:

Thomas Gleixner <tglx@linutronix.de> Reviewed-by:

Andi Kleen <ak@linux.intel.com> Link: https://lkml.kernel.org/r/1557309753-24073-8-git-send-email-chang.seok.bae@intel.com Link: https://lkml.kernel.org/r/20200528201402.1708239-6-sashal@kernel.org

arch/x86/kernel/process_64.c

+28 −6

Original line number	Diff line number	Diff line
		@@ -236,9 +236,19 @@ static __always_inline void save_fsgs(struct task_struct *task)
		{
		savesegment(fs, task->thread.fsindex);
		savesegment(gs, task->thread.gsindex);
		if (static_cpu_has(X86_FEATURE_FSGSBASE)) {
		/*
		* If FSGSBASE is enabled, we can't make any useful guesses
		* about the base, and user code expects us to save the current
		* value. Fortunately, reading the base directly is efficient.
		*/
		task->thread.fsbase = rdfsbase();
		task->thread.gsbase = __rdgsbase_inactive();
		} else {
		save_base_legacy(task, task->thread.fsindex, FS);
		save_base_legacy(task, task->thread.gsindex, GS);
		}
		}

		/*
		* While a process is running,current->thread.fsbase and current->thread.gsbase
		@@ -319,11 +329,23 @@ static __always_inline void load_seg_legacy(unsigned short prev_index,
		static __always_inline void x86_fsgsbase_load(struct thread_struct *prev,
		struct thread_struct *next)
		{
		if (static_cpu_has(X86_FEATURE_FSGSBASE)) {
		/* Update the FS and GS selectors if they could have changed. */
		if (unlikely(prev->fsindex \|\| next->fsindex))
		loadseg(FS, next->fsindex);
		if (unlikely(prev->gsindex \|\| next->gsindex))
		loadseg(GS, next->gsindex);

		/* Update the bases. */
		wrfsbase(next->fsbase);
		__wrgsbase_inactive(next->gsbase);
		} else {
		load_seg_legacy(prev->fsindex, prev->fsbase,
		next->fsindex, next->fsbase, FS);
		load_seg_legacy(prev->gsindex, prev->gsbase,
		next->gsindex, next->gsbase, GS);
		}
		}

		static unsigned long x86_fsgsbase_read_task(struct task_struct *task,
		unsigned short selector)

Admin message