Commit a493d1ca authored by Andy Lutomirski's avatar Andy Lutomirski Committed by Thomas Gleixner
Browse files

x86/membarrier: Get rid of a dubious optimization 



sync_core_before_usermode() had an incorrect optimization.  If the kernel
returns from an interrupt, it can get to usermode without IRET. It just has
to schedule to a different task in the same mm and do SYSRET.  Fortunately,
there were no callers of sync_core_before_usermode() that could have had
in_irq() or in_nmi() equal to true, because it's only ever called from the
scheduler.

While at it, clarify a related comment.

Fixes: 70216e18 ("membarrier: Provide core serializing command, *_SYNC_CORE")
Signed-off-by: default avatarAndy Lutomirski <luto@kernel.org>
Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
Reviewed-by: default avatarMathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/r/5afc7632be1422f91eaf7611aaaa1b5b8580a086.1607058304.git.luto@kernel.org
parent 0477e928

arch/x86/include/asm/sync_core.h

+5 −4
Original line number Diff line number Diff line
@@ -98,12 +98,13 @@ static inline void sync_core_before_usermode(void) 
	/* With PTI, we unconditionally serialize before running user code. */

	if (static_cpu_has(X86_FEATURE_PTI))

		return;



	/*

	 * Return from interrupt and NMI is done through iret, which is core

	 * serializing.

	 * Even if we're in an interrupt, we might reschedule before returning,

	 * in which case we could switch to a different thread in the same mm

	 * and return using SYSRET or SYSEXIT.  Instead of trying to keep

	 * track of our need to sync the core, just sync right away.

	 */

	if (in_irq() || in_nmi())

		return;

	sync_core();

}

arch/x86/mm/tlb.c

+8 −2
Original line number Diff line number Diff line
@@ -474,8 +474,14 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next, 
	/*

	 * The membarrier system call requires a full memory barrier and

	 * core serialization before returning to user-space, after

	 * storing to rq->curr. Writing to CR3 provides that full

	 * memory barrier and core serializing instruction.

	 * storing to rq->curr, when changing mm.  This is because

	 * membarrier() sends IPIs to all CPUs that are in the target mm

	 * to make them issue memory barriers.  However, if another CPU

	 * switches to/from the target mm concurrently with

	 * membarrier(), it can cause that CPU not to receive an IPI

	 * when it really should issue a memory barrier.  Writing to CR3

	 * provides that full memory barrier and core serializing

	 * instruction.

	 */

	if (real_prev == next) {

		VM_WARN_ON(this_cpu_read(cpu_tlbstate.ctxs[prev_asid].ctx_id) !=

CRA Git | Maintained and supported by SUSTech CRA and CCSE