Commit ca6c99c0 authored May 22, 2017 by Andy Lutomirski Committed by Ingo Molnar May 24, 2017

x86/mm: Reimplement flush_tlb_page() using flush_tlb_mm_range()



flush_tlb_page() was very similar to flush_tlb_mm_range() except that
it had a couple of issues:

 - It was missing an smp_mb() in the case where
   current->active_mm != mm.  (This is a longstanding bug reported by Nadav Amit)

 - It was missing tracepoints and vm counter updates.

The only reason that I can see for keeping it at as a separate
function is that it could avoid a few branches that
flush_tlb_mm_range() needs to decide to flush just one page.  This
hardly seems worthwhile.  If we decide we want to get rid of those
branches again, a better way would be to introduce an
__flush_tlb_mm_range() helper and make both flush_tlb_page() and
flush_tlb_mm_range() use it.

Signed-off-by: Andy Lutomirski <luto@kernel.org>
Acked-by: Kees Cook <keescook@chromium.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Borislav Petkov <bpetkov@suse.de>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Nadav Amit <nadav.amit@gmail.com>
Cc: Nadav Amit <namit@vmware.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-mm@kvack.org
Link: http://lkml.kernel.org/r/3cc3847cf888d8907577569b8bac3f01992ef8f9.1495492063.git.luto@kernel.org


Signed-off-by: Ingo Molnar <mingo@kernel.org>

parent 56fff1bb

arch/x86/include/asm/tlbflush.h

+5 −1

Original line number	Diff line number	Diff line
		@@ -307,11 +307,15 @@ static inline void flush_tlb_kernel_range(unsigned long start,
		flush_tlb_mm_range(vma->vm_mm, start, end, vma->vm_flags)

		extern void flush_tlb_all(void);
		extern void flush_tlb_page(struct vm_area_struct *, unsigned long);
		extern void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start,
		unsigned long end, unsigned long vmflag);
		extern void flush_tlb_kernel_range(unsigned long start, unsigned long end);

		static inline void flush_tlb_page(struct vm_area_struct *vma, unsigned long a)
		{
		flush_tlb_mm_range(vma->vm_mm, a, a + PAGE_SIZE, VM_NONE);
		}

		void native_flush_tlb_others(const struct cpumask *cpumask,
		struct mm_struct *mm,
		unsigned long start, unsigned long end);

arch/x86/mm/tlb.c

+0 −27

Original line number	Diff line number	Diff line
		@@ -354,33 +354,6 @@ out:
		preempt_enable();
		}

		void flush_tlb_page(struct vm_area_struct *vma, unsigned long start)
		{
		struct mm_struct *mm = vma->vm_mm;

		preempt_disable();

		if (current->active_mm == mm) {
		if (current->mm) {
		/*
		* Implicit full barrier (INVLPG) that synchronizes
		* with switch_mm.
		*/
		__flush_tlb_one(start);
		} else {
		leave_mm(smp_processor_id());

		/* Synchronize with switch_mm. */
		smp_mb();
		}
		}

		if (cpumask_any_but(mm_cpumask(mm), smp_processor_id()) < nr_cpu_ids)
		flush_tlb_others(mm_cpumask(mm), mm, start, start + PAGE_SIZE);

		preempt_enable();
		}

		static void do_flush_tlb_all(void *info)
		{
		count_vm_tlb_event(NR_TLB_REMOTE_FLUSH_RECEIVED);

Admin message