mm/mmu_gather: invalidate TLB correctly on batch allocation failure and flush (0ed13259) · Commits · 戴 / test

Architectures for which we have hardware walkers of Linux page table should flush TLB on mmu gather batch allocation failures and batch flush. Some architectures like POWER supports multiple translation modes (hash and radix) and in the case of POWER only radix translation mode needs the above TLBI. This is because for hash translation mode kernel wants to avoid this extra flush since there are no hardware walkers of linux page table. With radix translation, the hardware also walks linux page table and with that, kernel needs to make sure to TLB invalidate page walk cache before page table pages are freed. More details in commit ("mm/tlb, x86/mm: Support invalidating TLB caches for RCU_TABLE_FREE") The changes to sparc are to make sure we keep the old behavior since we are now removing HAVE_RCU_TABLE_NO_INVALIDATE. The default value for tlb_needs_table_invalidate is to always force an invalidate and sparc can avoid the table invalidate. Hence we define tlb_needs_table_invalidate to false for sparc architecture. Link: http://lkml.kernel.org/r/20200116064531.483522-3-aneesh.kumar@linux.ibm.com Fixes: ("powerpc/mm/radix: Improve TLB/PWC flushes") Signed-off-by:

Peter Zijlstra (Intel) <peterz@infradead.org> Signed-off-by:

Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com> Acked-by: Michael Ellerman <mpe@ellerman.id.au> [powerpc] Cc: <stable@vger.kernel.org> [4.14+] Signed-off-by:

Andrew Morton <akpm@linux-foundation.org> Signed-off-by:

Linus Torvalds <torvalds@linux-foundation.org>

arch/Kconfig

+0 −3

Original line number	Diff line number	Diff line
		@@ -396,9 +396,6 @@ config HAVE_ARCH_JUMP_LABEL_RELATIVE
		config HAVE_RCU_TABLE_FREE
		bool

		config HAVE_RCU_TABLE_NO_INVALIDATE
		bool

		config HAVE_MMU_GATHER_PAGE_SIZE
		bool

arch/powerpc/Kconfig

+0 −1

Original line number	Diff line number	Diff line
		@@ -223,7 +223,6 @@ config PPC
		select HAVE_PERF_REGS
		select HAVE_PERF_USER_STACK_DUMP
		select HAVE_RCU_TABLE_FREE
		select HAVE_RCU_TABLE_NO_INVALIDATE if HAVE_RCU_TABLE_FREE
		select HAVE_MMU_GATHER_PAGE_SIZE
		select HAVE_REGS_AND_STACK_ACCESS_API
		select HAVE_RELIABLE_STACKTRACE if PPC_BOOK3S_64 && CPU_LITTLE_ENDIAN

arch/powerpc/include/asm/tlb.h

+11 −0

Original line number	Diff line number	Diff line
		@@ -26,6 +26,17 @@

		#define tlb_flush tlb_flush
		extern void tlb_flush(struct mmu_gather *tlb);
		/*
		* book3s:
		* Hash does not use the linux page-tables, so we can avoid
		* the TLB invalidate for page-table freeing, Radix otoh does use the
		* page-tables and needs the TLBI.
		*
		* nohash:
		* We still do TLB invalidate in the __pte_free_tlb routine before we
		* add the page table pages to mmu gather table batch.
		*/
		#define tlb_needs_table_invalidate() radix_enabled()

		/* Get the generic bits... */
		#include <asm-generic/tlb.h>

arch/sparc/Kconfig

+0 −1

Original line number	Diff line number	Diff line
		@@ -65,7 +65,6 @@ config SPARC64
		select HAVE_KRETPROBES
		select HAVE_KPROBES
		select HAVE_RCU_TABLE_FREE if SMP
		select HAVE_RCU_TABLE_NO_INVALIDATE if HAVE_RCU_TABLE_FREE
		select HAVE_MEMBLOCK_NODE_MAP
		select HAVE_ARCH_TRANSPARENT_HUGEPAGE
		select HAVE_DYNAMIC_FTRACE

arch/sparc/include/asm/tlb_64.h

+9 −0

Original line number	Diff line number	Diff line
		@@ -28,6 +28,15 @@ void flush_tlb_pending(void);
		#define __tlb_remove_tlb_entry(tlb, ptep, address) do { } while (0)
		#define tlb_flush(tlb) flush_tlb_pending()

		/*
		* SPARC64's hardware TLB fill does not use the Linux page-tables
		* and therefore we don't need a TLBI when freeing page-table pages.
		*/

		#ifdef CONFIG_HAVE_RCU_TABLE_FREE
		#define tlb_needs_table_invalidate() (false)
		#endif

		#include <asm-generic/tlb.h>

		#endif /* _SPARC64_TLB_H */

Admin message