Commit 0bc40e54 authored by Linus Torvalds's avatar Linus Torvalds
Browse files

Merge branch 'x86-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 

Pull x86 mm updates from Ingo Molnar:
 "The changes in here are:

   - text_poke() fixes and an extensive set of executability lockdowns,
     to (hopefully) eliminate the last residual circumstances under
     which we are using W|X mappings even temporarily on x86 kernels.
     This required a broad range of surgery in text patching facilities,
     module loading, trampoline handling and other bits.

   - tweak page fault messages to be more informative and more
     structured.

   - remove DISCONTIGMEM support on x86-32 and make SPARSEMEM the
     default.

   - reduce KASLR granularity on 5-level paging kernels from 512 GB to
     1 GB.

   - misc other changes and updates"

* 'x86-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (36 commits)
  x86/mm: Initialize PGD cache during mm initialization
  x86/alternatives: Add comment about module removal races
  x86/kprobes: Use vmalloc special flag
  x86/ftrace: Use vmalloc special flag
  bpf: Use vmalloc special flag
  modules: Use vmalloc special flag
  mm/vmalloc: Add flag for freeing of special permsissions
  mm/hibernation: Make hibernation handle unmapped pages
  x86/mm/cpa: Add set_direct_map_*() functions
  x86/alternatives: Remove the return value of text_poke_*()
  x86/jump-label: Remove support for custom text poker
  x86/modules: Avoid breaking W^X while loading modules
  x86/kprobes: Set instruction page as executable
  x86/ftrace: Set trampoline pages as executable
  x86/kgdb: Avoid redundant comparison of patched code
  x86/alternatives: Use temporary mm for text poking
  x86/alternatives: Initialize temporary mm for patching
  fork: Provide a function for copying init_mm
  uprobes: Initialize uprobes earlier
  x86/mm: Save debug registers when loading a temporary mm
  ...
parents e913c4a4 caa84136

Documentation/x86/x86_64/mm.txt

+3 −3
Original line number Original line Diff line number Diff line
@@ -72,7 +72,7 @@ Complete virtual memory map with 5-level page tables 
Notes:

Notes:





 - With 56-bit addresses, user-space memory gets expanded by a factor of 512x,

 - With 56-bit addresses, user-space memory gets expanded by a factor of 512x,

   from 0.125 PB to 64 PB. All kernel mappings shift down to the -64 PT starting

   from 0.125 PB to 64 PB. All kernel mappings shift down to the -64 PB starting

   offset and many of the regions expand to support the much larger physical

   offset and many of the regions expand to support the much larger physical

   memory supported.

   memory supported.
@@ -83,7 +83,7 @@ Notes: 
 0000000000000000 |    0       | 00ffffffffffffff |   64 PB | user-space virtual memory, different per mm

 0000000000000000 |    0       | 00ffffffffffffff |   64 PB | user-space virtual memory, different per mm

__________________|____________|__________________|_________|___________________________________________________________

__________________|____________|__________________|_________|___________________________________________________________

                  |            |                  |         |

                  |            |                  |         |

 0000800000000000 |  +64    PB | ffff7fffffffffff | ~16K PB | ... huge, still almost 64 bits wide hole of non-canonical

 0100000000000000 |  +64    PB | feffffffffffffff | ~16K PB | ... huge, still almost 64 bits wide hole of non-canonical

                  |            |                  |         |     virtual memory addresses up to the -64 PB

                  |            |                  |         |     virtual memory addresses up to the -64 PB

                  |            |                  |         |     starting offset of kernel mappings.

                  |            |                  |         |     starting offset of kernel mappings.

__________________|____________|__________________|_________|___________________________________________________________

__________________|____________|__________________|_________|___________________________________________________________
@@ -99,7 +99,7 @@ ____________________________________________________________|___________________ 
 ffd2000000000000 |  -11.5  PB | ffd3ffffffffffff |  0.5 PB | ... unused hole

 ffd2000000000000 |  -11.5  PB | ffd3ffffffffffff |  0.5 PB | ... unused hole

 ffd4000000000000 |  -11    PB | ffd5ffffffffffff |  0.5 PB | virtual memory map (vmemmap_base)

 ffd4000000000000 |  -11    PB | ffd5ffffffffffff |  0.5 PB | virtual memory map (vmemmap_base)

 ffd6000000000000 |  -10.5  PB | ffdeffffffffffff | 2.25 PB | ... unused hole

 ffd6000000000000 |  -10.5  PB | ffdeffffffffffff | 2.25 PB | ... unused hole

 ffdf000000000000 |   -8.25 PB | fffffdffffffffff |   ~8 PB | KASAN shadow memory

 ffdf000000000000 |   -8.25 PB | fffffbffffffffff |   ~8 PB | KASAN shadow memory

__________________|____________|__________________|_________|____________________________________________________________

__________________|____________|__________________|_________|____________________________________________________________

                                                            |

                                                            |

                                                            | Identical layout to the 47-bit one from here on:

                                                            | Identical layout to the 47-bit one from here on:

arch/Kconfig

+4 −0
Original line number Original line Diff line number Diff line
@@ -249,6 +249,10 @@ config ARCH_HAS_FORTIFY_SOURCE 
config ARCH_HAS_SET_MEMORY

config ARCH_HAS_SET_MEMORY

	bool

	bool





# Select if arch has all set_direct_map_invalid/default() functions

config ARCH_HAS_SET_DIRECT_MAP

	bool



# Select if arch init_task must go in the __init_task_data section

# Select if arch init_task must go in the __init_task_data section

config ARCH_TASK_STRUCT_ON_STACK

config ARCH_TASK_STRUCT_ON_STACK

       bool

       bool

arch/x86/Kconfig

+4 −7
Original line number Original line Diff line number Diff line
@@ -65,6 +65,7 @@ config X86 
	select ARCH_HAS_UACCESS_FLUSHCACHE	if X86_64

	select ARCH_HAS_UACCESS_FLUSHCACHE	if X86_64

	select ARCH_HAS_UACCESS_MCSAFE		if X86_64 && X86_MCE

	select ARCH_HAS_UACCESS_MCSAFE		if X86_64 && X86_MCE

	select ARCH_HAS_SET_MEMORY

	select ARCH_HAS_SET_MEMORY

	select ARCH_HAS_SET_DIRECT_MAP

	select ARCH_HAS_STRICT_KERNEL_RWX

	select ARCH_HAS_STRICT_KERNEL_RWX

	select ARCH_HAS_STRICT_MODULE_RWX

	select ARCH_HAS_STRICT_MODULE_RWX

	select ARCH_HAS_SYNC_CORE_BEFORE_USERMODE

	select ARCH_HAS_SYNC_CORE_BEFORE_USERMODE
@@ -1592,12 +1593,9 @@ config ARCH_FLATMEM_ENABLE 
	depends on X86_32 && !NUMA

	depends on X86_32 && !NUMA





config ARCH_DISCONTIGMEM_ENABLE

config ARCH_DISCONTIGMEM_ENABLE

	def_bool y

	def_bool n

	depends on NUMA && X86_32



config ARCH_DISCONTIGMEM_DEFAULT

	def_bool y

	depends on NUMA && X86_32

	depends on NUMA && X86_32

	depends on BROKEN





config ARCH_SPARSEMEM_ENABLE

config ARCH_SPARSEMEM_ENABLE

	def_bool y

	def_bool y
@@ -1606,8 +1604,7 @@ config ARCH_SPARSEMEM_ENABLE 
	select SPARSEMEM_VMEMMAP_ENABLE if X86_64

	select SPARSEMEM_VMEMMAP_ENABLE if X86_64





config ARCH_SPARSEMEM_DEFAULT

config ARCH_SPARSEMEM_DEFAULT

	def_bool y

	def_bool X86_64 || (NUMA && X86_32)

	depends on X86_64





config ARCH_SELECT_MEMORY_MODEL

config ARCH_SELECT_MEMORY_MODEL

	def_bool y

	def_bool y

arch/x86/include/asm/fixmap.h

+0 −2
Original line number Original line Diff line number Diff line
@@ -103,8 +103,6 @@ enum fixed_addresses { 
#ifdef CONFIG_PARAVIRT

#ifdef CONFIG_PARAVIRT

	FIX_PARAVIRT_BOOTMAP,

	FIX_PARAVIRT_BOOTMAP,

#endif

#endif

	FIX_TEXT_POKE1,	/* reserve 2 pages for text_poke() */

	FIX_TEXT_POKE0, /* first page is last, because allocation is backward */

#ifdef	CONFIG_X86_INTEL_MID

#ifdef	CONFIG_X86_INTEL_MID

	FIX_LNW_VRTC,

	FIX_LNW_VRTC,

#endif

#endif

arch/x86/include/asm/mmu_context.h

+56 −0
Original line number Original line Diff line number Diff line
@@ -13,6 +13,7 @@ 
#include <asm/tlbflush.h>

#include <asm/tlbflush.h>

#include <asm/paravirt.h>

#include <asm/paravirt.h>

#include <asm/mpx.h>

#include <asm/mpx.h>

#include <asm/debugreg.h>





extern atomic64_t last_mm_ctx_id;

extern atomic64_t last_mm_ctx_id;
@@ -356,4 +357,59 @@ static inline unsigned long __get_current_cr3_fast(void) 
	return cr3;

	return cr3;

}

}





typedef struct {

	struct mm_struct *mm;

} temp_mm_state_t;



/*

 * Using a temporary mm allows to set temporary mappings that are not accessible

 * by other CPUs. Such mappings are needed to perform sensitive memory writes

 * that override the kernel memory protections (e.g., W^X), without exposing the

 * temporary page-table mappings that are required for these write operations to

 * other CPUs. Using a temporary mm also allows to avoid TLB shootdowns when the

 * mapping is torn down.

 *

 * Context: The temporary mm needs to be used exclusively by a single core. To

 *          harden security IRQs must be disabled while the temporary mm is

 *          loaded, thereby preventing interrupt handler bugs from overriding

 *          the kernel memory protection.

 */

static inline temp_mm_state_t use_temporary_mm(struct mm_struct *mm)

{

	temp_mm_state_t temp_state;



	lockdep_assert_irqs_disabled();

	temp_state.mm = this_cpu_read(cpu_tlbstate.loaded_mm);

	switch_mm_irqs_off(NULL, mm, current);



	/*

	 * If breakpoints are enabled, disable them while the temporary mm is

	 * used. Userspace might set up watchpoints on addresses that are used

	 * in the temporary mm, which would lead to wrong signals being sent or

	 * crashes.

	 *

	 * Note that breakpoints are not disabled selectively, which also causes

	 * kernel breakpoints (e.g., perf's) to be disabled. This might be

	 * undesirable, but still seems reasonable as the code that runs in the

	 * temporary mm should be short.

	 */

	if (hw_breakpoint_active())

		hw_breakpoint_disable();



	return temp_state;

}



static inline void unuse_temporary_mm(temp_mm_state_t prev_state)

{

	lockdep_assert_irqs_disabled();

	switch_mm_irqs_off(NULL, prev_state.mm, current);



	/*

	 * Restore the breakpoints if they were disabled before the temporary mm

	 * was loaded.

	 */

	if (hw_breakpoint_active())

		hw_breakpoint_restore();

}



#endif /* _ASM_X86_MMU_CONTEXT_H */
 
#endif /* _ASM_X86_MMU_CONTEXT_H */

CRA Git | Maintained and supported by SUSTech CRA and CCSE