Commit 99792e0c authored by Linus Torvalds's avatar Linus Torvalds
Browse files

Merge branch 'x86-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 

Pull x86 mm updates from Ingo Molnar:
 "Lots of changes in this cycle:

   - Lots of CPA (change page attribute) optimizations and related
     cleanups (Thomas Gleixner, Peter Zijstra)

   - Make lazy TLB mode even lazier (Rik van Riel)

   - Fault handler cleanups and improvements (Dave Hansen)

   - kdump, vmcore: Enable kdumping encrypted memory with AMD SME
     enabled (Lianbo Jiang)

   - Clean up VM layout documentation (Baoquan He, Ingo Molnar)

   - ... plus misc other fixes and enhancements"

* 'x86-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (51 commits)
  x86/stackprotector: Remove the call to boot_init_stack_canary() from cpu_startup_entry()
  x86/mm: Kill stray kernel fault handling comment
  x86/mm: Do not warn about PCI BIOS W+X mappings
  resource: Clean it up a bit
  resource: Fix find_next_iomem_res() iteration issue
  resource: Include resource end in walk_*() interfaces
  x86/kexec: Correct KEXEC_BACKUP_SRC_END off-by-one error
  x86/mm: Remove spurious fault pkey check
  x86/mm/vsyscall: Consider vsyscall page part of user address space
  x86/mm: Add vsyscall address helper
  x86/mm: Fix exception table comments
  x86/mm: Add clarifying comments for user addr space
  x86/mm: Break out user address space handling
  x86/mm: Break out kernel address space handling
  x86/mm: Clarify hardware vs. software "error_code"
  x86/mm/tlb: Make lazy TLB mode lazier
  x86/mm/tlb: Add freed_tables element to flush_tlb_info
  x86/mm/tlb: Add freed_tables argument to flush_tlb_mm_range
  smp,cpumask: introduce on_each_cpu_cond_mask
  smp: use __cpumask_set_cpu in on_each_cpu_cond
  ...
parents 382d72a9 977e4be5

Documentation/x86/x86_64/mm.txt

+120 −51
Original line number Diff line number Diff line 
====================================================

Complete virtual memory map with 4-level page tables

====================================================



Virtual memory map with 4 level page tables:



0000000000000000 - 00007fffffffffff (=47 bits) user space, different per mm

hole caused by [47:63] sign extension

ffff800000000000 - ffff87ffffffffff (=43 bits) guard hole, reserved for hypervisor

ffff880000000000 - ffffc7ffffffffff (=64 TB) direct mapping of all phys. memory

ffffc80000000000 - ffffc8ffffffffff (=40 bits) hole

ffffc90000000000 - ffffe8ffffffffff (=45 bits) vmalloc/ioremap space

ffffe90000000000 - ffffe9ffffffffff (=40 bits) hole

ffffea0000000000 - ffffeaffffffffff (=40 bits) virtual memory map (1TB)

... unused hole ...

ffffec0000000000 - fffffbffffffffff (=44 bits) kasan shadow memory (16TB)

... unused hole ...

				    vaddr_end for KASLR

fffffe0000000000 - fffffe7fffffffff (=39 bits) cpu_entry_area mapping

fffffe8000000000 - fffffeffffffffff (=39 bits) LDT remap for PTI

ffffff0000000000 - ffffff7fffffffff (=39 bits) %esp fixup stacks

... unused hole ...

ffffffef00000000 - fffffffeffffffff (=64 GB) EFI region mapping space

... unused hole ...

ffffffff80000000 - ffffffff9fffffff (=512 MB)  kernel text mapping, from phys 0

ffffffffa0000000 - fffffffffeffffff (1520 MB) module mapping space

[fixmap start]   - ffffffffff5fffff kernel-internal fixmap range

ffffffffff600000 - ffffffffff600fff (=4 kB) legacy vsyscall ABI

ffffffffffe00000 - ffffffffffffffff (=2 MB) unused hole



Virtual memory map with 5 level page tables:



0000000000000000 - 00ffffffffffffff (=56 bits) user space, different per mm

hole caused by [56:63] sign extension

ff00000000000000 - ff0fffffffffffff (=52 bits) guard hole, reserved for hypervisor

ff10000000000000 - ff8fffffffffffff (=55 bits) direct mapping of all phys. memory

ff90000000000000 - ff9fffffffffffff (=52 bits) LDT remap for PTI

ffa0000000000000 - ffd1ffffffffffff (=54 bits) vmalloc/ioremap space (12800 TB)

ffd2000000000000 - ffd3ffffffffffff (=49 bits) hole

ffd4000000000000 - ffd5ffffffffffff (=49 bits) virtual memory map (512TB)

... unused hole ...

ffdf000000000000 - fffffc0000000000 (=53 bits) kasan shadow memory (8PB)

... unused hole ...

				    vaddr_end for KASLR

fffffe0000000000 - fffffe7fffffffff (=39 bits) cpu_entry_area mapping

... unused hole ...

ffffff0000000000 - ffffff7fffffffff (=39 bits) %esp fixup stacks

... unused hole ...

ffffffef00000000 - fffffffeffffffff (=64 GB) EFI region mapping space

... unused hole ...

ffffffff80000000 - ffffffff9fffffff (=512 MB)  kernel text mapping, from phys 0

ffffffffa0000000 - fffffffffeffffff (1520 MB) module mapping space

[fixmap start]   - ffffffffff5fffff kernel-internal fixmap range

ffffffffff600000 - ffffffffff600fff (=4 kB) legacy vsyscall ABI

ffffffffffe00000 - ffffffffffffffff (=2 MB) unused hole

Notes:



 - Negative addresses such as "-23 TB" are absolute addresses in bytes, counted down

   from the top of the 64-bit address space. It's easier to understand the layout

   when seen both in absolute addresses and in distance-from-top notation.



   For example 0xffffe90000000000 == -23 TB, it's 23 TB lower than the top of the

   64-bit address space (ffffffffffffffff).



   Note that as we get closer to the top of the address space, the notation changes

   from TB to GB and then MB/KB.



 - "16M TB" might look weird at first sight, but it's an easier to visualize size

   notation than "16 EB", which few will recognize at first sight as 16 exabytes.

   It also shows it nicely how incredibly large 64-bit address space is.



========================================================================================================================

    Start addr    |   Offset   |     End addr     |  Size   | VM area description

========================================================================================================================

                  |            |                  |         |

 0000000000000000 |    0       | 00007fffffffffff |  128 TB | user-space virtual memory, different per mm

__________________|____________|__________________|_________|___________________________________________________________

                  |            |                  |         |

 0000800000000000 | +128    TB | ffff7fffffffffff | ~16M TB | ... huge, almost 64 bits wide hole of non-canonical

                  |            |                  |         |     virtual memory addresses up to the -128 TB

                  |            |                  |         |     starting offset of kernel mappings.

__________________|____________|__________________|_________|___________________________________________________________

                                                            |

                                                            | Kernel-space virtual memory, shared between all processes:

____________________________________________________________|___________________________________________________________

                  |            |                  |         |

 ffff800000000000 | -128    TB | ffff87ffffffffff |    8 TB | ... guard hole, also reserved for hypervisor

 ffff880000000000 | -120    TB | ffffc7ffffffffff |   64 TB | direct mapping of all physical memory (page_offset_base)

 ffffc80000000000 |  -56    TB | ffffc8ffffffffff |    1 TB | ... unused hole

 ffffc90000000000 |  -55    TB | ffffe8ffffffffff |   32 TB | vmalloc/ioremap space (vmalloc_base)

 ffffe90000000000 |  -23    TB | ffffe9ffffffffff |    1 TB | ... unused hole

 ffffea0000000000 |  -22    TB | ffffeaffffffffff |    1 TB | virtual memory map (vmemmap_base)

 ffffeb0000000000 |  -21    TB | ffffebffffffffff |    1 TB | ... unused hole

 ffffec0000000000 |  -20    TB | fffffbffffffffff |   16 TB | KASAN shadow memory

 fffffc0000000000 |   -4    TB | fffffdffffffffff |    2 TB | ... unused hole

                  |            |                  |         | vaddr_end for KASLR

 fffffe0000000000 |   -2    TB | fffffe7fffffffff |  0.5 TB | cpu_entry_area mapping

 fffffe8000000000 |   -1.5  TB | fffffeffffffffff |  0.5 TB | LDT remap for PTI

 ffffff0000000000 |   -1    TB | ffffff7fffffffff |  0.5 TB | %esp fixup stacks

__________________|____________|__________________|_________|____________________________________________________________

                                                            |

                                                            | Identical layout to the 47-bit one from here on:

____________________________________________________________|____________________________________________________________

                  |            |                  |         |

 ffffff8000000000 | -512    GB | ffffffeeffffffff |  444 GB | ... unused hole

 ffffffef00000000 |  -68    GB | fffffffeffffffff |   64 GB | EFI region mapping space

 ffffffff00000000 |   -4    GB | ffffffff7fffffff |    2 GB | ... unused hole

 ffffffff80000000 |   -2    GB | ffffffff9fffffff |  512 MB | kernel text mapping, mapped to physical address 0

 ffffffff80000000 |-2048    MB |                  |         |

 ffffffffa0000000 |-1536    MB | fffffffffeffffff | 1520 MB | module mapping space

 ffffffffff000000 |  -16    MB |                  |         |

    FIXADDR_START | ~-11    MB | ffffffffff5fffff | ~0.5 MB | kernel-internal fixmap range, variable size and offset

 ffffffffff600000 |  -10    MB | ffffffffff600fff |    4 kB | legacy vsyscall ABI

 ffffffffffe00000 |   -2    MB | ffffffffffffffff |    2 MB | ... unused hole

__________________|____________|__________________|_________|___________________________________________________________





====================================================

Complete virtual memory map with 5-level page tables

====================================================



Notes:



 - With 56-bit addresses, user-space memory gets expanded by a factor of 512x,

   from 0.125 PB to 64 PB. All kernel mappings shift down to the -64 PT starting

   offset and many of the regions expand to support the much larger physical

   memory supported.



========================================================================================================================

    Start addr    |   Offset   |     End addr     |  Size   | VM area description

========================================================================================================================

                  |            |                  |         |

 0000000000000000 |    0       | 00ffffffffffffff |   64 PB | user-space virtual memory, different per mm

__________________|____________|__________________|_________|___________________________________________________________

                  |            |                  |         |

 0000800000000000 |  +64    PB | ffff7fffffffffff | ~16K PB | ... huge, still almost 64 bits wide hole of non-canonical

                  |            |                  |         |     virtual memory addresses up to the -128 TB

                  |            |                  |         |     starting offset of kernel mappings.

__________________|____________|__________________|_________|___________________________________________________________

                                                            |

                                                            | Kernel-space virtual memory, shared between all processes:

____________________________________________________________|___________________________________________________________

                  |            |                  |         |

 ff00000000000000 |  -64    PB | ff0fffffffffffff |    4 PB | ... guard hole, also reserved for hypervisor

 ff10000000000000 |  -60    PB | ff8fffffffffffff |   32 PB | direct mapping of all physical memory (page_offset_base)

 ff90000000000000 |  -28    PB | ff9fffffffffffff |    4 PB | LDT remap for PTI

 ffa0000000000000 |  -24    PB | ffd1ffffffffffff | 12.5 PB | vmalloc/ioremap space (vmalloc_base)

 ffd2000000000000 |  -11.5  PB | ffd3ffffffffffff |  0.5 PB | ... unused hole

 ffd4000000000000 |  -11    PB | ffd5ffffffffffff |  0.5 PB | virtual memory map (vmemmap_base)

 ffd6000000000000 |  -10.5  PB | ffdeffffffffffff | 2.25 PB | ... unused hole

 ffdf000000000000 |   -8.25 PB | fffffdffffffffff |   ~8 PB | KASAN shadow memory

 fffffc0000000000 |   -4    TB | fffffdffffffffff |    2 TB | ... unused hole

                  |            |                  |         | vaddr_end for KASLR

 fffffe0000000000 |   -2    TB | fffffe7fffffffff |  0.5 TB | cpu_entry_area mapping

 fffffe8000000000 |   -1.5  TB | fffffeffffffffff |  0.5 TB | ... unused hole

 ffffff0000000000 |   -1    TB | ffffff7fffffffff |  0.5 TB | %esp fixup stacks

__________________|____________|__________________|_________|____________________________________________________________

                                                            |

                                                            | Identical layout to the 47-bit one from here on:

____________________________________________________________|____________________________________________________________

                  |            |                  |         |

 ffffff8000000000 | -512    GB | ffffffeeffffffff |  444 GB | ... unused hole

 ffffffef00000000 |  -68    GB | fffffffeffffffff |   64 GB | EFI region mapping space

 ffffffff00000000 |   -4    GB | ffffffff7fffffff |    2 GB | ... unused hole

 ffffffff80000000 |   -2    GB | ffffffff9fffffff |  512 MB | kernel text mapping, mapped to physical address 0

 ffffffff80000000 |-2048    MB |                  |         |

 ffffffffa0000000 |-1536    MB | fffffffffeffffff | 1520 MB | module mapping space

 ffffffffff000000 |  -16    MB |                  |         |

    FIXADDR_START | ~-11    MB | ffffffffff5fffff | ~0.5 MB | kernel-internal fixmap range, variable size and offset

 ffffffffff600000 |  -10    MB | ffffffffff600fff |    4 kB | legacy vsyscall ABI

 ffffffffffe00000 |   -2    MB | ffffffffffffffff |    2 MB | ... unused hole

__________________|____________|__________________|_________|___________________________________________________________



Architecture defines a 64-bit virtual address. Implementations can support

less. Currently supported are 48- and 57-bit virtual addresses. Bits 63

arch/x86/Kconfig

+8 −0
Original line number Diff line number Diff line
@@ -1487,6 +1487,14 @@ config X86_DIRECT_GBPAGES 
	  supports them), so don't confuse the user by printing

	  that we have them enabled.



config X86_CPA_STATISTICS

	bool "Enable statistic for Change Page Attribute"

	depends on DEBUG_FS

	---help---

	  Expose statistics about the Change Page Attribute mechanims, which

	  helps to determine the effectivness of preserving large and huge

	  page mappings when mapping protections are changed.



config ARCH_HAS_MEM_ENCRYPT

	def_bool y

arch/x86/include/asm/io.h

+2 −1
Original line number Diff line number Diff line
@@ -187,11 +187,12 @@ extern void __iomem *ioremap_nocache(resource_size_t offset, unsigned long size) 
#define ioremap_nocache ioremap_nocache

extern void __iomem *ioremap_uc(resource_size_t offset, unsigned long size);

#define ioremap_uc ioremap_uc



extern void __iomem *ioremap_cache(resource_size_t offset, unsigned long size);

#define ioremap_cache ioremap_cache

extern void __iomem *ioremap_prot(resource_size_t offset, unsigned long size, unsigned long prot_val);

#define ioremap_prot ioremap_prot

extern void __iomem *ioremap_encrypted(resource_size_t phys_addr, unsigned long size);

#define ioremap_encrypted ioremap_encrypted



/**

 * ioremap     -   map bus memory into CPU space

arch/x86/include/asm/kexec.h

+1 −1
Original line number Diff line number Diff line
@@ -67,7 +67,7 @@ struct kimage; 


/* Memory to backup during crash kdump */

#define KEXEC_BACKUP_SRC_START	(0UL)

#define KEXEC_BACKUP_SRC_END	(640 * 1024UL)	/* 640K */

#define KEXEC_BACKUP_SRC_END	(640 * 1024UL - 1)	/* 640K */



/*

 * CPU does not save ss and sp on stack if execution is already

arch/x86/include/asm/page_64_types.h

+9 −6
Original line number Diff line number Diff line
@@ -59,13 +59,16 @@ 
#endif



/*

 * Kernel image size is limited to 1GiB due to the fixmap living in the

 * next 1GiB (see level2_kernel_pgt in arch/x86/kernel/head_64.S). Use

 * 512MiB by default, leaving 1.5GiB for modules once the page tables

 * are fully set up. If kernel ASLR is configured, it can extend the

 * kernel page table mapping, reducing the size of the modules area.

 * Maximum kernel image size is limited to 1 GiB, due to the fixmap living

 * in the next 1 GiB (see level2_kernel_pgt in arch/x86/kernel/head_64.S).

 *

 * On KASLR use 1 GiB by default, leaving 1 GiB for modules once the

 * page tables are fully set up.

 *

 * If KASLR is disabled we can shrink it to 0.5 GiB and increase the size

 * of the modules area to 1.5 GiB.

 */

#if defined(CONFIG_RANDOMIZE_BASE)

#ifdef CONFIG_RANDOMIZE_BASE

#define KERNEL_IMAGE_SIZE	(1024 * 1024 * 1024)

#else

#define KERNEL_IMAGE_SIZE	(512 * 1024 * 1024)

CRA Git | Maintained and supported by SUSTech CRA and CCSE