Merge tag 'powerpc-5.4-1' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux

			  or C0.2 state. The time is an unsigned 32-bit number.

			  Note that a value of zero means there is no limit.

			  Low order two bits must be zero.

What:		/sys/devices/system/cpu/svm

Date:		August 2019

Contact:	Linux kernel mailing list <linux-kernel@vger.kernel.org>

		Linux for PowerPC mailing list <linuxppc-dev@ozlabs.org>

Description:	Secure Virtual Machine

		If 1, it means the system is using the Protected Execution

		Facility in POWER9 and newer processors. i.e., it is a Secure

		Virtual Machine.

	disable_radix	[PPC]

			Disable RADIX MMU mode on POWER9

	disable_tlbie	[PPC]

			Disable TLBIE instruction. Currently does not work

			with KVM, with HASH MMU, or with coherent accelerators.

	disable_cpu_apicid= [X86,APIC,SMP]

			Format: <int>

			The number of initial APIC ID for the

			/sys/power/pm_test). Only available when CONFIG_PM_DEBUG

			is set. Default value is 5.

	svm=		[PPC]

			Format: { on | off | y | n | 1 | 0 }

			This parameter controls use of the Protected

			Execution Facility on pSeries.

	swapaccount=[0|1]

			[KNL] Enable accounting of swap in memory resource

			controller if no parameter or 1 is given or disable

			A hex value specifying bitmask with supplemental xhci

			host controller quirks. Meaning of each bit can be

			consulted in header drivers/usb/host/xhci.h.

	xmon		[PPC]

			Format: { early | on | rw | ro | off }

			Controls if xmon debugger is enabled. Default is off.

			Passing only "xmon" is equivalent to "xmon=early".

			early	Call xmon as early as possible on boot; xmon

				debugger is called from setup_arch().

			on	xmon debugger hooks will be installed so xmon

				is only called on a kernel crash. Default mode,

				i.e. either "ro" or "rw" mode, is controlled

				with CONFIG_XMON_DEFAULT_RO_MODE.

			rw	xmon debugger hooks will be installed so xmon

				is called only on a kernel crash, mode is write,

				meaning SPR registers, memory and, other data

				can be written using xmon commands.

			ro 	same as "rw" option above but SPR registers,

				memory, and other data can't be written using

				xmon commands.

			off	xmon is disabled.

==========================

ELF Note PowerPC Namespace

==========================

The PowerPC namespace in an ELF Note of the kernel binary is used to store

capabilities and information which can be used by a bootloader or userland.

Types and Descriptors

---------------------

The types to be used with the "PowerPC" namesapce are defined in [#f1]_.

	1) PPC_ELFNOTE_CAPABILITIES

Define the capabilities supported/required by the kernel. This type uses a

bitmap as "descriptor" field. Each bit is described below:

- Ultravisor-capable bit (PowerNV only).

.. code-block:: c

	#define PPCCAP_ULTRAVISOR_BIT (1 << 0)

Indicate that the powerpc kernel binary knows how to run in an

ultravisor-enabled system.

In an ultravisor-enabled system, some machine resources are now controlled

by the ultravisor. If the kernel is not ultravisor-capable, but it ends up

being run on a machine with ultravisor, the kernel will probably crash

trying to access ultravisor resources. For instance, it may crash in early

boot trying to set the partition table entry 0.

In an ultravisor-enabled system, a bootloader could warn the user or prevent

the kernel from being run if the PowerPC ultravisor capability doesn't exist

or the Ultravisor-capable bit is not set.

References

----------

.. [#f1] arch/powerpc/include/asm/elfnote.h

to minimize the total elapsed time until the system is back

in production use.

- Firmware assisted dump (fadump) infrastructure is intended to replace

- Firmware-Assisted Dump (FADump) infrastructure is intended to replace

  the existing phyp assisted dump.

- Fadump uses the same firmware interfaces and memory reservation model

  as phyp assisted dump.

- Unlike phyp dump, fadump exports the memory dump through /proc/vmcore

- Unlike phyp dump, FADump exports the memory dump through /proc/vmcore

  in the ELF format in the same way as kdump. This helps us reuse the

  kdump infrastructure for dump capture and filtering.

- Unlike phyp dump, userspace tool does not need to refer any sysfs

  interface while reading /proc/vmcore.

- Unlike phyp dump, fadump allows user to release all the memory reserved

- Unlike phyp dump, FADump allows user to release all the memory reserved

  for dump, with a single operation of echo 1 > /sys/kernel/fadump_release_mem.

- Once enabled through kernel boot parameter, fadump can be

- Once enabled through kernel boot parameter, FADump can be

  started/stopped through /sys/kernel/fadump_registered interface (see

  sysfs files section below) and can be easily integrated with kdump

  service start/stop init scripts.

   in a clean, consistent state.

-  Once the dump is copied out, the memory that held the dump

   is immediately available to the running kernel. And therefore,

   unlike kdump, fadump doesn't need a 2nd reboot to get back

   unlike kdump, FADump doesn't need a 2nd reboot to get back

   the system to the production configuration.

The above can only be accomplished by coordination with,

   These registered sections of memory are reserved by the first

   kernel during early boot.

-  When a system crashes, the Power firmware will save

   the low memory (boot memory of size larger of 5% of system RAM

   or 256MB) of RAM to the previous registered region. It will

   also save system registers, and hardware PTE's.

-  When system crashes, the Power firmware will copy the registered

   low memory regions (boot memory) from source to destination area.

   It will also save hardware PTE's.

   NOTE:

         The term 'boot memory' means size of the low memory chunk

         the default calculated size. Use this option if default

         boot memory size is not sufficient for second kernel to

         boot successfully. For syntax of crashkernel= parameter,

         refer to Documentation/admin-guide/kdump/kdump.rst. If any offset is

         provided in crashkernel= parameter, it will be ignored

         as fadump uses a predefined offset to reserve memory

         refer to Documentation/admin-guide/kdump/kdump.rst. If any

         offset is provided in crashkernel= parameter, it will be

         ignored as FADump uses a predefined offset to reserve memory

         for boot memory dump preservation in case of a crash.

-  After the low memory (boot memory) area has been saved, the

   *not* clear the RAM. It will then launch the bootloader, as

   normal.

-  The freshly booted kernel will notice that there is a new

   node (ibm,dump-kernel) in the device tree, indicating that

-  The freshly booted kernel will notice that there is a new node

   (rtas/ibm,kernel-dump on pSeries or ibm,opal/dump/mpipl-boot

   on OPAL platform) in the device tree, indicating that

   there is crash data available from a previous boot. During

   the early boot OS will reserve rest of the memory above

   boot memory size effectively booting with restricted memory

   size. This will make sure that the second kernel will not

   touch any of the dump memory area.

   size. This will make sure that this kernel (also, referred

   to as second kernel or capture kernel) will not touch any

   of the dump memory area.

-  User-space tools will read /proc/vmcore to obtain the contents

   of memory, which holds the previous crashed kernel dump in ELF

     # echo 1 > /sys/kernel/fadump_release_mem

Please note that the firmware-assisted dump feature

is only available on Power6 and above systems with recent

firmware versions.

is only available on POWER6 and above systems on pSeries

(PowerVM) platform and POWER9 and above systems with OP940

or later firmware versions on PowerNV (OPAL) platform.

Note that, OPAL firmware exports ibm,opal/dump node when

FADump is supported on PowerNV platform.

On OPAL based machines, system first boots into an intermittent

kernel (referred to as petitboot kernel) before booting into the

capture kernel. This kernel would have minimal kernel and/or

userspace support to process crash data. Such kernel needs to

preserve previously crash'ed kernel's memory for the subsequent

capture kernel boot to process this crash data. Kernel config

option CONFIG_PRESERVE_FA_DUMP has to be enabled on such kernel

to ensure that crash data is preserved to process later.

-- On OPAL based machines (PowerNV), if the kernel is build with

   CONFIG_OPAL_CORE=y, OPAL memory at the time of crash is also

   exported as /sys/firmware/opal/core file. This procfs file is

   helpful in debugging OPAL crashes with GDB. The kernel memory

   used for exporting this procfs file can be released by echo'ing

   '1' to /sys/kernel/fadump_release_opalcore node.

   e.g.

     # echo 1 > /sys/kernel/fadump_release_opalcore

Implementation details:

-----------------------

/sys/kernel/fadump_release_mem file is created, and the reserved

memory is held.

If there is no waiting dump data, then only the memory required

to hold CPU state, HPTE region, boot memory dump and elfcore

header, is usually reserved at an offset greater than boot memory

size (see Fig. 1). This area is *not* released: this region will

be kept permanently reserved, so that it can act as a receptacle

for a copy of the boot memory content in addition to CPU state

and HPTE region, in the case a crash does occur. Since this reserved

memory area is used only after the system crash, there is no point in

blocking this significant chunk of memory from production kernel.

Hence, the implementation uses the Linux kernel's Contiguous Memory

Allocator (CMA) for memory reservation if CMA is configured for kernel.

With CMA reservation this memory will be available for applications to

use it, while kernel is prevented from using it. With this fadump will

still be able to capture all of the kernel memory and most of the user

space memory except the user pages that were present in CMA region::

If there is no waiting dump data, then only the memory required to

hold CPU state, HPTE region, boot memory dump, FADump header and

elfcore header, is usually reserved at an offset greater than boot

memory size (see Fig. 1). This area is *not* released: this region

will be kept permanently reserved, so that it can act as a receptacle

for a copy of the boot memory content in addition to CPU state and

HPTE region, in the case a crash does occur.

Since this reserved memory area is used only after the system crash,

there is no point in blocking this significant chunk of memory from

production kernel. Hence, the implementation uses the Linux kernel's

Contiguous Memory Allocator (CMA) for memory reservation if CMA is

configured for kernel. With CMA reservation this memory will be

available for applications to use it, while kernel is prevented from

using it. With this FADump will still be able to capture all of the

kernel memory and most of the user space memory except the user pages

that were present in CMA region::

  o Memory Reservation during first kernel

  Low memory                                                 Top of memory

  0      boot memory size                                       |

  |           |                |<--Reserved dump area -->|      |

  V           V                |   Permanent Reservation |      V

  +-----------+----------/ /---+---+----+-----------+----+------+

  |           |                |CPU|HPTE|  DUMP     |ELF |      |

  +-----------+----------/ /---+---+----+-----------+----+------+

        |                                           ^

        |                                           |

        \                                           /

         -------------------------------------------

          Boot memory content gets transferred to

          reserved area by firmware at the time of

          crash

  0    boot memory size   |<--- Reserved dump area --->|       |

  |           |           |    Permanent Reservation   |       |

  V           V           |                            |       V

  +-----------+-----/ /---+---+----+-------+-----+-----+----+--+

  |           |           |///|////|  DUMP | HDR | ELF |////|  |

  +-----------+-----/ /---+---+----+-------+-----+-----+----+--+

        |                   ^    ^     ^      ^           ^

        |                   |    |     |      |           |

        \                  CPU  HPTE   /      |           |

         ------------------------------       |           |

      Boot memory content gets transferred    |           |

      to reserved area by firmware at the     |           |

      time of crash.                          |           |

                                          FADump Header   |

                                           (meta area)    |

                                                          |

                                                          |

                      Metadata: This area holds a metadata struture whose

                      address is registered with f/w and retrieved in the

                      second kernel after crash, on platforms that support

                      tags (OPAL). Having such structure with info needed

                      to process the crashdump eases dump capture process.

                   Fig. 1

  o Memory Reservation during second kernel after crash

  Low memory                                              Top of memory

  0      boot memory size                                      |

  |           |<------------- Reserved dump area ----------- -->|

  V           V                                                 V

  +-----------+----------/ /---+---+----+-----------+----+------+

  |           |                |CPU|HPTE|  DUMP     |ELF |      |

  +-----------+----------/ /---+---+----+-----------+----+------+

  |           |<------------ Crash preserved area ------------>|

  V           V           |<--- Reserved dump area --->|       |

  +-----------+-----/ /---+---+----+-------+-----+-----+----+--+

  |           |           |///|////|  DUMP | HDR | ELF |////|  |

  +-----------+-----/ /---+---+----+-------+-----+-----+----+--+

        |                                           |

        V                                           V

   Used by second                             /proc/vmcore

   kernel to boot

        +---+

        |///| -> Regions (CPU, HPTE & Metadata) marked like this in the above

        +---+    figures are not always present. For example, OPAL platform

                 does not have CPU & HPTE regions while Metadata region is

                 not supported on pSeries currently.

                   Fig. 2

Currently the dump will be copied from /proc/vmcore to a

a new file upon user intervention. The dump data available through

/proc/vmcore will be in ELF format. Hence the existing kdump

infrastructure (kdump scripts) to save the dump works fine with

minor modifications.

Currently the dump will be copied from /proc/vmcore to a new file upon

user intervention. The dump data available through /proc/vmcore will be

in ELF format. Hence the existing kdump infrastructure (kdump scripts)

to save the dump works fine with minor modifications. KDump scripts on

major Distro releases have already been modified to work seemlessly (no

user intervention in saving the dump) when FADump is used, instead of

KDump, as dump mechanism.

The tools to examine the dump will be same as the ones

used for kdump.

How to enable firmware-assisted dump (fadump):

How to enable firmware-assisted dump (FADump):

----------------------------------------------

1. Set config option CONFIG_FA_DUMP=y and build kernel.

2. Boot into linux kernel with 'fadump=on' kernel cmdline option.

   By default, fadump reserved memory will be initialized as CMA area.

   By default, FADump reserved memory will be initialized as CMA area.

   Alternatively, user can boot linux kernel with 'fadump=nocma' to

   prevent fadump to use CMA.

   prevent FADump to use CMA.

3. Optionally, user can also set 'crashkernel=' kernel cmdline

   to specify size of the memory to reserve for boot memory dump

   preservation.

Here is the list of files under kernel sysfs:

 /sys/kernel/fadump_enabled

    This is used to display the fadump status.

    This is used to display the FADump status.

    - 0 = fadump is disabled

    - 1 = fadump is enabled

    - 0 = FADump is disabled

    - 1 = FADump is enabled

    This interface can be used by kdump init scripts to identify if

    fadump is enabled in the kernel and act accordingly.

    FADump is enabled in the kernel and act accordingly.

 /sys/kernel/fadump_registered

    This is used to display the fadump registration status as well

    as to control (start/stop) the fadump registration.

    This is used to display the FADump registration status as well

    as to control (start/stop) the FADump registration.

    - 0 = fadump is not registered.

    - 1 = fadump is registered and ready to handle system crash.

    - 0 = FADump is not registered.

    - 1 = FADump is registered and ready to handle system crash.

    To register fadump echo 1 > /sys/kernel/fadump_registered and

    To register FADump echo 1 > /sys/kernel/fadump_registered and

    echo 0 > /sys/kernel/fadump_registered for un-register and stop the

    fadump. Once the fadump is un-registered, the system crash will not

    FADump. Once the FADump is un-registered, the system crash will not

    be handled and vmcore will not be captured. This interface can be

    easily integrated with kdump service start/stop.

 /sys/kernel/fadump_release_mem

    This file is available only when fadump is active during

    This file is available only when FADump is active during

    second kernel. This is used to release the reserved memory

    region that are held for saving crash dump. To release the

    reserved memory echo 1 to it::

    enhanced to use this interface to release the memory reserved for

    dump and continue without 2nd reboot.

 /sys/kernel/fadump_release_opalcore

    This file is available only on OPAL based machines when FADump is

    active during capture kernel. This is used to release the memory

    used by the kernel to export /sys/firmware/opal/core file. To

    release this memory, echo '1' to it:

    echo 1  > /sys/kernel/fadump_release_opalcore

Here is the list of files under powerpc debugfs:

(Assuming debugfs is mounted on /sys/kernel/debug directory.)

 /sys/kernel/debug/powerpc/fadump_region

    This file shows the reserved memory regions if fadump is

    This file shows the reserved memory regions if FADump is

    enabled otherwise this file is empty. The output format

    is::

      <region>: [<start>-<end>] <reserved-size> bytes, Dumped: <dump-size>

    and for kernel DUMP region is:

    DUMP: Src: <src-addr>, Dest: <dest-addr>, Size: <size>, Dumped: # bytes

    e.g.

    Contents when fadump is registered during first kernel::

    Contents when FADump is registered during first kernel::

      # cat /sys/kernel/debug/powerpc/fadump_region

      CPU : [0x0000006ffb0000-0x0000006fff001f] 0x40020 bytes, Dumped: 0x0

      HPTE: [0x0000006fff0020-0x0000006fff101f] 0x1000 bytes, Dumped: 0x0

      DUMP: [0x0000006fff1020-0x0000007fff101f] 0x10000000 bytes, Dumped: 0x0

    Contents when fadump is active during second kernel::

    Contents when FADump is active during second kernel::

      # cat /sys/kernel/debug/powerpc/fadump_region

      CPU : [0x0000006ffb0000-0x0000006fff001f] 0x40020 bytes, Dumped: 0x40020

      DUMP: [0x0000006fff1020-0x0000007fff101f] 0x10000000 bytes, Dumped: 0x10000000

          : [0x00000010000000-0x0000006ffaffff] 0x5ffb0000 bytes, Dumped: 0x5ffb0000

NOTE:

      Please refer to Documentation/filesystems/debugfs.txt on

      how to mount the debugfs filesystem.

 - Need to come up with the better approach to find out more

   accurate boot memory size that is required for a kernel to

   boot successfully when booted with restricted memory.

 - The fadump implementation introduces a fadump crash info structure

 - The FADump implementation introduces a FADump crash info structure

   in the scratch area before the ELF core header. The idea of introducing

   this structure is to pass some important crash info data to the second

   kernel which will help second kernel to populate ELF core header with

    dawr-power9

    dscr

    eeh-pci-error-recovery

    elfnote

    firmware-assisted-dump

    hvcs

    isa-versions

    qe_firmware

    syscall64-abi

    transactional_memory

    ultravisor

.. only::  subproject and html