Merge branch 'x86-ras-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

			APEI Error INJection

			~~~~~~~~~~~~~~~~~~~~

EINJ provides a hardware error injection mechanism

It is very useful for debugging and testing of other APEI and RAS features.

EINJ provides a hardware error injection mechanism. It is very useful

for debugging and testing APEI and RAS features in general.

To use EINJ, make sure the following are enabled in your kernel

You need to check whether your BIOS supports EINJ first. For that, look

for early boot messages similar to this one:

ACPI: EINJ 0x000000007370A000 000150 (v01 INTEL           00000001 INTL 00000001)

which shows that the BIOS is exposing an EINJ table - it is the

mechanism through which the injection is done.

Alternatively, look in /sys/firmware/acpi/tables for an "EINJ" file,

which is a different representation of the same thing.

It doesn't necessarily mean that EINJ is not supported if those above

don't exist: before you give up, go into BIOS setup to see if the BIOS

has an option to enable error injection. Look for something called WHEA

or similar. Often, you need to enable an ACPI5 support option prior, in

order to see the APEI,EINJ,... functionality supported and exposed by

the BIOS menu.

To use EINJ, make sure the following are options enabled in your kernel

configuration:

CONFIG_DEBUG_FS

CONFIG_ACPI_APEI

CONFIG_ACPI_APEI_EINJ

The user interface of EINJ is debug file system, under the

directory apei/einj. The following files are provided.

The EINJ user interface is in <debugfs mount point>/apei/einj.

The following files belong to it:

- available_error_type

  Reading this file returns the error injection capability of the

  platform, that is, which error types are supported. The error type

  definition is as follow, the left field is the error type value, the

  right field is error description.

  This file shows which error types are supported:

  Error Type Value	Error Description

  ================	=================

  0x00000001		Processor Correctable

  0x00000002		Processor Uncorrectable non-fatal

  0x00000004		Processor Uncorrectable fatal

  0x00000400		Platform Uncorrectable non-fatal

  0x00000800		Platform Uncorrectable fatal

  The format of file contents are as above, except there are only the

  available error type lines.

  The format of the file contents are as above, except present are only

  the available error types.

- error_type

  This file is used to set the error type value. The error type value

  is defined in "available_error_type" description.

  Set the value of the error type being injected. Possible error types

  are defined in the file available_error_type above.

- error_inject

  Write any integer to this file to trigger the error

  injection. Before this, please specify all necessary error

  parameters.

  Write any integer to this file to trigger the error injection. Make

  sure you have specified all necessary error parameters, i.e. this

  write should be the last step when injecting errors.

- flags

  Present for kernel version 3.13 and above. Used to specify which

  of param{1..4} are valid and should be used by BIOS during injection.

  Value is a bitmask as specified in ACPI5.0 spec for the

  Present for kernel versions 3.13 and above. Used to specify which

  of param{1..4} are valid and should be used by the firmware during

  injection. Value is a bitmask as specified in ACPI5.0 spec for the

  SET_ERROR_TYPE_WITH_ADDRESS data structure:

	Bit 0 - Processor APIC field valid (see param3 below)

	Bit 1 - Memory address and mask valid (param1 and param2)

	Bit 2 - PCIe (seg,bus,dev,fn) valid (param4 below)

  If set to zero, legacy behaviour is used where the type of injection

  specifies just one bit set, and param1 is multiplexed.

	Bit 0 - Processor APIC field valid (see param3 below).

	Bit 1 - Memory address and mask valid (param1 and param2).

	Bit 2 - PCIe (seg,bus,dev,fn) valid (see param4 below).

  If set to zero, legacy behavior is mimicked where the type of

  injection specifies just one bit set, and param1 is multiplexed.

- param1

  This file is used to set the first error parameter value. Effect of

  parameter depends on error_type specified. For example, if error

  type is memory related type, the param1 should be a valid physical

  memory address. [Unless "flag" is set - see above]

  This file is used to set the first error parameter value. Its effect

  depends on the error type specified in error_type. For example, if

  error type is memory related type, the param1 should be a valid

  physical memory address. [Unless "flag" is set - see above]

- param2

  This file is used to set the second error parameter value. Effect of

  parameter depends on error_type specified. For example, if error

  type is memory related type, the param2 should be a physical memory

  address mask. Linux requires page or narrower granularity, say,

  0xfffffffffffff000.

  Same use as param1 above. For example, if error type is of memory

  related type, then param2 should be a physical memory address mask.

  Linux requires page or narrower granularity, say, 0xfffffffffffff000.

- param3

  Used when the 0x1 bit is set in "flag" to specify the APIC id

  Used when the 0x1 bit is set in "flags" to specify the APIC id

- param4

  Used when the 0x4 bit is set in "flag" to specify target PCIe device

  Used when the 0x4 bit is set in "flags" to specify target PCIe device

- notrigger

  The EINJ mechanism is a two step process. First inject the error, then

  perform some actions to trigger it. Setting "notrigger" to 1 skips the

  trigger phase, which *may* allow the user to cause the error in some other

  context by a simple access to the cpu, memory location, or device that is

  the target of the error injection. Whether this actually works depends

  on what operations the BIOS actually includes in the trigger phase.

BIOS versions based in the ACPI 4.0 specification have limited options

to control where the errors are injected.  Your BIOS may support an

extension (enabled with the param_extension=1 module parameter, or

boot command line einj.param_extension=1). This allows the address

and mask for memory injections to be specified by the param1 and

param2 files in apei/einj.

BIOS versions using the ACPI 5.0 specification have more control over

the target of the injection. For processor related errors (type 0x1,

0x2 and 0x4) the APICID of the target should be provided using the

param1 file in apei/einj. For memory errors (type 0x8, 0x10 and 0x20)

the address is set using param1 with a mask in param2 (0x0 is equivalent

to all ones). For PCI express errors (type 0x40, 0x80 and 0x100) the

segment, bus, device and function are specified using param1:

  The error injection mechanism is a two-step process. First inject the

  error, then perform some actions to trigger it. Setting "notrigger"

  to 1 skips the trigger phase, which *may* allow the user to cause the

  error in some other context by a simple access to the CPU, memory

  location, or device that is the target of the error injection. Whether

  this actually works depends on what operations the BIOS actually

  includes in the trigger phase.

BIOS versions based on the ACPI 4.0 specification have limited options

in controlling where the errors are injected. Your BIOS may support an

extension (enabled with the param_extension=1 module parameter, or boot

command line einj.param_extension=1). This allows the address and mask

for memory injections to be specified by the param1 and param2 files in

apei/einj.

BIOS versions based on the ACPI 5.0 specification have more control over

the target of the injection. For processor-related errors (type 0x1, 0x2

and 0x4), you can set flags to 0x3 (param3 for bit 0, and param1 and

param2 for bit 1) so that you have more information added to the error

signature being injected. The actual data passed is this:

	memory_address = param1;

	memory_address_range = param2;

	apicid = param3;

	pcie_sbdf = param4;

For memory errors (type 0x8, 0x10 and 0x20) the address is set using

param1 with a mask in param2 (0x0 is equivalent to all ones). For PCI

express errors (type 0x40, 0x80 and 0x100) the segment, bus, device and

function are specified using param1:

         31     24 23    16 15    11 10      8  7        0

	+-------------------------------------------------+

	| segment |   bus  | device | function | reserved |

	+-------------------------------------------------+

An ACPI 5.0 BIOS may also allow vendor specific errors to be injected.

Anyway, you get the idea, if there's doubt just take a look at the code

in drivers/acpi/apei/einj.c.

An ACPI 5.0 BIOS may also allow vendor-specific errors to be injected.

In this case a file named vendor will contain identifying information

from the BIOS that hopefully will allow an application wishing to use

the vendor specific extension to tell that they are running on a BIOS

the vendor-specific extension to tell that they are running on a BIOS

that supports it. All vendor extensions have the 0x80000000 bit set in

error_type. A file vendor_flags controls the interpretation of param1

and param2 (1 = PROCESSOR, 2 = MEMORY, 4 = PCI). See your BIOS vendor

documentation for details (and expect changes to this API if vendors

creativity in using this feature expands beyond our expectations).

Example:

An error injection example:

# cd /sys/kernel/debug/apei/einj

# cat available_error_type		# See which errors can be injected

0x00000002	Processor Uncorrectable non-fatal

0x00000008	Memory Correctable

0x00000010	Memory Uncorrectable non-fatal

# echo 0x12345000 > param1		# Set memory address for injection

# echo 0xfffffffffffff000 > param2	# Mask - anywhere in this page

# echo $((-1 << 12)) > param2		# Mask 0xfffffffffffff000 - anywhere in this page

# echo 0x8 > error_type			# Choose correctable memory error

# echo 1 > error_inject			# Inject now

You should see something like this in dmesg:

[22715.830801] EDAC sbridge MC3: HANDLING MCE MEMORY ERROR

[22715.834759] EDAC sbridge MC3: CPU 0: Machine Check Event: 0 Bank 7: 8c00004000010090

[22715.834759] EDAC sbridge MC3: TSC 0

[22715.834759] EDAC sbridge MC3: ADDR 12345000 EDAC sbridge MC3: MISC 144780c86

[22715.834759] EDAC sbridge MC3: PROCESSOR 0:306e7 TIME 1422553404 SOCKET 0 APIC 0

[22716.616173] EDAC MC3: 1 CE memory read error on CPU_SrcID#0_Channel#0_DIMM#0 (channel:0 slot:0 page:0x12345 offset:0x0 grain:32 syndrome:0x0 -  area:DRAM err_code:0001:0090 socket:0 channel_mask:1 rank:0)

For more information about EINJ, please refer to ACPI specification

version 4.0, section 17.5 and ACPI 5.0, section 18.6.

	u32 rip_msr;

};

struct mce_vendor_flags {

	__u64		overflow_recov	: 1, /* cpuid_ebx(80000007) */

			__reserved_0	: 63;

};

extern struct mce_vendor_flags mce_flags;

extern struct mca_config mca_cfg;

extern void mce_register_decode_chain(struct notifier_block *nb);

extern void mce_unregister_decode_chain(struct notifier_block *nb);

#ifdef CONFIG_X86_MCE

int mcheck_init(void);

void mcheck_cpu_init(struct cpuinfo_x86 *c);

void mcheck_vendor_init_severity(void);

#else

static inline int mcheck_init(void) { return 0; }

static inline void mcheck_cpu_init(struct cpuinfo_x86 *c) {}

static inline void mcheck_vendor_init_severity(void) {}

#endif

#ifdef CONFIG_X86_ANCIENT_MCE

DECLARE_PER_CPU(mce_banks_t, mce_poll_banks);

enum mcp_flags {

	MCP_TIMESTAMP = (1 << 0),	/* log time stamp */

	MCP_UC = (1 << 1),		/* log uncorrected errors */

	MCP_DONTLOG = (1 << 2),		/* only clear, don't log */

	MCP_TIMESTAMP	= BIT(0),	/* log time stamp */

	MCP_UC		= BIT(1),	/* log uncorrected errors */

	MCP_DONTLOG	= BIT(2),	/* only clear, don't log */

};

void machine_check_poll(enum mcp_flags flags, mce_banks_t *b);

bool machine_check_poll(enum mcp_flags flags, mce_banks_t *b);

int mce_notify_irq(void);

};

#define ATTR_LEN		16

#define INITIAL_CHECK_INTERVAL	5 * 60 /* 5 minutes */

/* One object for each MCE bank, shared by all CPUs */

struct mce_bank {

	char			attrname[ATTR_LEN];	/* attribute name */

};

int mce_severity(struct mce *a, int tolerant, char **msg, bool is_excp);

extern int (*mce_severity)(struct mce *a, int tolerant, char **msg, bool is_excp);

struct dentry *mce_get_debugfs_dir(void);

extern struct mce_bank *mce_banks;

extern mce_banks_t mce_banks_ce_disabled;

#ifdef CONFIG_X86_MCE_INTEL

unsigned long mce_intel_adjust_timer(unsigned long interval);

void mce_intel_cmci_poll(void);

unsigned long cmci_intel_adjust_timer(unsigned long interval);

bool mce_intel_cmci_poll(void);

void mce_intel_hcpu_update(unsigned long cpu);

void cmci_disable_bank(int bank);

#else

# define mce_intel_adjust_timer mce_adjust_timer_default

static inline void mce_intel_cmci_poll(void) { }

# define cmci_intel_adjust_timer mce_adjust_timer_default

static inline bool mce_intel_cmci_poll(void) { return false; }

static inline void mce_intel_hcpu_update(unsigned long cpu) { }

static inline void cmci_disable_bank(int bank) { }

#endif

	return ((m->cs & 3) == 3) ? IN_USER : IN_KERNEL;

}

int mce_severity(struct mce *m, int tolerant, char **msg, bool is_excp)

/*

 * See AMD Error Scope Hierarchy table in a newer BKDG. For example

 * 49125_15h_Models_30h-3Fh_BKDG.pdf, section "RAS Features"

 */

static int mce_severity_amd(struct mce *m, int tolerant, char **msg, bool is_excp)

{

	enum context ctx = error_context(m);

	/* Processor Context Corrupt, no need to fumble too much, die! */

	if (m->status & MCI_STATUS_PCC)

		return MCE_PANIC_SEVERITY;

	if (m->status & MCI_STATUS_UC) {

		/*

		 * On older systems where overflow_recov flag is not present, we

		 * should simply panic if an error overflow occurs. If

		 * overflow_recov flag is present and set, then software can try

		 * to at least kill process to prolong system operation.

		 */

		if (mce_flags.overflow_recov) {

			/* software can try to contain */

			if (!(m->mcgstatus & MCG_STATUS_RIPV) && (ctx == IN_KERNEL))

				return MCE_PANIC_SEVERITY;

			/* kill current process */

			return MCE_AR_SEVERITY;

		} else {

			/* at least one error was not logged */

			if (m->status & MCI_STATUS_OVER)

				return MCE_PANIC_SEVERITY;

		}

		/*

		 * For any other case, return MCE_UC_SEVERITY so that we log the

		 * error and exit #MC handler.

		 */

		return MCE_UC_SEVERITY;

	}

	/*

	 * deferred error: poll handler catches these and adds to mce_ring so

	 * memory-failure can take recovery actions.

	 */

	if (m->status & MCI_STATUS_DEFERRED)

		return MCE_DEFERRED_SEVERITY;

	/*

	 * corrected error: poll handler catches these and passes responsibility

	 * of decoding the error to EDAC

	 */

	return MCE_KEEP_SEVERITY;

}

static int mce_severity_intel(struct mce *m, int tolerant, char **msg, bool is_excp)

{

	enum exception excp = (is_excp ? EXCP_CONTEXT : NO_EXCP);

	enum context ctx = error_context(m);

	}

}

/* Default to mce_severity_intel */

int (*mce_severity)(struct mce *m, int tolerant, char **msg, bool is_excp) =

		    mce_severity_intel;

void __init mcheck_vendor_init_severity(void)

{

	if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD)

		mce_severity = mce_severity_amd;

}

#ifdef CONFIG_DEBUG_FS

static void *s_start(struct seq_file *f, loff_t *pos)

{

DEFINE_PER_CPU(unsigned, mce_exception_count);

struct mce_bank *mce_banks __read_mostly;

struct mce_vendor_flags mce_flags __read_mostly;

struct mca_config mca_cfg __read_mostly = {

	.bootlog  = -1,

static DEFINE_PER_CPU(struct mce, mces_seen);

static int			cpu_missing;

/* CMCI storm detection filter */

static DEFINE_PER_CPU(unsigned long, mce_polled_error);

/*

 * MCA banks polled by the period polling timer for corrected events.

 * With Intel CMCI, this only has MCA banks which do not support CMCI (if any).

 * is already totally * confused. In this case it's likely it will

 * not fully execute the machine check handler either.

 */

void machine_check_poll(enum mcp_flags flags, mce_banks_t *b)

bool machine_check_poll(enum mcp_flags flags, mce_banks_t *b)

{

	bool error_logged = false;

	struct mce m;

	int severity;

	int i;

		if (!(m.status & MCI_STATUS_VAL))

			continue;

		this_cpu_write(mce_polled_error, 1);

		/*

		 * Uncorrected or signalled events are handled by the exception

		 * handler when it is enabled, so don't process those here.

		 * Don't get the IP here because it's unlikely to

		 * have anything to do with the actual error location.

		 */

		if (!(flags & MCP_DONTLOG) && !mca_cfg.dont_log_ce)

		if (!(flags & MCP_DONTLOG) && !mca_cfg.dont_log_ce) {

			error_logged = true;

			mce_log(&m);

		}

		/*

		 * Clear state for this bank.

	 */

	sync_core();

	return error_logged;

}

EXPORT_SYMBOL_GPL(machine_check_poll);

	 * other CPUs.

	 */

	if (m && global_worst >= MCE_PANIC_SEVERITY && mca_cfg.tolerant < 3)

		mce_panic("Fatal Machine check", m, msg);

		mce_panic("Fatal machine check", m, msg);

	/*

	 * For UC somewhere we let the CPU who detects it handle it.

	 * source or one CPU is hung. Panic.

	 */

	if (global_worst <= MCE_KEEP_SEVERITY && mca_cfg.tolerant < 3)

		mce_panic("Machine check from unknown source", NULL, NULL);

		mce_panic("Fatal machine check from unknown source", NULL, NULL);

	/*

	 * Now clear all the mces_seen so that they don't reappear on

 * poller finds an MCE, poll 2x faster.  When the poller finds no more

 * errors, poll 2x slower (up to check_interval seconds).

 */

static unsigned long check_interval = 5 * 60; /* 5 minutes */

static unsigned long check_interval = INITIAL_CHECK_INTERVAL;

static DEFINE_PER_CPU(unsigned long, mce_next_interval); /* in jiffies */

static DEFINE_PER_CPU(struct timer_list, mce_timer);

	return interval;

}

static unsigned long (*mce_adjust_timer)(unsigned long interval) =

	mce_adjust_timer_default;

static unsigned long (*mce_adjust_timer)(unsigned long interval) = mce_adjust_timer_default;

static int cmc_error_seen(void)

static void __restart_timer(struct timer_list *t, unsigned long interval)

{

	unsigned long *v = this_cpu_ptr(&mce_polled_error);

	unsigned long when = jiffies + interval;

	unsigned long flags;

	local_irq_save(flags);

	if (timer_pending(t)) {

		if (time_before(when, t->expires))

			mod_timer_pinned(t, when);

	} else {

		t->expires = round_jiffies(when);

		add_timer_on(t, smp_processor_id());

	}

	return test_and_clear_bit(0, v);

	local_irq_restore(flags);

}

static void mce_timer_fn(unsigned long data)

{

	struct timer_list *t = this_cpu_ptr(&mce_timer);

	int cpu = smp_processor_id();

	unsigned long iv;

	int notify;

	WARN_ON(smp_processor_id() != data);

	WARN_ON(cpu != data);

	iv = __this_cpu_read(mce_next_interval);

	if (mce_available(this_cpu_ptr(&cpu_info))) {

		machine_check_poll(MCP_TIMESTAMP,

				this_cpu_ptr(&mce_poll_banks));

		mce_intel_cmci_poll();

		machine_check_poll(MCP_TIMESTAMP, this_cpu_ptr(&mce_poll_banks));

		if (mce_intel_cmci_poll()) {

			iv = mce_adjust_timer(iv);

			goto done;

		}

	}

	/*

	 * Alert userspace if needed.  If we logged an MCE, reduce the

	 * polling interval, otherwise increase the polling interval.

	 * Alert userspace if needed. If we logged an MCE, reduce the polling

	 * interval, otherwise increase the polling interval.

	 */

	iv = __this_cpu_read(mce_next_interval);

	notify = mce_notify_irq();

	notify |= cmc_error_seen();

	if (notify) {

	if (mce_notify_irq())

		iv = max(iv / 2, (unsigned long) HZ/100);

	} else {

	else

		iv = min(iv * 2, round_jiffies_relative(check_interval * HZ));

		iv = mce_adjust_timer(iv);

	}

done:

	__this_cpu_write(mce_next_interval, iv);

	/* Might have become 0 after CMCI storm subsided */

	if (iv) {

		t->expires = jiffies + iv;

		add_timer_on(t, smp_processor_id());

	}

	__restart_timer(t, iv);

}

/*

void mce_timer_kick(unsigned long interval)

{

	struct timer_list *t = this_cpu_ptr(&mce_timer);

	unsigned long when = jiffies + interval;

	unsigned long iv = __this_cpu_read(mce_next_interval);

	if (timer_pending(t)) {

		if (time_before(when, t->expires))

			mod_timer_pinned(t, when);

	} else {

		t->expires = round_jiffies(when);

		add_timer_on(t, smp_processor_id());

	}

	__restart_timer(t, interval);

	if (interval < iv)

		__this_cpu_write(mce_next_interval, interval);

}

		if (c->x86 == 6 && cfg->banks > 0)

			mce_banks[0].ctl = 0;

		/*

		 * overflow_recov is supported for F15h Models 00h-0fh

		 * even though we don't have a CPUID bit for it.

		 */

		if (c->x86 == 0x15 && c->x86_model <= 0xf)

			mce_flags.overflow_recov = 1;

		/*

		 * Turn off MC4_MISC thresholding banks on those models since

		 * they're not supported there.

		if (c->x86 == 0x15 &&

		    (c->x86_model >= 0x10 && c->x86_model <= 0x1f)) {

			int i;

			 u64 val, hwcr;

			u64 hwcr;

			bool need_toggle;

			u32 msrs[] = {

				0x00000413, /* MC4_MISC0 */

			if (need_toggle)

				wrmsrl(MSR_K7_HWCR, hwcr | BIT(18));

			 for (i = 0; i < ARRAY_SIZE(msrs); i++) {

				 rdmsrl(msrs[i], val);

				 /* CntP bit set? */

				 if (val & BIT_64(62)) {

					val &= ~BIT_64(62);

					wrmsrl(msrs[i], val);

				 }

			 }

			/* Clear CntP bit safely */

			for (i = 0; i < ARRAY_SIZE(msrs); i++)

				msr_clear_bit(msrs[i], 62);

			/* restore old settings */

			if (need_toggle)

	switch (c->x86_vendor) {

	case X86_VENDOR_INTEL:

		mce_intel_feature_init(c);

		mce_adjust_timer = mce_intel_adjust_timer;

		mce_adjust_timer = cmci_intel_adjust_timer;

		break;

	case X86_VENDOR_AMD:

		mce_amd_feature_init(c);

		mce_flags.overflow_recov = cpuid_ebx(0x80000007) & 0x1;

		break;

	default:

		break;

int __init mcheck_init(void)

{

	mcheck_intel_therm_init();

	mcheck_vendor_init_severity();

	return 0;

}