Commit 88db5e14 authored May 09, 2007 by Alexey Starikovskiy Committed by Len Brown May 09, 2007

ACPI: created a dedicated workqueue for notify() execution

HP nx6125/nx6325/... machines have a _GPE handler with an infinite
loop sending Notify() events to different ACPI subsystems.

Notify handler in ACPI driver is a C-routine, which may call ACPI
interpreter again to get access to some ACPI variables
(acpi_evaluate_xxx).
On these HP machines such an evaluation changes state of some variable
and lets the loop above break.

In the current ACPI implementation Notify requests are being deferred
to the same kacpid workqueue on which the above GPE handler with
infinite loop is executing. Thus we have a deadlock -- loop will
continue to spin, sending notify events, and at the same time
preventing these notify events from being run on a workqueue. All
notify events are deferred, thus we see increase in memory consumption
noticed by author of the thread. Also as GPE handling is bloked,
machines overheat. Eventually by external poll of the same
acpi_evaluate, kacpid is released and all the queued notify events are
free to run, thus 100% cpu utilization by kacpid for several seconds
or more.

To prevent all these horrors it's needed to not put notify events to
kacpid workqueue by either executing them immediately or putting them
on some other thread. It's dangerous to execute notify events in
place, as it will put several ACPI interpreter stacks on top of each
other (at least 4 in case of nx6125), thus causing kernel  stack
overflow.

First attempt to create a new thread was done by Peter Wainwright
He created a bunch of threads, which were stealing work from a kacpid
workqueue.
This patch appeared in 2.6.15 kernel shipped with Ubuntu 6.06 LTS.

Second attempt was done by me, I created a new thread for each Notify
event. This worked OK on HP nx machines, but broke Linus' Compaq
n620c, by producing threads with a speed what they stopped the machine
completely. Thus this patch was reverted from 18-rc2 as I remember.
I re-made the patch to create second workqueue just for notify events,
thus hopping it will not break Linus' machine. Patch was tested on the
same HP nx machines in #5534 and #7122, but I did not received reply
from Linus on a test patch sent to him.
Patch went to 19-rc and was rejected with much fanfare again.
There was 4th patch, which inserted schedule_timeout(1) into deferred
execution of kacpid, if we had any notify requests pending, but Linus
decided that it was too complex (involved either changes to workqueue
to see if it's empty or atomic inc/dec).
Now you see last variant which adds yield() to every GPE execution.

http://bugzilla.kernel.org/show_bug.cgi?id=5534
http://bugzilla.kernel.org/show_bug.cgi?id=8385



Signed-off-by: Alexey Starikovskiy <alexey.y.starikovskiy@intel.com>
Signed-off-by: Len Brown <len.brown@intel.com>

parent 262a7a28

drivers/acpi/osl.c

+35 −10

Original line number	Diff line number	Diff line
		@@ -71,6 +71,7 @@ static unsigned int acpi_irq_irq;
		static acpi_osd_handler acpi_irq_handler;
		static void *acpi_irq_context;
		static struct workqueue_struct *kacpid_wq;
		static struct workqueue_struct *kacpi_notify_wq;

		static void __init acpi_request_region (struct acpi_generic_address *addr,
		unsigned int length, char *desc)
		@@ -137,8 +138,9 @@ acpi_status acpi_os_initialize1(void)
		return AE_NULL_ENTRY;
		}
		kacpid_wq = create_singlethread_workqueue("kacpid");
		kacpi_notify_wq = create_singlethread_workqueue("kacpi_notify");
		BUG_ON(!kacpid_wq);

		BUG_ON(!kacpi_notify_wq);
		return AE_OK;
		}

		@@ -150,6 +152,7 @@ acpi_status acpi_os_terminate(void)
		}

		destroy_workqueue(kacpid_wq);
		destroy_workqueue(kacpi_notify_wq);

		return AE_OK;
		}
		@@ -601,6 +604,23 @@ void acpi_os_derive_pci_id(acpi_handle rhandle, /* upper bound */
		}

		static void acpi_os_execute_deferred(struct work_struct *work)
		{
		struct acpi_os_dpc *dpc = container_of(work, struct acpi_os_dpc, work);
		if (!dpc) {
		printk(KERN_ERR PREFIX "Invalid (NULL) context\n");
		return;
		}

		dpc->function(dpc->context);
		kfree(dpc);

		/* Yield cpu to notify thread */
		cond_resched();

		return;
		}

		static void acpi_os_execute_notify(struct work_struct *work)
		{
		struct acpi_os_dpc *dpc = container_of(work, struct acpi_os_dpc, work);

		@@ -637,14 +657,12 @@ acpi_status acpi_os_execute(acpi_execute_type type,
		acpi_status status = AE_OK;
		struct acpi_os_dpc *dpc;

		ACPI_FUNCTION_TRACE("os_queue_for_execution");

		ACPI_DEBUG_PRINT((ACPI_DB_EXEC,
		"Scheduling function [%p(%p)] for deferred execution.\n",
		function, context));

		if (!function)
		return_ACPI_STATUS(AE_BAD_PARAMETER);
		return AE_BAD_PARAMETER;

		/*
		* Allocate/initialize DPC structure. Note that this memory will be
		@@ -662,14 +680,21 @@ acpi_status acpi_os_execute(acpi_execute_type type,
		dpc->function = function;
		dpc->context = context;

		if (type == OSL_NOTIFY_HANDLER) {
		INIT_WORK(&dpc->work, acpi_os_execute_notify);
		if (!queue_work(kacpi_notify_wq, &dpc->work)) {
		status = AE_ERROR;
		kfree(dpc);
		}
		} else {
		INIT_WORK(&dpc->work, acpi_os_execute_deferred);
		if (!queue_work(kacpid_wq, &dpc->work)) {
		ACPI_DEBUG_PRINT((ACPI_DB_ERROR,
		"Call to queue_work() failed.\n"));
		kfree(dpc);
		status = AE_ERROR;
		kfree(dpc);
		}
		}

		return_ACPI_STATUS(status);
		}

Admin message