Commit f3b0946d authored Jul 13, 2016 by Marc Zyngier Committed by Thomas Gleixner Aug 09, 2016

genirq/msi: Make sure PCI MSIs are activated early



Bharat Kumar Gogada reported issues with the generic MSI code, where the
end-point ended up with garbage in its MSI configuration (both for the vector
and the message).

It turns out that the two MSI paths in the kernel are doing slightly different
things:

generic MSI: disable MSI -> allocate MSI -> enable MSI -> setup EP
PCI MSI: disable MSI -> allocate MSI -> setup EP -> enable MSI

And it turns out that end-points are allowed to latch the content of the MSI
configuration registers as soon as MSIs are enabled.  In Bharat's case, the
end-point ends up using whatever was there already, which is not what you
want.

In order to make things converge, we introduce a new MSI domain flag
(MSI_FLAG_ACTIVATE_EARLY) that is unconditionally set for PCI/MSI. When set,
this flag forces the programming of the end-point as soon as the MSIs are
allocated.

A consequence of this is that we have an extra activate in irq_startup, but
that should be without much consequence.

tglx: 

 - Several people reported a VMWare regression with PCI/MSI-X passthrough. It
   turns out that the patch also cures that issue.

 - We need to have a look at the MSI disable interrupt path, where we write
   the msg to all zeros without disabling MSI in the PCI device. Is that
   correct?

Fixes: 52f518a3 "x86/MSI: Use hierarchical irqdomains to manage MSI interrupts"
Reported-and-tested-by: Bharat Kumar Gogada <bharat.kumar.gogada@xilinx.com>
Reported-and-tested-by: Foster Snowhill <forst@forstwoof.ru>
Reported-by: Matthias Prager <linux@matthiasprager.de>
Reported-by: Jason Taylor <jason.taylor@simplivity.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: linux-pci@vger.kernel.org
Cc: stable@vger.kernel.org
Link: http://lkml.kernel.org/r/1468426713-31431-1-git-send-email-marc.zyngier@arm.com


Signed-off-by: Thomas Gleixner <tglx@linutronix.de>

parent 81abf252

drivers/pci/msi.c

+2 −0

Original line number	Diff line number	Diff line
		@@ -1411,6 +1411,8 @@ struct irq_domain pci_msi_create_irq_domain(struct fwnode_handle fwnode,
		if (info->flags & MSI_FLAG_USE_DEF_CHIP_OPS)
		pci_msi_domain_update_chip_ops(info);

		info->flags \|= MSI_FLAG_ACTIVATE_EARLY;

		domain = msi_create_irq_domain(fwnode, info, parent);
		if (!domain)
		return NULL;

include/linux/msi.h

+2 −0

Original line number	Diff line number	Diff line
		@@ -270,6 +270,8 @@ enum {
		MSI_FLAG_MULTI_PCI_MSI = (1 << 2),
		/* Support PCI MSIX interrupts */
		MSI_FLAG_PCI_MSIX = (1 << 3),
		/* Needs early activate, required for PCI */
		MSI_FLAG_ACTIVATE_EARLY = (1 << 4),
		};

		int msi_domain_set_affinity(struct irq_data data, const struct cpumask mask,

kernel/irq/msi.c

+11 −0

Original line number	Diff line number	Diff line
		@@ -359,6 +359,17 @@ int msi_domain_alloc_irqs(struct irq_domain domain, struct device dev,
		else
		dev_dbg(dev, "irq [%d-%d] for MSI\n",
		virq, virq + desc->nvec_used - 1);
		/*
		* This flag is set by the PCI layer as we need to activate
		* the MSI entries before the PCI layer enables MSI in the
		* card. Otherwise the card latches a random msi message.
		*/
		if (info->flags & MSI_FLAG_ACTIVATE_EARLY) {
		struct irq_data *irq_data;

		irq_data = irq_domain_get_irq_data(domain, desc->irq);
		irq_domain_activate_irq(irq_data);
		}
		}

		return 0;

Admin message