IB/mlx4: Signal node desc changes to SM by using FW to generate trap 144 (d0d68b86) · Commits · 戴 / test

The Node Description cannot be changed via MADs (it is read-only). Until now, it was changed in the driver via sysfs, and the new Node Description was simply inserted by the driver into MAD responses (replacing the description returned by FW). System startup scripts use the sysfs interface to change the node description at driver startup to show the hostname, etc. However, this has a race condition: the SM could discover the original FW node description rather than the system-specific description if it queried the port before the startup scripts finish running. For mlx4, we fix this with a new FW command (SET_NODE) that allows passing the new node description to FW. When this command is invoked, FW sends a trap 144 to the SM. When it gets this trap, the SM can query the node to obtain the new node description -- thus eliminating the effects of the race. This patch simply calls SET_NODE command when a new node description is entered via sysfs (thus causing trap 144 to be issued by the FW). We ignore all failures of the SET_NODE command (including those caused by using a device FW that predates the SET_NODE command), since in that case things work just as before. Signed-off-by:

Jack Morgenstein <jackm@dev.mellanox.co.il> Signed-off-by:

Roland Dreier <rolandd@cisco.com>

drivers/infiniband/hw/mlx4/main.c

+23 −5

Original line number	Original line	Diff line number	Diff line
	@@ -272,14 +272,32 @@ out:
	static int mlx4_ib_modify_device(struct ib_device *ibdev, int mask,		static int mlx4_ib_modify_device(struct ib_device *ibdev, int mask,
	struct ib_device_modify *props)		struct ib_device_modify *props)
	{		{
			struct mlx4_cmd_mailbox *mailbox;

	if (mask & ~IB_DEVICE_MODIFY_NODE_DESC)		if (mask & ~IB_DEVICE_MODIFY_NODE_DESC)
	return -EOPNOTSUPP;		return -EOPNOTSUPP;

	if (mask & IB_DEVICE_MODIFY_NODE_DESC) {		if (!(mask & IB_DEVICE_MODIFY_NODE_DESC))
			return 0;

	spin_lock(&to_mdev(ibdev)->sm_lock);		spin_lock(&to_mdev(ibdev)->sm_lock);
	memcpy(ibdev->node_desc, props->node_desc, 64);		memcpy(ibdev->node_desc, props->node_desc, 64);
	spin_unlock(&to_mdev(ibdev)->sm_lock);		spin_unlock(&to_mdev(ibdev)->sm_lock);
	}
			/*
			* If possible, pass node desc to FW, so it can generate
			* a 144 trap. If cmd fails, just ignore.
			*/
			mailbox = mlx4_alloc_cmd_mailbox(to_mdev(ibdev)->dev);
			if (IS_ERR(mailbox))
			return 0;

			memset(mailbox->buf, 0, 256);
			memcpy(mailbox->buf, props->node_desc, 64);
			mlx4_cmd(to_mdev(ibdev)->dev, mailbox->dma, 1, 0,
			MLX4_CMD_SET_NODE, MLX4_CMD_TIME_CLASS_A);

			mlx4_free_cmd_mailbox(to_mdev(ibdev)->dev, mailbox);

	return 0;		return 0;
	}		}

include/linux/mlx4/cmd.h

+1 −0

Original line number	Original line	Diff line number	Diff line
	@@ -57,6 +57,7 @@ enum {
	MLX4_CMD_QUERY_PORT = 0x43,		MLX4_CMD_QUERY_PORT = 0x43,
	MLX4_CMD_SENSE_PORT = 0x4d,		MLX4_CMD_SENSE_PORT = 0x4d,
	MLX4_CMD_SET_PORT = 0xc,		MLX4_CMD_SET_PORT = 0xc,
			MLX4_CMD_SET_NODE = 0x5a,
	MLX4_CMD_ACCESS_DDR = 0x2e,		MLX4_CMD_ACCESS_DDR = 0x2e,
	MLX4_CMD_MAP_ICM = 0xffa,		MLX4_CMD_MAP_ICM = 0xffa,
	MLX4_CMD_UNMAP_ICM = 0xff9,		MLX4_CMD_UNMAP_ICM = 0xff9,

Admin message