cxlflash: Superpipe support

0xB3	00	linux/mmc/ioctl.h

0xC0	00-0F	linux/usb/iowarrior.h

0xCA	00-0F	uapi/misc/cxl.h

0xCA	80-8F	uapi/scsi/cxlflash_ioctl.h

0xCB	00-1F	CBM serial IEC bus	in development:

					<mailto:michael.klein@puffin.lb.shuttle.de>

0xCD	01	linux/reiserfs_fs.h

Introduction

============

    The IBM Power architecture provides support for CAPI (Coherent

    Accelerator Power Interface), which is available to certain PCIe slots

    on Power 8 systems. CAPI can be thought of as a special tunneling

    protocol through PCIe that allow PCIe adapters to look like special

    purpose co-processors which can read or write an application's

    memory and generate page faults. As a result, the host interface to

    an adapter running in CAPI mode does not require the data buffers to

    be mapped to the device's memory (IOMMU bypass) nor does it require

    memory to be pinned.

    On Linux, Coherent Accelerator (CXL) kernel services present CAPI

    devices as a PCI device by implementing a virtual PCI host bridge.

    This abstraction simplifies the infrastructure and programming

    model, allowing for drivers to look similar to other native PCI

    device drivers.

    CXL provides a mechanism by which user space applications can

    directly talk to a device (network or storage) bypassing the typical

    kernel/device driver stack. The CXL Flash Adapter Driver enables a

    user space application direct access to Flash storage.

    The CXL Flash Adapter Driver is a kernel module that sits in the

    SCSI stack as a low level device driver (below the SCSI disk and

    protocol drivers) for the IBM CXL Flash Adapter. This driver is

    responsible for the initialization of the adapter, setting up the

    special path for user space access, and performing error recovery. It

    communicates directly the Flash Accelerator Functional Unit (AFU)

    as described in Documentation/powerpc/cxl.txt.

    The cxlflash driver supports two, mutually exclusive, modes of

    operation at the device (LUN) level:

        - Any flash device (LUN) can be configured to be accessed as a

          regular disk device (i.e.: /dev/sdc). This is the default mode.

        - Any flash device (LUN) can be configured to be accessed from

          user space with a special block library. This mode further

          specifies the means of accessing the device and provides for

          either raw access to the entire LUN (referred to as direct

          or physical LUN access) or access to a kernel/AFU-mediated

          partition of the LUN (referred to as virtual LUN access). The

          segmentation of a disk device into virtual LUNs is assisted

          by special translation services provided by the Flash AFU.

Overview

========

    The Coherent Accelerator Interface Architecture (CAIA) introduces a

    concept of a master context. A master typically has special privileges

    granted to it by the kernel or hypervisor allowing it to perform AFU

    wide management and control. The master may or may not be involved

    directly in each user I/O, but at the minimum is involved in the

    initial setup before the user application is allowed to send requests

    directly to the AFU.

    The CXL Flash Adapter Driver establishes a master context with the

    AFU. It uses memory mapped I/O (MMIO) for this control and setup. The

    Adapter Problem Space Memory Map looks like this:

                     +-------------------------------+

                     |    512 * 64 KB User MMIO      |

                     |        (per context)          |

                     |       User Accessible         |

                     +-------------------------------+

                     |    512 * 128 B per context    |

                     |    Provisioning and Control   |

                     |   Trusted Process accessible  |

                     +-------------------------------+

                     |         64 KB Global          |

                     |   Trusted Process accessible  |

                     +-------------------------------+

    This driver configures itself into the SCSI software stack as an

    adapter driver. The driver is the only entity that is considered a

    Trusted Process to program the Provisioning and Control and Global

    areas in the MMIO Space shown above.  The master context driver

    discovers all LUNs attached to the CXL Flash adapter and instantiates

    scsi block devices (/dev/sdb, /dev/sdc etc.) for each unique LUN

    seen from each path.

    Once these scsi block devices are instantiated, an application

    written to a specification provided by the block library may get

    access to the Flash from user space (without requiring a system call).

    This master context driver also provides a series of ioctls for this

    block library to enable this user space access.  The driver supports

    two modes for accessing the block device.

    The first mode is called a virtual mode. In this mode a single scsi

    block device (/dev/sdb) may be carved up into any number of distinct

    virtual LUNs. The virtual LUNs may be resized as long as the sum of

    the sizes of all the virtual LUNs, along with the meta-data associated

    with it does not exceed the physical capacity.

    The second mode is called the physical mode. In this mode a single

    block device (/dev/sdb) may be opened directly by the block library

    and the entire space for the LUN is available to the application.

    Only the physical mode provides persistence of the data.  i.e. The

    data written to the block device will survive application exit and

    restart and also reboot. The virtual LUNs do not persist (i.e. do

    not survive after the application terminates or the system reboots).

Block library API

=================

    Applications intending to get access to the CXL Flash from user

    space should use the block library, as it abstracts the details of

    interfacing directly with the cxlflash driver that are necessary for

    performing administrative actions (i.e.: setup, tear down, resize).

    The block library can be thought of as a 'user' of services,

    implemented as IOCTLs, that are provided by the cxlflash driver

    specifically for devices (LUNs) operating in user space access

    mode. While it is not a requirement that applications understand

    the interface between the block library and the cxlflash driver,

    a high-level overview of each supported service (IOCTL) is provided

    below.

    The block library can be found on GitHub:

    http://www.github.com/mikehollinger/ibmcapikv

CXL Flash Driver IOCTLs

=======================

    Users, such as the block library, that wish to interface with a flash

    device (LUN) via user space access need to use the services provided

    by the cxlflash driver. As these services are implemented as ioctls,

    a file descriptor handle must first be obtained in order to establish

    the communication channel between a user and the kernel.  This file

    descriptor is obtained by opening the device special file associated

    with the scsi disk device (/dev/sdb) that was created during LUN

    discovery. As per the location of the cxlflash driver within the

    SCSI protocol stack, this open is actually not seen by the cxlflash

    driver. Upon successful open, the user receives a file descriptor

    (herein referred to as fd1) that should be used for issuing the

    subsequent ioctls listed below.

    The structure definitions for these IOCTLs are available in:

    uapi/scsi/cxlflash_ioctl.h

DK_CXLFLASH_ATTACH

------------------

    This ioctl obtains, initializes, and starts a context using the CXL

    kernel services. These services specify a context id (u16) by which

    to uniquely identify the context and its allocated resources. The

    services additionally provide a second file descriptor (herein

    referred to as fd2) that is used by the block library to initiate

    memory mapped I/O (via mmap()) to the CXL flash device and poll for

    completion events. This file descriptor is intentionally installed by

    this driver and not the CXL kernel services to allow for intermediary

    notification and access in the event of a non-user-initiated close(),

    such as a killed process. This design point is described in further

    detail in the description for the DK_CXLFLASH_DETACH ioctl.

    There are a few important aspects regarding the "tokens" (context id

    and fd2) that are provided back to the user:

        - These tokens are only valid for the process under which they

          were created. The child of a forked process cannot continue

          to use the context id or file descriptor created by its parent.

        - These tokens are only valid for the lifetime of the context and

          the process under which they were created. Once either is

          destroyed, the tokens are to be considered stale and subsequent

          usage will result in errors.

        - When a context is no longer needed, the user shall detach from

          the context via the DK_CXLFLASH_DETACH ioctl.

        - A close on fd2 will invalidate the tokens. This operation is not

          required by the user.

DK_CXLFLASH_USER_DIRECT

-----------------------

    This ioctl is responsible for transitioning the LUN to direct

    (physical) mode access and configuring the AFU for direct access from

    user space on a per-context basis. Additionally, the block size and

    last logical block address (LBA) are returned to the user.

    As mentioned previously, when operating in user space access mode,

    LUNs may be accessed in whole or in part. Only one mode is allowed

    at a time and if one mode is active (outstanding references exist),

    requests to use the LUN in a different mode are denied.

    The AFU is configured for direct access from user space by adding an

    entry to the AFU's resource handle table. The index of the entry is

    treated as a resource handle that is returned to the user. The user

    is then able to use the handle to reference the LUN during I/O.

DK_CXLFLASH_RELEASE

-------------------

    This ioctl is responsible for releasing a previously obtained

    reference to either a physical or virtual LUN. This can be

    thought of as the inverse of the DK_CXLFLASH_USER_DIRECT or

    DK_CXLFLASH_USER_VIRTUAL ioctls. Upon success, the resource handle

    is no longer valid and the entry in the resource handle table is

    made available to be used again.

    As part of the release process for virtual LUNs, the virtual LUN

    is first resized to 0 to clear out and free the translation tables

    associated with the virtual LUN reference.

DK_CXLFLASH_DETACH

------------------

    This ioctl is responsible for unregistering a context with the

    cxlflash driver and release outstanding resources that were

    not explicitly released via the DK_CXLFLASH_RELEASE ioctl. Upon

    success, all "tokens" which had been provided to the user from the

    DK_CXLFLASH_ATTACH onward are no longer valid.

DK_CXLFLASH_VERIFY

------------------

    This ioctl is used to detect various changes such as the capacity of

    the disk changing, the number of LUNs visible changing, etc. In cases

    where the changes affect the application (such as a LUN resize), the

    cxlflash driver will report the changed state to the application.

    The user calls in when they want to validate that a LUN hasn't been

    changed in response to a check condition. As the user is operating out

    of band from the kernel, they will see these types of events without

    the kernel's knowledge. When encountered, the user's architected

    behavior is to call in to this ioctl, indicating what they want to

    verify and passing along any appropriate information. For now, only

    verifying a LUN change (ie: size different) with sense data is

    supported.

DK_CXLFLASH_RECOVER_AFU

-----------------------

    This ioctl is used to drive recovery (if such an action is warranted)

    of a specified user context. Any state associated with the user context

    is re-established upon successful recovery.

    User contexts are put into an error condition when the device needs to

    be reset or is terminating. Users are notified of this error condition

    by seeing all 0xF's on an MMIO read. Upon encountering this, the

    architected behavior for a user is to call into this ioctl to recover

    their context. A user may also call into this ioctl at any time to

    check if the device is operating normally. If a failure is returned

    from this ioctl, the user is expected to gracefully clean up their

    context via release/detach ioctls. Until they do, the context they

    hold is not relinquished. The user may also optionally exit the process

    at which time the context/resources they held will be freed as part of

    the release fop.

DK_CXLFLASH_MANAGE_LUN

----------------------

    This ioctl is used to switch a LUN from a mode where it is available

    for file-system access (legacy), to a mode where it is set aside for

    exclusive user space access (superpipe). In case a LUN is visible

    across multiple ports and adapters, this ioctl is used to uniquely

    identify each LUN by its World Wide Node Name (WWNN).

obj-$(CONFIG_CXLFLASH) += cxlflash.o

cxlflash-y += main.o

cxlflash-y += main.o superpipe.o lunmgt.o

	struct pci_pool *cxlflash_cmd_pool;

	struct pci_dev *parent_dev;

	atomic_t recovery_threads;

	struct mutex ctx_recovery_mutex;

	struct mutex ctx_tbl_list_mutex;

	struct ctx_info *ctx_tbl[MAX_CONTEXT];

	struct list_head ctx_err_recovery; /* contexts w/ recovery pending */

	struct file_operations cxl_fops;

	atomic_t num_user_contexts;

	struct list_head lluns; /* list of llun_info structs */

	wait_queue_head_t tmf_waitq;

	bool tmf_active;

	wait_queue_head_t limbo_waitq;

struct afu_cmd *cxlflash_cmd_checkout(struct afu *);

void cxlflash_cmd_checkin(struct afu_cmd *);

int cxlflash_afu_sync(struct afu *, ctx_hndl_t, res_hndl_t, u8);

void cxlflash_list_init(void);

void cxlflash_term_global_luns(void);

void cxlflash_free_errpage(void);

int cxlflash_ioctl(struct scsi_device *, int, void __user *);

void cxlflash_stop_term_user_contexts(struct cxlflash_cfg *);

int cxlflash_mark_contexts_error(struct cxlflash_cfg *);

void cxlflash_term_local_luns(struct cxlflash_cfg *);

#endif /* ifndef _CXLFLASH_COMMON_H */

/*

 * CXL Flash Device Driver

 *

 * Written by: Manoj N. Kumar <manoj@linux.vnet.ibm.com>, IBM Corporation

 *             Matthew R. Ochs <mrochs@linux.vnet.ibm.com>, IBM Corporation

 *

 * Copyright (C) 2015 IBM Corporation

 *

 * This program is free software; you can redistribute it and/or

 * modify it under the terms of the GNU General Public License

 * as published by the Free Software Foundation; either version

 * 2 of the License, or (at your option) any later version.

 */

#include <misc/cxl.h>

#include <asm/unaligned.h>

#include <scsi/scsi_host.h>

#include <uapi/scsi/cxlflash_ioctl.h>

#include "sislite.h"

#include "common.h"

#include "superpipe.h"

/**

 * create_local() - allocate and initialize a local LUN information structure

 * @sdev:	SCSI device associated with LUN.

 * @wwid:	World Wide Node Name for LUN.

 *

 * Return: Allocated local llun_info structure on success, NULL on failure

 */

static struct llun_info *create_local(struct scsi_device *sdev, u8 *wwid)

{

	struct llun_info *lli = NULL;

	lli = kzalloc(sizeof(*lli), GFP_KERNEL);

	if (unlikely(!lli)) {

		pr_err("%s: could not allocate lli\n", __func__);

		goto out;

	}

	lli->sdev = sdev;

	lli->newly_created = true;

	lli->host_no = sdev->host->host_no;

	memcpy(lli->wwid, wwid, DK_CXLFLASH_MANAGE_LUN_WWID_LEN);

out:

	return lli;

}

/**

 * create_global() - allocate and initialize a global LUN information structure

 * @sdev:	SCSI device associated with LUN.

 * @wwid:	World Wide Node Name for LUN.

 *

 * Return: Allocated global glun_info structure on success, NULL on failure

 */

static struct glun_info *create_global(struct scsi_device *sdev, u8 *wwid)

{

	struct glun_info *gli = NULL;

	gli = kzalloc(sizeof(*gli), GFP_KERNEL);

	if (unlikely(!gli)) {

		pr_err("%s: could not allocate gli\n", __func__);

		goto out;

	}

	mutex_init(&gli->mutex);

	memcpy(gli->wwid, wwid, DK_CXLFLASH_MANAGE_LUN_WWID_LEN);

out:

	return gli;

}

/**

 * refresh_local() - find and update local LUN information structure by WWID

 * @cfg:	Internal structure associated with the host.

 * @wwid:	WWID associated with LUN.

 *

 * When the LUN is found, mark it by updating it's newly_created field.

 *

 * Return: Found local lun_info structure on success, NULL on failure

 * If a LUN with the WWID is found in the list, refresh it's state.

 */

static struct llun_info *refresh_local(struct cxlflash_cfg *cfg, u8 *wwid)

{

	struct llun_info *lli, *temp;

	list_for_each_entry_safe(lli, temp, &cfg->lluns, list)

		if (!memcmp(lli->wwid, wwid, DK_CXLFLASH_MANAGE_LUN_WWID_LEN)) {

			lli->newly_created = false;

			return lli;

		}

	return NULL;

}

/**

 * lookup_global() - find a global LUN information structure by WWID

 * @wwid:	WWID associated with LUN.

 *

 * Return: Found global lun_info structure on success, NULL on failure

 */

static struct glun_info *lookup_global(u8 *wwid)

{

	struct glun_info *gli, *temp;

	list_for_each_entry_safe(gli, temp, &global.gluns, list)

		if (!memcmp(gli->wwid, wwid, DK_CXLFLASH_MANAGE_LUN_WWID_LEN))

			return gli;

	return NULL;

}

/**

 * find_and_create_lun() - find or create a local LUN information structure

 * @sdev:	SCSI device associated with LUN.

 * @wwid:	WWID associated with LUN.

 *

 * The LUN is kept both in a local list (per adapter) and in a global list

 * (across all adapters). Certain attributes of the LUN are local to the

 * adapter (such as index, port selection mask etc.).

 * The block allocation map is shared across all adapters (i.e. associated

 * wih the global list). Since different attributes are associated with

 * the per adapter and global entries, allocate two separate structures for each

 * LUN (one local, one global).

 *

 * Keep a pointer back from the local to the global entry.

 *

 * Return: Found/Allocated local lun_info structure on success, NULL on failure

 */

static struct llun_info *find_and_create_lun(struct scsi_device *sdev, u8 *wwid)

{

	struct llun_info *lli = NULL;

	struct glun_info *gli = NULL;

	struct Scsi_Host *shost = sdev->host;

	struct cxlflash_cfg *cfg = shost_priv(shost);

	mutex_lock(&global.mutex);

	if (unlikely(!wwid))

		goto out;

	lli = refresh_local(cfg, wwid);

	if (lli)

		goto out;

	lli = create_local(sdev, wwid);

	if (unlikely(!lli))

		goto out;

	gli = lookup_global(wwid);

	if (gli) {

		lli->parent = gli;

		list_add(&lli->list, &cfg->lluns);

		goto out;

	}

	gli = create_global(sdev, wwid);

	if (unlikely(!gli)) {

		kfree(lli);

		lli = NULL;

		goto out;

	}

	lli->parent = gli;

	list_add(&lli->list, &cfg->lluns);

	list_add(&gli->list, &global.gluns);

out:

	mutex_unlock(&global.mutex);

	pr_debug("%s: returning %p\n", __func__, lli);

	return lli;

}

/**

 * cxlflash_term_local_luns() - Delete all entries from local LUN list, free.

 * @cfg:	Internal structure associated with the host.

 */

void cxlflash_term_local_luns(struct cxlflash_cfg *cfg)

{

	struct llun_info *lli, *temp;

	mutex_lock(&global.mutex);

	list_for_each_entry_safe(lli, temp, &cfg->lluns, list) {

		list_del(&lli->list);

		kfree(lli);

	}

	mutex_unlock(&global.mutex);

}

/**

 * cxlflash_list_init() - initializes the global LUN list

 */

void cxlflash_list_init(void)

{

	INIT_LIST_HEAD(&global.gluns);

	mutex_init(&global.mutex);

	global.err_page = NULL;

}

/**

 * cxlflash_term_global_luns() - frees resources associated with global LUN list

 */

void cxlflash_term_global_luns(void)

{

	struct glun_info *gli, *temp;

	mutex_lock(&global.mutex);

	list_for_each_entry_safe(gli, temp, &global.gluns, list) {

		list_del(&gli->list);

		kfree(gli);

	}

	mutex_unlock(&global.mutex);

}

/**

 * cxlflash_manage_lun() - handles LUN management activities

 * @sdev:	SCSI device associated with LUN.

 * @manage:	Manage ioctl data structure.

 *

 * This routine is used to notify the driver about a LUN's WWID and associate

 * SCSI devices (sdev) with a global LUN instance. Additionally it serves to

 * change a LUN's operating mode: legacy or superpipe.

 *

 * Return: 0 on success, -errno on failure

 */

int cxlflash_manage_lun(struct scsi_device *sdev,

			struct dk_cxlflash_manage_lun *manage)

{

	int rc = 0;

	struct llun_info *lli = NULL;

	u64 flags = manage->hdr.flags;

	u32 chan = sdev->channel;

	lli = find_and_create_lun(sdev, manage->wwid);

	pr_debug("%s: ENTER: WWID = %016llX%016llX, flags = %016llX li = %p\n",

		 __func__, get_unaligned_le64(&manage->wwid[0]),

		 get_unaligned_le64(&manage->wwid[8]),

		 manage->hdr.flags, lli);

	if (unlikely(!lli)) {

		rc = -ENOMEM;

		goto out;

	}

	if (flags & DK_CXLFLASH_MANAGE_LUN_ENABLE_SUPERPIPE) {

		if (lli->newly_created)

			lli->port_sel = CHAN2PORT(chan);

		else

			lli->port_sel = BOTH_PORTS;

		/* Store off lun in unpacked, AFU-friendly format */

		lli->lun_id[chan] = lun_to_lunid(sdev->lun);

		sdev->hostdata = lli;

	} else if (flags & DK_CXLFLASH_MANAGE_LUN_DISABLE_SUPERPIPE) {

		if (lli->parent->mode != MODE_NONE)

			rc = -EBUSY;

		else

			sdev->hostdata = NULL;

	}

out:

	pr_debug("%s: returning rc=%d\n", __func__, rc);

	return rc;

}