[PATCH] FUSE - device functions

Definitions

~~~~~~~~~~~

Userspace filesystem:

  A filesystem in which data and metadata are provided by an ordinary

  userspace process.  The filesystem can be accessed normally through

  the kernel interface.

Filesystem daemon:

  The process(es) providing the data and metadata of the filesystem.

Non-privileged mount (or user mount):

  A userspace filesystem mounted by a non-privileged (non-root) user.

  The filesystem daemon is running with the privileges of the mounting

  user.  NOTE: this is not the same as mounts allowed with the "user"

  option in /etc/fstab, which is not discussed here.

Mount owner:

  The user who does the mounting.

User:

  The user who is performing filesystem operations.

What is FUSE?

~~~~~~~~~~~~~

FUSE is a userspace filesystem framework.  It consists of a kernel

module (fuse.ko), a userspace library (libfuse.*) and a mount utility

(fusermount).

One of the most important features of FUSE is allowing secure,

non-privileged mounts.  This opens up new possibilities for the use of

filesystems.  A good example is sshfs: a secure network filesystem

using the sftp protocol.

The userspace library and utilities are available from the FUSE

homepage:

  http://fuse.sourceforge.net/

Mount options

~~~~~~~~~~~~~

'fd=N'

  The file descriptor to use for communication between the userspace

  filesystem and the kernel.  The file descriptor must have been

  obtained by opening the FUSE device ('/dev/fuse').

'rootmode=M'

  The file mode of the filesystem's root in octal representation.

'user_id=N'

  The numeric user id of the mount owner.

'group_id=N'

  The numeric group id of the mount owner.

'default_permissions'

  By default FUSE doesn't check file access permissions, the

  filesystem is free to implement it's access policy or leave it to

  the underlying file access mechanism (e.g. in case of network

  filesystems).  This option enables permission checking, restricting

  access based on file mode.  This is option is usually useful

  together with the 'allow_other' mount option.

'allow_other'

  This option overrides the security measure restricting file access

  to the user mounting the filesystem.  This option is by default only

  allowed to root, but this restriction can be removed with a

  (userspace) configuration option.

'kernel_cache'

  This option disables flushing the cache of the file contents on

  every open().  This should only be enabled on filesystems, where the

  file data is never changed externally (not through the mounted FUSE

  filesystem).  Thus it is not suitable for network filesystems and

  other "intermediate" filesystems.

  NOTE: if this option is not specified (and neither 'direct_io') data

  is still cached after the open(), so a read() system call will not

  always initiate a read operation.

'direct_io'

  This option disables the use of page cache (file content cache) in

  the kernel for this filesystem.  This has several affects:

     - Each read() or write() system call will initiate one or more

       read or write operations, data will not be cached in the

       kernel.

     - The return value of the read() and write() system calls will

       correspond to the return values of the read and write

       operations.  This is useful for example if the file size is not

       known in advance (before reading it).

'max_read=N'

  With this option the maximum size of read operations can be set.

  The default is infinite.  Note that the size of read requests is

  limited anyway to 32 pages (which is 128kbyte on i386).

How do non-privileged mounts work?

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Since the mount() system call is a privileged operation, a helper

program (fusermount) is needed, which is installed setuid root.

The implication of providing non-privileged mounts is that the mount

owner must not be able to use this capability to compromise the

system.  Obvious requirements arising from this are:

 A) mount owner should not be able to get elevated privileges with the

    help of the mounted filesystem

 B) mount owner should not get illegitimate access to information from

    other users' and the super user's processes

 C) mount owner should not be able to induce undesired behavior in

    other users' or the super user's processes

How are requirements fulfilled?

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 A) The mount owner could gain elevated privileges by either:

     1) creating a filesystem containing a device file, then opening

	this device

     2) creating a filesystem containing a suid or sgid application,

	then executing this application

    The solution is not to allow opening device files and ignore

    setuid and setgid bits when executing programs.  To ensure this

    fusermount always adds "nosuid" and "nodev" to the mount options

    for non-privileged mounts.

 B) If another user is accessing files or directories in the

    filesystem, the filesystem daemon serving requests can record the

    exact sequence and timing of operations performed.  This

    information is otherwise inaccessible to the mount owner, so this

    counts as an information leak.

    The solution to this problem will be presented in point 2) of C).

 C) There are several ways in which the mount owner can induce

    undesired behavior in other users' processes, such as:

     1) mounting a filesystem over a file or directory which the mount

        owner could otherwise not be able to modify (or could only

        make limited modifications).

        This is solved in fusermount, by checking the access

        permissions on the mountpoint and only allowing the mount if

        the mount owner can do unlimited modification (has write

        access to the mountpoint, and mountpoint is not a "sticky"

        directory)

     2) Even if 1) is solved the mount owner can change the behavior

        of other users' processes.

         i) It can slow down or indefinitely delay the execution of a

           filesystem operation creating a DoS against the user or the

           whole system.  For example a suid application locking a

           system file, and then accessing a file on the mount owner's

           filesystem could be stopped, and thus causing the system

           file to be locked forever.

         ii) It can present files or directories of unlimited length, or

           directory structures of unlimited depth, possibly causing a

           system process to eat up diskspace, memory or other

           resources, again causing DoS.

	The solution to this as well as B) is not to allow processes

	to access the filesystem, which could otherwise not be

	monitored or manipulated by the mount owner.  Since if the

	mount owner can ptrace a process, it can do all of the above

	without using a FUSE mount, the same criteria as used in

	ptrace can be used to check if a process is allowed to access

	the filesystem or not.

	Note that the ptrace check is not strictly necessary to

	prevent B/2/i, it is enough to check if mount owner has enough

	privilege to send signal to the process accessing the

	filesystem, since SIGSTOP can be used to get a similar effect.

I think these limitations are unacceptable?

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If a sysadmin trusts the users enough, or can ensure through other

measures, that system processes will never enter non-privileged

mounts, it can relax the last limitation with a "user_allow_other"

config option.  If this config option is set, the mounting user can

add the "allow_other" mount option which disables the check for other

users' processes.

Kernel - userspace interface

~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The following diagram shows how a filesystem operation (in this

example unlink) is performed in FUSE.

NOTE: everything in this description is greatly simplified

 |  "rm /mnt/fuse/file"               |  FUSE filesystem daemon

 |                                    |

 |                                    |  >sys_read()

 |                                    |    >fuse_dev_read()

 |                                    |      >request_wait()

 |                                    |        [sleep on fc->waitq]

 |                                    |

 |  >sys_unlink()                     |

 |    >fuse_unlink()                  |

 |      [get request from             |

 |       fc->unused_list]             |

 |      >request_send()               |

 |        [queue req on fc->pending]  |

 |        [wake up fc->waitq]         |        [woken up]

 |        >request_wait_answer()      |

 |          [sleep on req->waitq]     |

 |                                    |      <request_wait()

 |                                    |      [remove req from fc->pending]

 |                                    |      [copy req to read buffer]

 |                                    |      [add req to fc->processing]

 |                                    |    <fuse_dev_read()

 |                                    |  <sys_read()

 |                                    |

 |                                    |  [perform unlink]

 |                                    |

 |                                    |  >sys_write()

 |                                    |    >fuse_dev_write()

 |                                    |      [look up req in fc->processing]

 |                                    |      [remove from fc->processing]

 |                                    |      [copy write buffer to req]

 |          [woken up]                |      [wake up req->waitq]

 |                                    |    <fuse_dev_write()

 |                                    |  <sys_write()

 |        <request_wait_answer()      |

 |      <request_send()               |

 |      [add request to               |

 |       fc->unused_list]             |

 |    <fuse_unlink()                  |

 |  <sys_unlink()                     |

There are a couple of ways in which to deadlock a FUSE filesystem.

Since we are talking about unprivileged userspace programs,

something must be done about these.

Scenario 1 -  Simple deadlock

-----------------------------

 |  "rm /mnt/fuse/file"               |  FUSE filesystem daemon

 |                                    |

 |  >sys_unlink("/mnt/fuse/file")     |

 |    [acquire inode semaphore        |

 |     for "file"]                    |

 |    >fuse_unlink()                  |

 |      [sleep on req->waitq]         |

 |                                    |  <sys_read()

 |                                    |  >sys_unlink("/mnt/fuse/file")

 |                                    |    [acquire inode semaphore

 |                                    |     for "file"]

 |                                    |    *DEADLOCK*

The solution for this is to allow requests to be interrupted while

they are in userspace:

 |      [interrupted by signal]       |

 |    <fuse_unlink()                  |

 |    [release semaphore]             |    [semaphore acquired]

 |  <sys_unlink()                     |

 |                                    |    >fuse_unlink()

 |                                    |      [queue req on fc->pending]

 |                                    |      [wake up fc->waitq]

 |                                    |      [sleep on req->waitq]

If the filesystem daemon was single threaded, this will stop here,

since there's no other thread to dequeue and execute the request.

In this case the solution is to kill the FUSE daemon as well.  If

there are multiple serving threads, you just have to kill them as

long as any remain.

Moral: a filesystem which deadlocks, can soon find itself dead.

Scenario 2 - Tricky deadlock

----------------------------

This one needs a carefully crafted filesystem.  It's a variation on

the above, only the call back to the filesystem is not explicit,

but is caused by a pagefault.

 |  Kamikaze filesystem thread 1      |  Kamikaze filesystem thread 2

 |                                    |

 |  [fd = open("/mnt/fuse/file")]     |  [request served normally]

 |  [mmap fd to 'addr']               |

 |  [close fd]                        |  [FLUSH triggers 'magic' flag]

 |  [read a byte from addr]           |

 |    >do_page_fault()                |

 |      [find or create page]         |

 |      [lock page]                   |

 |      >fuse_readpage()              |

 |         [queue READ request]       |

 |         [sleep on req->waitq]      |

 |                                    |  [read request to buffer]

 |                                    |  [create reply header before addr]

 |                                    |  >sys_write(addr - headerlength)

 |                                    |    >fuse_dev_write()

 |                                    |      [look up req in fc->processing]

 |                                    |      [remove from fc->processing]

 |                                    |      [copy write buffer to req]

 |                                    |        >do_page_fault()

 |                                    |           [find or create page]

 |                                    |           [lock page]

 |                                    |           * DEADLOCK *

Solution is again to let the the request be interrupted (not

elaborated further).

An additional problem is that while the write buffer is being

copied to the request, the request must not be interrupted.  This

is because the destination address of the copy may not be valid

after the request is interrupted.

This is solved with doing the copy atomically, and allowing

interruption while the page(s) belonging to the write buffer are

faulted with get_user_pages().  The 'req->locked' flag indicates

when the copy is taking place, and interruption is delayed until

this flag is unset.

obj-$(CONFIG_FUSE_FS) += fuse.o

fuse-objs := inode.o

fuse-objs := dev.o inode.o

#include <linux/backing-dev.h>

#include <asm/semaphore.h>

/** Max number of pages that can be used in a single read request */

#define FUSE_MAX_PAGES_PER_REQ 32

/** If more requests are outstanding, then the operation will block */

#define FUSE_MAX_OUTSTANDING 10

/** FUSE inode */

struct fuse_inode {

	/** Inode data */

	unsigned long i_time;

};

/** One input argument of a request */

struct fuse_in_arg {

	unsigned size;

	const void *value;

};

/** The request input */

struct fuse_in {

	/** The request header */

	struct fuse_in_header h;

	/** True if the data for the last argument is in req->pages */

	unsigned argpages:1;

	/** Number of arguments */

	unsigned numargs;

	/** Array of arguments */

	struct fuse_in_arg args[3];

};

/** One output argument of a request */

struct fuse_arg {

	unsigned size;

	void *value;

};

/** The request output */

struct fuse_out {

	/** Header returned from userspace */

	struct fuse_out_header h;

	/** Last argument is variable length (can be shorter than

	    arg->size) */

	unsigned argvar:1;

	/** Last argument is a list of pages to copy data to */

	unsigned argpages:1;

	/** Zero partially or not copied pages */

	unsigned page_zeroing:1;

	/** Number or arguments */

	unsigned numargs;

	/** Array of arguments */

	struct fuse_arg args[3];

};

struct fuse_req;

struct fuse_conn;

/**

 * A request to the client

 */

struct fuse_req {

	/** This can be on either unused_list, pending or processing

	    lists in fuse_conn */

	struct list_head list;

	/** refcount */

	atomic_t count;

	/** True if the request has reply */

	unsigned isreply:1;

	/** The request is preallocated */

	unsigned preallocated:1;

	/** The request was interrupted */

	unsigned interrupted:1;

	/** Request is sent in the background */

	unsigned background:1;

	/** Data is being copied to/from the request */

	unsigned locked:1;

	/** Request has been sent to userspace */

	unsigned sent:1;

	/** The request is finished */

	unsigned finished:1;

	/** The request input */

	struct fuse_in in;

	/** The request output */

	struct fuse_out out;

	/** Used to wake up the task waiting for completion of request*/

	wait_queue_head_t waitq;

	/** Data for asynchronous requests */

	union {

		struct fuse_init_in_out init_in_out;

	} misc;

	/** page vector */

	struct page *pages[FUSE_MAX_PAGES_PER_REQ];

	/** number of pages in vector */

	unsigned num_pages;

	/** offset of data on first page */

	unsigned page_offset;

	/** Inode used in the request */

	struct inode *inode;

	/** Second inode used in the request (or NULL) */

	struct inode *inode2;

	/** File used in the request (or NULL) */

	struct file *file;

};

/**

 * A Fuse connection.

 *

	/** The superblock of the mounted filesystem */

	struct super_block *sb;

	/** The opened client device */

	struct file *file;

	/** The user id for this mount */

	uid_t user_id;

	/** Readers of the connection are waiting on this */

	wait_queue_head_t waitq;

	/** The list of pending requests */

	struct list_head pending;

	/** The list of requests being processed */

	struct list_head processing;

	/** Controls the maximum number of outstanding requests */

	struct semaphore outstanding_sem;

	/** This counts the number of outstanding requests if

	    outstanding_sem would go negative */

	unsigned outstanding_debt;

	/** The list of unused requests */

	struct list_head unused_list;

	/** The next unique request id */

	u64 reqctr;

	/** Connection failed (version mismatch) */

	unsigned conn_error : 1;

	/** Backing dev info */

	struct backing_dev_info bdi;

};

	return get_fuse_inode(inode)->nodeid;

}

/** Device operations */

extern struct file_operations fuse_dev_operations;

/**

 * This is the single global spinlock which protects FUSE's structures

 *

 * The following data is protected by this lock:

 *

 *  - the private_data field of the device file

 *  - the s_fs_info field of the super block

 *  - unused_list, pending, processing lists in fuse_conn

 *  - the unique request ID counter reqctr in fuse_conn

 *  - the sb (super_block) field in fuse_conn

 *  - the file (device file) field in fuse_conn

 */

extern spinlock_t fuse_lock;

 */

void fuse_release_conn(struct fuse_conn *fc);

/**

 * Initialize the client device

 */

int fuse_dev_init(void);

/**

 * Cleanup the client device

 */

void fuse_dev_cleanup(void);

/**

 * Allocate a request

 */

struct fuse_req *fuse_request_alloc(void);

/**

 * Free a request

 */

void fuse_request_free(struct fuse_req *req);

/**

 * Reinitialize a request, the preallocated flag is left unmodified

 */

void fuse_reset_request(struct fuse_req *req);

/**

 * Reserve a preallocated request

 */

struct fuse_req *fuse_get_request(struct fuse_conn *fc);

/**

 * Reserve a preallocated request, only interruptible by SIGKILL

 */

struct fuse_req *fuse_get_request_nonint(struct fuse_conn *fc);

/**

 * Decrement reference count of a request.  If count goes to zero put

 * on unused list (preallocated) or free reqest (not preallocated).

 */

void fuse_put_request(struct fuse_conn *fc, struct fuse_req *req);

/**

 * Send a request (synchronous, interruptible)

 */

void request_send(struct fuse_conn *fc, struct fuse_req *req);

/**

 * Send a request (synchronous, non-interruptible except by SIGKILL)

 */

void request_send_nonint(struct fuse_conn *fc, struct fuse_req *req);

/**

 * Send a request with no reply

 */

void request_send_noreply(struct fuse_conn *fc, struct fuse_req *req);

/**

 * Send a request in the background

 */

void request_send_background(struct fuse_conn *fc, struct fuse_req *req);

/**

 * Send the INIT message

 */

void fuse_send_init(struct fuse_conn *fc);

	mount_count --;

	fc->sb = NULL;

	fc->user_id = 0;

	/* Flush all readers on this fs */

	wake_up_all(&fc->waitq);

	fuse_release_conn(fc);

	*get_fuse_conn_super_p(sb) = NULL;

	spin_unlock(&fuse_lock);

	return 0;

}

void fuse_release_conn(struct fuse_conn *fc)

static void free_conn(struct fuse_conn *fc)

{

	while (!list_empty(&fc->unused_list)) {

		struct fuse_req *req;

		req = list_entry(fc->unused_list.next, struct fuse_req, list);

		list_del(&req->list);

		fuse_request_free(req);

	}

	kfree(fc);

}

/* Must be called with the fuse lock held */

void fuse_release_conn(struct fuse_conn *fc)

{

	if (!fc->sb && !fc->file)

		free_conn(fc);

}

static struct fuse_conn *new_conn(void)

{

	struct fuse_conn *fc;

	fc = kmalloc(sizeof(*fc), GFP_KERNEL);

	if (fc != NULL) {

		int i;

		memset(fc, 0, sizeof(*fc));

		fc->sb = NULL;

		fc->file = NULL;

		fc->user_id = 0;

		init_waitqueue_head(&fc->waitq);

		INIT_LIST_HEAD(&fc->pending);

		INIT_LIST_HEAD(&fc->processing);

		INIT_LIST_HEAD(&fc->unused_list);

		sema_init(&fc->outstanding_sem, 0);

		for (i = 0; i < FUSE_MAX_OUTSTANDING; i++) {

			struct fuse_req *req = fuse_request_alloc();

			if (!req) {

				free_conn(fc);

				return NULL;

			}

			list_add(&req->list, &fc->unused_list);

		}

		fc->bdi.ra_pages = (VM_MAX_READAHEAD * 1024) / PAGE_CACHE_SIZE;

		fc->bdi.unplug_io_fn = default_unplug_io_fn;

		fc->reqctr = 0;

	}

	return fc;

}

{

	struct fuse_conn *fc;

	if (file->f_op != &fuse_dev_operations)

		return ERR_PTR(-EINVAL);

	fc = new_conn();

	if (fc == NULL)

		return NULL;

		return ERR_PTR(-ENOMEM);

	spin_lock(&fuse_lock);

	if (file->private_data) {

		free_conn(fc);

		fc = ERR_PTR(-EINVAL);

	} else {

		file->private_data = fc;

		fc->sb = sb;

		fc->file = file;

	}

	spin_unlock(&fuse_lock);

	return fc;

}

	fc = get_conn(file, sb);

	fput(file);

	if (fc == NULL)

		return -EINVAL;

	if (IS_ERR(fc))

		return PTR_ERR(fc);

	fc->user_id = d.user_id;

		iput(root);

		goto err;

	}

	fuse_send_init(fc);

	return 0;

 err:

	if (res)

		goto err;

	res = fuse_dev_init();

	if (res)

		goto err_fs_cleanup;

	return 0;

 err_fs_cleanup:

	fuse_fs_cleanup();

 err:

	return res;

}

	printk(KERN_DEBUG "fuse exit\n");

	fuse_fs_cleanup();

	fuse_dev_cleanup();

}

module_init(fuse_init);