Merge tag 'fscrypt-for-linus' of git://git.kernel.org/pub/scm/fs/fscrypt/fscrypt

the master keys may be wrapped in userspace, e.g. as is done by the

`fscrypt <https://github.com/google/fscrypt>`_ tool.

Including the inode number in the IVs was considered.  However, it was

rejected as it would have prevented ext4 filesystems from being

resized, and by itself still wouldn't have been sufficient to prevent

the same key from being directly reused for both XTS and CTS-CBC.

DIRECT_KEY and per-mode keys

----------------------------

DIRECT_KEY policies

-------------------

The Adiantum encryption mode (see `Encryption modes and usage`_) is

suitable for both contents and filenames encryption, and it accepts

  key derived using the KDF.  Users may use the same master key for

  other v2 encryption policies.

IV_INO_LBLK_64 policies

-----------------------

When FSCRYPT_POLICY_FLAG_IV_INO_LBLK_64 is set in the fscrypt policy,

the encryption keys are derived from the master key, encryption mode

number, and filesystem UUID.  This normally results in all files

protected by the same master key sharing a single contents encryption

key and a single filenames encryption key.  To still encrypt different

files' data differently, inode numbers are included in the IVs.

Consequently, shrinking the filesystem may not be allowed.

This format is optimized for use with inline encryption hardware

compliant with the UFS or eMMC standards, which support only 64 IV

bits per I/O request and may have only a small number of keyslots.

Key identifiers

---------------

AES-128-CBC was added only for low-powered embedded devices with

crypto accelerators such as CAAM or CESA that do not support XTS.  To

use AES-128-CBC, CONFIG_CRYPTO_SHA256 (or another SHA-256

implementation) must be enabled so that ESSIV can be used.

use AES-128-CBC, CONFIG_CRYPTO_ESSIV and CONFIG_CRYPTO_SHA256 (or

another SHA-256 implementation) must be enabled so that ESSIV can be

used.

Adiantum is a (primarily) stream cipher-based mode that is fast even

on CPUs without dedicated crypto instructions.  It's also a true

  is encrypted with AES-256 where the AES-256 key is the SHA-256 hash

  of the file's data encryption key.

- In the "direct key" configuration (FSCRYPT_POLICY_FLAG_DIRECT_KEY

  set in the fscrypt_policy), the file's nonce is also appended to the

  IV.  Currently this is only allowed with the Adiantum encryption

  mode.

- With `DIRECT_KEY policies`_, the file's nonce is appended to the IV.

  Currently this is only allowed with the Adiantum encryption mode.

- With `IV_INO_LBLK_64 policies`_, the logical block number is limited

  to 32 bits and is placed in bits 0-31 of the IV.  The inode number

  (which is also limited to 32 bits) is placed in bits 32-63.

Note that because file logical block numbers are included in the IVs,

filesystems must enforce that blocks are never shifted around within

encrypted files, e.g. via "collapse range" or "insert range".

Filenames encryption

--------------------

filenames of up to 255 bytes, the same IV is used for every filename

in a directory.

However, each encrypted directory still uses a unique key; or

alternatively (for the "direct key" configuration) has the file's

nonce included in the IVs.  Thus, IV reuse is limited to within a

single directory.

However, each encrypted directory still uses a unique key, or

alternatively has the file's nonce (for `DIRECT_KEY policies`_) or

inode number (for `IV_INO_LBLK_64 policies`_) included in the IVs.

Thus, IV reuse is limited to within a single directory.

With CTS-CBC, the IV reuse means that when the plaintext filenames

share a common prefix at least as long as the cipher block size (16

  (1) for ``contents_encryption_mode`` and FSCRYPT_MODE_AES_256_CTS

  (4) for ``filenames_encryption_mode``.

- ``flags`` must contain a value from ``<linux/fscrypt.h>`` which

  identifies the amount of NUL-padding to use when encrypting

  filenames.  If unsure, use FSCRYPT_POLICY_FLAGS_PAD_32 (0x3).

  Additionally, if the encryption modes are both

  FSCRYPT_MODE_ADIANTUM, this can contain

  FSCRYPT_POLICY_FLAG_DIRECT_KEY; see `DIRECT_KEY and per-mode keys`_.

- ``flags`` contains optional flags from ``<linux/fscrypt.h>``:

  - FSCRYPT_POLICY_FLAGS_PAD_*: The amount of NUL padding to use when

    encrypting filenames.  If unsure, use FSCRYPT_POLICY_FLAGS_PAD_32

    (0x3).

  - FSCRYPT_POLICY_FLAG_DIRECT_KEY: See `DIRECT_KEY policies`_.

  - FSCRYPT_POLICY_FLAG_IV_INO_LBLK_64: See `IV_INO_LBLK_64

    policies`_.  This is mutually exclusive with DIRECT_KEY and is not

    supported on v1 policies.

- For v2 encryption policies, ``__reserved`` must be zeroed.

context structs also contain a nonce.  The nonce is randomly generated

by the kernel and is used as KDF input or as a tweak to cause

different files to be encrypted differently; see `Per-file keys`_ and

`DIRECT_KEY and per-mode keys`_.

`DIRECT_KEY policies`_.

Data path changes

-----------------

'f'   00-0F  fs/ext4/ext4.h                                          conflict!

'f'   00-0F  linux/fs.h                                              conflict!

'f'   00-0F  fs/ocfs2/ocfs2_fs.h                                     conflict!

'f'   13-27  linux/fscrypt.h

'f'   81-8F  linux/fsverity.h

'g'   00-0F  linux/usb/gadgetfs.h

'g'   20-2F  linux/usb/g_printer.h

#include <linux/namei.h>

#include "fscrypt_private.h"

static void __fscrypt_decrypt_bio(struct bio *bio, bool done)

void fscrypt_decrypt_bio(struct bio *bio)

{

	struct bio_vec *bv;

	struct bvec_iter_all iter_all;

							   bv->bv_offset);

		if (ret)

			SetPageError(page);

		else if (done)

			SetPageUptodate(page);

		if (done)

			unlock_page(page);

	}

}

void fscrypt_decrypt_bio(struct bio *bio)

{

	__fscrypt_decrypt_bio(bio, false);

}

EXPORT_SYMBOL(fscrypt_decrypt_bio);

static void completion_pages(struct work_struct *work)

{

	struct fscrypt_ctx *ctx = container_of(work, struct fscrypt_ctx, work);

	struct bio *bio = ctx->bio;

	__fscrypt_decrypt_bio(bio, true);

	fscrypt_release_ctx(ctx);

	bio_put(bio);

}

void fscrypt_enqueue_decrypt_bio(struct fscrypt_ctx *ctx, struct bio *bio)

{

	INIT_WORK(&ctx->work, completion_pages);

	ctx->bio = bio;

	fscrypt_enqueue_decrypt_work(&ctx->work);

}

EXPORT_SYMBOL(fscrypt_enqueue_decrypt_bio);

int fscrypt_zeroout_range(const struct inode *inode, pgoff_t lblk,

				sector_t pblk, unsigned int len)

{

#include <linux/ratelimit.h>

#include <linux/dcache.h>

#include <linux/namei.h>

#include <crypto/aes.h>

#include <crypto/skcipher.h>

#include "fscrypt_private.h"

static unsigned int num_prealloc_crypto_pages = 32;

static unsigned int num_prealloc_crypto_ctxs = 128;

module_param(num_prealloc_crypto_pages, uint, 0444);

MODULE_PARM_DESC(num_prealloc_crypto_pages,

		"Number of crypto pages to preallocate");

module_param(num_prealloc_crypto_ctxs, uint, 0444);

MODULE_PARM_DESC(num_prealloc_crypto_ctxs,

		"Number of crypto contexts to preallocate");

static mempool_t *fscrypt_bounce_page_pool = NULL;

static LIST_HEAD(fscrypt_free_ctxs);

static DEFINE_SPINLOCK(fscrypt_ctx_lock);

static struct workqueue_struct *fscrypt_read_workqueue;

static DEFINE_MUTEX(fscrypt_init_mutex);

static struct kmem_cache *fscrypt_ctx_cachep;

struct kmem_cache *fscrypt_info_cachep;

void fscrypt_enqueue_decrypt_work(struct work_struct *work)

}

EXPORT_SYMBOL(fscrypt_enqueue_decrypt_work);

/**

 * fscrypt_release_ctx() - Release a decryption context

 * @ctx: The decryption context to release.

 *

 * If the decryption context was allocated from the pre-allocated pool, return

 * it to that pool.  Else, free it.

 */

void fscrypt_release_ctx(struct fscrypt_ctx *ctx)

{

	unsigned long flags;

	if (ctx->flags & FS_CTX_REQUIRES_FREE_ENCRYPT_FL) {

		kmem_cache_free(fscrypt_ctx_cachep, ctx);

	} else {

		spin_lock_irqsave(&fscrypt_ctx_lock, flags);

		list_add(&ctx->free_list, &fscrypt_free_ctxs);

		spin_unlock_irqrestore(&fscrypt_ctx_lock, flags);

	}

}

EXPORT_SYMBOL(fscrypt_release_ctx);

/**

 * fscrypt_get_ctx() - Get a decryption context

 * @gfp_flags:   The gfp flag for memory allocation

 *

 * Allocate and initialize a decryption context.

 *

 * Return: A new decryption context on success; an ERR_PTR() otherwise.

 */

struct fscrypt_ctx *fscrypt_get_ctx(gfp_t gfp_flags)

{

	struct fscrypt_ctx *ctx;

	unsigned long flags;

	/*

	 * First try getting a ctx from the free list so that we don't have to

	 * call into the slab allocator.

	 */

	spin_lock_irqsave(&fscrypt_ctx_lock, flags);

	ctx = list_first_entry_or_null(&fscrypt_free_ctxs,

					struct fscrypt_ctx, free_list);

	if (ctx)

		list_del(&ctx->free_list);

	spin_unlock_irqrestore(&fscrypt_ctx_lock, flags);

	if (!ctx) {

		ctx = kmem_cache_zalloc(fscrypt_ctx_cachep, gfp_flags);

		if (!ctx)

			return ERR_PTR(-ENOMEM);

		ctx->flags |= FS_CTX_REQUIRES_FREE_ENCRYPT_FL;

	} else {

		ctx->flags &= ~FS_CTX_REQUIRES_FREE_ENCRYPT_FL;

	}

	return ctx;

}

EXPORT_SYMBOL(fscrypt_get_ctx);

struct page *fscrypt_alloc_bounce_page(gfp_t gfp_flags)

{

	return mempool_alloc(fscrypt_bounce_page_pool, gfp_flags);

void fscrypt_generate_iv(union fscrypt_iv *iv, u64 lblk_num,

			 const struct fscrypt_info *ci)

{

	u8 flags = fscrypt_policy_flags(&ci->ci_policy);

	memset(iv, 0, ci->ci_mode->ivsize);

	iv->lblk_num = cpu_to_le64(lblk_num);

	if (fscrypt_is_direct_key_policy(&ci->ci_policy))

	if (flags & FSCRYPT_POLICY_FLAG_IV_INO_LBLK_64) {

		WARN_ON_ONCE((u32)lblk_num != lblk_num);

		lblk_num |= (u64)ci->ci_inode->i_ino << 32;

	} else if (flags & FSCRYPT_POLICY_FLAG_DIRECT_KEY) {

		memcpy(iv->nonce, ci->ci_nonce, FS_KEY_DERIVATION_NONCE_SIZE);

	if (ci->ci_essiv_tfm != NULL)

		crypto_cipher_encrypt_one(ci->ci_essiv_tfm, iv->raw, iv->raw);

	}

	iv->lblk_num = cpu_to_le64(lblk_num);

}

/* Encrypt or decrypt a single filesystem block of file contents */

	.d_revalidate = fscrypt_d_revalidate,

};

static void fscrypt_destroy(void)

{

	struct fscrypt_ctx *pos, *n;

	list_for_each_entry_safe(pos, n, &fscrypt_free_ctxs, free_list)

		kmem_cache_free(fscrypt_ctx_cachep, pos);

	INIT_LIST_HEAD(&fscrypt_free_ctxs);

	mempool_destroy(fscrypt_bounce_page_pool);

	fscrypt_bounce_page_pool = NULL;

}

/**

 * fscrypt_initialize() - allocate major buffers for fs encryption.

 * @cop_flags:  fscrypt operations flags

 * We only call this when we start accessing encrypted files, since it

 * results in memory getting allocated that wouldn't otherwise be used.

 *

 * Return: Zero on success, non-zero otherwise.

 * Return: 0 on success; -errno on failure

 */

int fscrypt_initialize(unsigned int cop_flags)

{

	int i, res = -ENOMEM;

	int err = 0;

	/* No need to allocate a bounce page pool if this FS won't use it. */

	if (cop_flags & FS_CFLG_OWN_PAGES)

	mutex_lock(&fscrypt_init_mutex);

	if (fscrypt_bounce_page_pool)

		goto already_initialized;

	for (i = 0; i < num_prealloc_crypto_ctxs; i++) {

		struct fscrypt_ctx *ctx;

		ctx = kmem_cache_zalloc(fscrypt_ctx_cachep, GFP_NOFS);

		if (!ctx)

			goto fail;

		list_add(&ctx->free_list, &fscrypt_free_ctxs);

	}

		goto out_unlock;

	err = -ENOMEM;

	fscrypt_bounce_page_pool =

		mempool_create_page_pool(num_prealloc_crypto_pages, 0);

	if (!fscrypt_bounce_page_pool)

		goto fail;

		goto out_unlock;

already_initialized:

	mutex_unlock(&fscrypt_init_mutex);

	return 0;

fail:

	fscrypt_destroy();

	err = 0;

out_unlock:

	mutex_unlock(&fscrypt_init_mutex);

	return res;

	return err;

}

void fscrypt_msg(const struct inode *inode, const char *level,

	if (!fscrypt_read_workqueue)

		goto fail;

	fscrypt_ctx_cachep = KMEM_CACHE(fscrypt_ctx, SLAB_RECLAIM_ACCOUNT);

	if (!fscrypt_ctx_cachep)

		goto fail_free_queue;

	fscrypt_info_cachep = KMEM_CACHE(fscrypt_info, SLAB_RECLAIM_ACCOUNT);

	if (!fscrypt_info_cachep)

		goto fail_free_ctx;

		goto fail_free_queue;

	err = fscrypt_init_keyring();

	if (err)

fail_free_info:

	kmem_cache_destroy(fscrypt_info_cachep);

fail_free_ctx:

	kmem_cache_destroy(fscrypt_ctx_cachep);

fail_free_queue:

	destroy_workqueue(fscrypt_read_workqueue);

fail:

	/* The actual crypto transform used for encryption and decryption */

	struct crypto_skcipher *ci_ctfm;

	/*

	 * Cipher for ESSIV IV generation.  Only set for CBC contents

	 * encryption, otherwise is NULL.

	 */

	struct crypto_cipher *ci_essiv_tfm;

	/* True if the key should be freed when this fscrypt_info is freed */

	bool ci_owns_key;

	/*

	 * Encryption mode used for this inode.  It corresponds to either the

	FS_ENCRYPT,

} fscrypt_direction_t;

#define FS_CTX_REQUIRES_FREE_ENCRYPT_FL		0x00000001

static inline bool fscrypt_valid_enc_modes(u32 contents_mode,

					   u32 filenames_mode)

{

 */

#define HKDF_CONTEXT_KEY_IDENTIFIER	1

#define HKDF_CONTEXT_PER_FILE_KEY	2

#define HKDF_CONTEXT_PER_MODE_KEY	3

#define HKDF_CONTEXT_DIRECT_KEY		3

#define HKDF_CONTEXT_IV_INO_LBLK_64_KEY	4

extern int fscrypt_hkdf_expand(struct fscrypt_hkdf *hkdf, u8 context,

			       const u8 *info, unsigned int infolen,

	struct list_head	mk_decrypted_inodes;

	spinlock_t		mk_decrypted_inodes_lock;

	/* Per-mode tfms for DIRECT_KEY policies, allocated on-demand */

	struct crypto_skcipher	*mk_mode_keys[__FSCRYPT_MODE_MAX + 1];

	/* Crypto API transforms for DIRECT_KEY policies, allocated on-demand */

	struct crypto_skcipher	*mk_direct_tfms[__FSCRYPT_MODE_MAX + 1];

	/*

	 * Crypto API transforms for filesystem-layer implementation of

	 * IV_INO_LBLK_64 policies, allocated on-demand.

	 */

	struct crypto_skcipher	*mk_iv_ino_lblk_64_tfms[__FSCRYPT_MODE_MAX + 1];

} __randomize_layout;

	const char *cipher_str;

	int keysize;

	int ivsize;

	bool logged_impl_name;

	bool needs_essiv;

	int logged_impl_name;

};

static inline bool