Merge tag 'locking-core-2020-06-01' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip (60056060) · Commits · 戴 / test

Documentation/locking/locktypes.rst

+204 −11

Original line number	Diff line number	Diff line
		@@ -13,6 +13,7 @@ The kernel provides a variety of locking primitives which can be divided
		into two categories:

		- Sleeping locks
		- CPU local locks
		- Spinning locks

		This document conceptually describes these lock types and provides rules
		@@ -44,9 +45,23 @@ Sleeping lock types:

		On PREEMPT_RT kernels, these lock types are converted to sleeping locks:

		- local_lock
		- spinlock_t
		- rwlock_t


		CPU local locks
		---------------

		- local_lock

		On non-PREEMPT_RT kernels, local_lock functions are wrappers around
		preemption and interrupt disabling primitives. Contrary to other locking
		mechanisms, disabling preemption or interrupts are pure CPU local
		concurrency control mechanisms and not suited for inter-CPU concurrency
		control.


		Spinning locks
		--------------

		@@ -67,6 +82,7 @@ can have suffixes which apply further protections:
		_irqsave/restore() Save and disable / restore interrupt disabled state
		=================== ====================================================


		Owner semantics
		===============

		@@ -139,6 +155,56 @@ implementation, thus changing the fairness:
		writer from starving readers.


		local_lock
		==========

		local_lock provides a named scope to critical sections which are protected
		by disabling preemption or interrupts.

		On non-PREEMPT_RT kernels local_lock operations map to the preemption and
		interrupt disabling and enabling primitives:

		=========================== ======================
		local_lock(&llock) preempt_disable()
		local_unlock(&llock) preempt_enable()
		local_lock_irq(&llock) local_irq_disable()
		local_unlock_irq(&llock) local_irq_enable()
		local_lock_save(&llock) local_irq_save()
		local_lock_restore(&llock) local_irq_save()
		=========================== ======================

		The named scope of local_lock has two advantages over the regular
		primitives:

		- The lock name allows static analysis and is also a clear documentation
		of the protection scope while the regular primitives are scopeless and
		opaque.

		- If lockdep is enabled the local_lock gains a lockmap which allows to
		validate the correctness of the protection. This can detect cases where
		e.g. a function using preempt_disable() as protection mechanism is
		invoked from interrupt or soft-interrupt context. Aside of that
		lockdep_assert_held(&llock) works as with any other locking primitive.

		local_lock and PREEMPT_RT
		-------------------------

		PREEMPT_RT kernels map local_lock to a per-CPU spinlock_t, thus changing
		semantics:

		- All spinlock_t changes also apply to local_lock.

		local_lock usage
		----------------

		local_lock should be used in situations where disabling preemption or
		interrupts is the appropriate form of concurrency control to protect
		per-CPU data structures on a non PREEMPT_RT kernel.

		local_lock is not suitable to protect against preemption or interrupts on a
		PREEMPT_RT kernel due to the PREEMPT_RT specific spinlock_t semantics.


		raw_spinlock_t and spinlock_t
		=============================

		@@ -258,10 +324,82 @@ implementation, thus changing semantics:
		PREEMPT_RT caveats
		==================

		local_lock on RT
		----------------

		The mapping of local_lock to spinlock_t on PREEMPT_RT kernels has a few
		implications. For example, on a non-PREEMPT_RT kernel the following code
		sequence works as expected::

		local_lock_irq(&local_lock);
		raw_spin_lock(&lock);

		and is fully equivalent to::

		raw_spin_lock_irq(&lock);

		On a PREEMPT_RT kernel this code sequence breaks because local_lock_irq()
		is mapped to a per-CPU spinlock_t which neither disables interrupts nor
		preemption. The following code sequence works perfectly correct on both
		PREEMPT_RT and non-PREEMPT_RT kernels::

		local_lock_irq(&local_lock);
		spin_lock(&lock);

		Another caveat with local locks is that each local_lock has a specific
		protection scope. So the following substitution is wrong::

		func1()
		{
		local_irq_save(flags); -> local_lock_irqsave(&local_lock_1, flags);
		func3();
		local_irq_restore(flags); -> local_lock_irqrestore(&local_lock_1, flags);
		}

		func2()
		{
		local_irq_save(flags); -> local_lock_irqsave(&local_lock_2, flags);
		func3();
		local_irq_restore(flags); -> local_lock_irqrestore(&local_lock_2, flags);
		}

		func3()
		{
		lockdep_assert_irqs_disabled();
		access_protected_data();
		}

		On a non-PREEMPT_RT kernel this works correctly, but on a PREEMPT_RT kernel
		local_lock_1 and local_lock_2 are distinct and cannot serialize the callers
		of func3(). Also the lockdep assert will trigger on a PREEMPT_RT kernel
		because local_lock_irqsave() does not disable interrupts due to the
		PREEMPT_RT-specific semantics of spinlock_t. The correct substitution is::

		func1()
		{
		local_irq_save(flags); -> local_lock_irqsave(&local_lock, flags);
		func3();
		local_irq_restore(flags); -> local_lock_irqrestore(&local_lock, flags);
		}

		func2()
		{
		local_irq_save(flags); -> local_lock_irqsave(&local_lock, flags);
		func3();
		local_irq_restore(flags); -> local_lock_irqrestore(&local_lock, flags);
		}

		func3()
		{
		lockdep_assert_held(&local_lock);
		access_protected_data();
		}


		spinlock_t and rwlock_t
		-----------------------

		These changes in spinlock_t and rwlock_t semantics on PREEMPT_RT kernels
		The changes in spinlock_t and rwlock_t semantics on PREEMPT_RT kernels
		have a few implications. For example, on a non-PREEMPT_RT kernel the
		following code sequence works as expected::

		@@ -282,9 +420,61 @@ local_lock mechanism. Acquiring the local_lock pins the task to a CPU,
		allowing things like per-CPU interrupt disabled locks to be acquired.
		However, this approach should be used only where absolutely necessary.

		A typical scenario is protection of per-CPU variables in thread context::

		raw_spinlock_t
		--------------
		struct foo *p = get_cpu_ptr(&var1);

		spin_lock(&p->lock);
		p->count += this_cpu_read(var2);

		This is correct code on a non-PREEMPT_RT kernel, but on a PREEMPT_RT kernel
		this breaks. The PREEMPT_RT-specific change of spinlock_t semantics does
		not allow to acquire p->lock because get_cpu_ptr() implicitly disables
		preemption. The following substitution works on both kernels::

		struct foo *p;

		migrate_disable();
		p = this_cpu_ptr(&var1);
		spin_lock(&p->lock);
		p->count += this_cpu_read(var2);

		On a non-PREEMPT_RT kernel migrate_disable() maps to preempt_disable()
		which makes the above code fully equivalent. On a PREEMPT_RT kernel
		migrate_disable() ensures that the task is pinned on the current CPU which
		in turn guarantees that the per-CPU access to var1 and var2 are staying on
		the same CPU.

		The migrate_disable() substitution is not valid for the following
		scenario::

		func()
		{
		struct foo *p;

		migrate_disable();
		p = this_cpu_ptr(&var1);
		p->val = func2();

		While correct on a non-PREEMPT_RT kernel, this breaks on PREEMPT_RT because
		here migrate_disable() does not protect against reentrancy from a
		preempting task. A correct substitution for this case is::

		func()
		{
		struct foo *p;

		local_lock(&foo_lock);
		p = this_cpu_ptr(&var1);
		p->val = func2();

		On a non-PREEMPT_RT kernel this protects against reentrancy by disabling
		preemption. On a PREEMPT_RT kernel this is achieved by acquiring the
		underlying per-CPU spinlock.


		raw_spinlock_t on RT
		--------------------

		Acquiring a raw_spinlock_t disables preemption and possibly also
		interrupts, so the critical section must avoid acquiring a regular
		@@ -325,22 +515,25 @@ Lock type nesting rules

		The most basic rules are:

		- Lock types of the same lock category (sleeping, spinning) can nest
		arbitrarily as long as they respect the general lock ordering rules to
		prevent deadlocks.
		- Lock types of the same lock category (sleeping, CPU local, spinning)
		can nest arbitrarily as long as they respect the general lock ordering
		rules to prevent deadlocks.

		- Sleeping lock types cannot nest inside CPU local and spinning lock types.

		- Sleeping lock types cannot nest inside spinning lock types.
		- CPU local and spinning lock types can nest inside sleeping lock types.

		- Spinning lock types can nest inside sleeping lock types.
		- Spinning lock types can nest inside all lock types

		These constraints apply both in PREEMPT_RT and otherwise.

		The fact that PREEMPT_RT changes the lock category of spinlock_t and
		rwlock_t from spinning to sleeping means that they cannot be acquired while
		holding a raw spinlock. This results in the following nesting ordering:
		rwlock_t from spinning to sleeping and substitutes local_lock with a
		per-CPU spinlock_t means that they cannot be acquired while holding a raw
		spinlock. This results in the following nesting ordering:

		1) Sleeping locks
		2) spinlock_t and rwlock_t
		2) spinlock_t, rwlock_t, local_lock
		3) raw_spinlock_t and bit spinlocks

		Lockdep will complain if these constraints are violated, both in

drivers/block/zram/zcomp.c

+19 −25

Original line number	Diff line number	Diff line
		@@ -37,19 +37,16 @@ static void zcomp_strm_free(struct zcomp_strm *zstrm)
		if (!IS_ERR_OR_NULL(zstrm->tfm))
		crypto_free_comp(zstrm->tfm);
		free_pages((unsigned long)zstrm->buffer, 1);
		kfree(zstrm);
		zstrm->tfm = NULL;
		zstrm->buffer = NULL;
		}

		/*
		* allocate new zcomp_strm structure with ->tfm initialized by
		* backend, return NULL on error
		* Initialize zcomp_strm structure with ->tfm initialized by backend, and
		* ->buffer. Return a negative value on error.
		*/
		static struct zcomp_strm zcomp_strm_alloc(struct zcomp comp)
		static int zcomp_strm_init(struct zcomp_strm zstrm, struct zcomp comp)
		{
		struct zcomp_strm zstrm = kmalloc(sizeof(zstrm), GFP_KERNEL);
		if (!zstrm)
		return NULL;

		zstrm->tfm = crypto_alloc_comp(comp->name, 0, 0);
		/*
		* allocate 2 pages. 1 for compressed data, plus 1 extra for the
		@@ -58,9 +55,9 @@ static struct zcomp_strm zcomp_strm_alloc(struct zcomp comp)
		zstrm->buffer = (void *)__get_free_pages(GFP_KERNEL \| __GFP_ZERO, 1);
		if (IS_ERR_OR_NULL(zstrm->tfm) \|\| !zstrm->buffer) {
		zcomp_strm_free(zstrm);
		zstrm = NULL;
		return -ENOMEM;
		}
		return zstrm;
		return 0;
		}

		bool zcomp_available_algorithm(const char *comp)
		@@ -113,12 +110,13 @@ ssize_t zcomp_available_show(const char comp, char buf)

		struct zcomp_strm zcomp_stream_get(struct zcomp comp)
		{
		return *get_cpu_ptr(comp->stream);
		local_lock(&comp->stream->lock);
		return this_cpu_ptr(comp->stream);
		}

		void zcomp_stream_put(struct zcomp *comp)
		{
		put_cpu_ptr(comp->stream);
		local_unlock(&comp->stream->lock);
		}

		int zcomp_compress(struct zcomp_strm *zstrm,
		@@ -159,17 +157,15 @@ int zcomp_cpu_up_prepare(unsigned int cpu, struct hlist_node *node)
		{
		struct zcomp *comp = hlist_entry(node, struct zcomp, node);
		struct zcomp_strm *zstrm;
		int ret;

		if (WARN_ON(*per_cpu_ptr(comp->stream, cpu)))
		return 0;
		zstrm = per_cpu_ptr(comp->stream, cpu);
		local_lock_init(&zstrm->lock);

		zstrm = zcomp_strm_alloc(comp);
		if (IS_ERR_OR_NULL(zstrm)) {
		ret = zcomp_strm_init(zstrm, comp);
		if (ret)
		pr_err("Can't allocate a compression stream\n");
		return -ENOMEM;
		}
		*per_cpu_ptr(comp->stream, cpu) = zstrm;
		return 0;
		return ret;
		}

		int zcomp_cpu_dead(unsigned int cpu, struct hlist_node *node)
		@@ -177,10 +173,8 @@ int zcomp_cpu_dead(unsigned int cpu, struct hlist_node *node)
		struct zcomp *comp = hlist_entry(node, struct zcomp, node);
		struct zcomp_strm *zstrm;

		zstrm = *per_cpu_ptr(comp->stream, cpu);
		if (!IS_ERR_OR_NULL(zstrm))
		zstrm = per_cpu_ptr(comp->stream, cpu);
		zcomp_strm_free(zstrm);
		*per_cpu_ptr(comp->stream, cpu) = NULL;
		return 0;
		}

		@@ -188,7 +182,7 @@ static int zcomp_init(struct zcomp *comp)
		{
		int ret;

		comp->stream = alloc_percpu(struct zcomp_strm *);
		comp->stream = alloc_percpu(struct zcomp_strm);
		if (!comp->stream)
		return -ENOMEM;

drivers/block/zram/zcomp.h

+4 −1

Original line number	Diff line number	Diff line
		@@ -5,8 +5,11 @@

		#ifndef _ZCOMP_H_
		#define _ZCOMP_H_
		#include <linux/local_lock.h>

		struct zcomp_strm {
		/* The members ->buffer and ->tfm are protected by ->lock. */
		local_lock_t lock;
		/* compression/decompression buffer */
		void *buffer;
		struct crypto_comp *tfm;
		@@ -14,7 +17,7 @@ struct zcomp_strm {

		/* dynamic per-device compression frontend */
		struct zcomp {
		struct zcomp_strm * __percpu *stream;
		struct zcomp_strm __percpu *stream;
		const char *name;
		struct hlist_node node;
		};

drivers/connector/cn_proc.c

+14 −7

Original line number	Diff line number	Diff line
		@@ -18,6 +18,7 @@
		#include <linux/pid_namespace.h>

		#include <linux/cn_proc.h>
		#include <linux/local_lock.h>

		/*
		* Size of a cn_msg followed by a proc_event structure. Since the
		@@ -38,25 +39,31 @@ static inline struct cn_msg buffer_to_cn_msg(__u8 buffer)
		static atomic_t proc_event_num_listeners = ATOMIC_INIT(0);
		static struct cb_id cn_proc_event_id = { CN_IDX_PROC, CN_VAL_PROC };

		/* proc_event_counts is used as the sequence number of the netlink message */
		static DEFINE_PER_CPU(__u32, proc_event_counts) = { 0 };
		/* local_event.count is used as the sequence number of the netlink message */
		struct local_event {
		local_lock_t lock;
		__u32 count;
		};
		static DEFINE_PER_CPU(struct local_event, local_event) = {
		.lock = INIT_LOCAL_LOCK(lock),
		};

		static inline void send_msg(struct cn_msg *msg)
		{
		preempt_disable();
		local_lock(&local_event.lock);

		msg->seq = __this_cpu_inc_return(proc_event_counts) - 1;
		msg->seq = __this_cpu_inc_return(local_event.count) - 1;
		((struct proc_event *)msg->data)->cpu = smp_processor_id();

		/*
		* Preemption remains disabled during send to ensure the messages are
		* ordered according to their sequence numbers.
		* local_lock() disables preemption during send to ensure the messages
		* are ordered according to their sequence numbers.
		*
		* If cn_netlink_send() fails, the data is not sent.
		*/
		cn_netlink_send(msg, 0, CN_IDX_PROC, GFP_NOWAIT);

		preempt_enable();
		local_unlock(&local_event.lock);
		}

		void proc_fork_connector(struct task_struct *task)

fs/squashfs/decompressor_multi_percpu.c

+14 −7

Original line number	Diff line number	Diff line
		@@ -8,6 +8,7 @@
		#include <linux/slab.h>
		#include <linux/percpu.h>
		#include <linux/buffer_head.h>
		#include <linux/local_lock.h>

		#include "squashfs_fs.h"
		#include "squashfs_fs_sb.h"
		@@ -21,6 +22,7 @@

		struct squashfs_stream {
		void *stream;
		local_lock_t lock;
		};

		void squashfs_decompressor_create(struct squashfs_sb_info msblk,
		@@ -41,6 +43,7 @@ void squashfs_decompressor_create(struct squashfs_sb_info msblk,
		err = PTR_ERR(stream->stream);
		goto out;
		}
		local_lock_init(&stream->lock);
		}

		kfree(comp_opts);
		@@ -75,12 +78,16 @@ void squashfs_decompressor_destroy(struct squashfs_sb_info *msblk)
		int squashfs_decompress(struct squashfs_sb_info msblk, struct buffer_head *bh,
		int b, int offset, int length, struct squashfs_page_actor *output)
		{
		struct squashfs_stream __percpu *percpu =
		(struct squashfs_stream __percpu *) msblk->stream;
		struct squashfs_stream *stream = get_cpu_ptr(percpu);
		int res = msblk->decompressor->decompress(msblk, stream->stream, bh, b,
		struct squashfs_stream *stream;
		int res;

		local_lock(&msblk->stream->lock);
		stream = this_cpu_ptr(msblk->stream);

		res = msblk->decompressor->decompress(msblk, stream->stream, bh, b,
		offset, length, output);
		put_cpu_ptr(stream);

		local_unlock(&msblk->stream->lock);

		if (res < 0)
		ERROR("%s decompression failed, data probably corrupt\n",

Admin message