Commit 69b90729 authored by Alexey Kardashevskiy's avatar Alexey Kardashevskiy Committed by Paul E. McKenney
Browse files

list: Add lockless list traversal primitives 



Although list_for_each_entry_rcu() can in theory be used anywhere
preemption is disabled, it can result in calls to lockdep, which cannot
be used in certain constrained execution environments, such as exception
handlers that do not map the entire kernel into their address spaces.
This commit therefore adds list_entry_lockless() and
list_for_each_entry_lockless(), which never invoke lockdep and can
therefore safely be used from these constrained environments, but only
as long as those environments are non-preemptible (or items are never
deleted from the list).

Use synchronize_sched(), call_rcu_sched(), or synchronize_sched_expedited()
in updates for the needed grace periods.  Of course, if items are never
deleted from the list, there is no need to wait for grace periods.

Signed-off-by: default avatarAlexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
parent 45fed3e7

include/linux/rculist.h

+36 −0
Original line number Diff line number Diff line
@@ -304,6 +304,42 @@ static inline void list_splice_init_rcu(struct list_head *list, 
		&pos->member != (head); \

		pos = list_entry_rcu(pos->member.next, typeof(*pos), member))



/**

 * list_entry_lockless - get the struct for this entry

 * @ptr:        the &struct list_head pointer.

 * @type:       the type of the struct this is embedded in.

 * @member:     the name of the list_head within the struct.

 *

 * This primitive may safely run concurrently with the _rcu list-mutation

 * primitives such as list_add_rcu(), but requires some implicit RCU

 * read-side guarding.  One example is running within a special

 * exception-time environment where preemption is disabled and where

 * lockdep cannot be invoked (in which case updaters must use RCU-sched,

 * as in synchronize_sched(), call_rcu_sched(), and friends).  Another

 * example is when items are added to the list, but never deleted.

 */

#define list_entry_lockless(ptr, type, member) \

	container_of((typeof(ptr))lockless_dereference(ptr), type, member)



/**

 * list_for_each_entry_lockless - iterate over rcu list of given type

 * @pos:	the type * to use as a loop cursor.

 * @head:	the head for your list.

 * @member:	the name of the list_struct within the struct.

 *

 * This primitive may safely run concurrently with the _rcu list-mutation

 * primitives such as list_add_rcu(), but requires some implicit RCU

 * read-side guarding.  One example is running within a special

 * exception-time environment where preemption is disabled and where

 * lockdep cannot be invoked (in which case updaters must use RCU-sched,

 * as in synchronize_sched(), call_rcu_sched(), and friends).  Another

 * example is when items are added to the list, but never deleted.

 */

#define list_for_each_entry_lockless(pos, head, member) \

	for (pos = list_entry_lockless((head)->next, typeof(*pos), member); \

	     &pos->member != (head); \

	     pos = list_entry_lockless(pos->member.next, typeof(*pos), member))



/**

 * list_for_each_entry_continue_rcu - continue iteration over list of given type

 * @pos:	the type * to use as a loop cursor.

CRA Git | Maintained and supported by SUSTech CRA and CCSE