doc: Convert to rcubarrier.txt to ReST

   :maxdepth: 3

   arrayRCU

   rcubarrier

   rcu_dereference

   whatisRCU

   rcu

.. _rcu_barrier:

RCU and Unloadable Modules

==========================

[Originally published in LWN Jan. 14, 2007: http://lwn.net/Articles/217484/]

presence? There is a synchronize_rcu() primitive that blocks until all

pre-existing readers have completed. An updater wishing to delete an

element p from a linked list might do the following, while holding an

appropriate lock, of course:

appropriate lock, of course::

	list_del_rcu(p);

	synchronize_rcu();

rcu_head struct placed within the RCU-protected data structure and

another pointer to a function that may be invoked later to free that

structure. Code to delete an element p from the linked list from IRQ

context might then be as follows:

context might then be as follows::

	list_del_rcu(p);

	call_rcu(&p->rcu, p_callback);

Since call_rcu() never blocks, this code can safely be used from within

IRQ context. The function p_callback() might be defined as follows:

IRQ context. The function p_callback() might be defined as follows::

	static void p_callback(struct rcu_head *rp)

	{

Unloading Modules That Use call_rcu()

-------------------------------------

But what if p_callback is defined in an unloadable module?

rcu_barrier()

-------------

We instead need the rcu_barrier() primitive.  Rather than waiting for

a grace period to elapse, rcu_barrier() waits for all outstanding RCU

callbacks to complete.  Please note that rcu_barrier() does -not- imply

callbacks to complete.  Please note that rcu_barrier() does **not** imply

synchronize_rcu(), in particular, if there are no RCU callbacks queued

anywhere, rcu_barrier() is within its rights to return immediately,

without waiting for a grace period to elapse.

module uses multiple flavors of call_rcu(), then it must also use multiple

flavors of rcu_barrier() when unloading that module.  For example, if

it uses call_rcu(), call_srcu() on srcu_struct_1, and call_srcu() on

srcu_struct_2(), then the following three lines of code will be required

when unloading:

srcu_struct_2, then the following three lines of code will be required

when unloading::

 1 rcu_barrier();

 2 srcu_barrier(&srcu_struct_1);

 3 srcu_barrier(&srcu_struct_2);

The rcutorture module makes use of rcu_barrier() in its exit function

as follows:

as follows::

 1  static void

 2  rcu_torture_cleanup(void)

Then lines 55-62 print status and do operation-specific cleanup, and

then return, permitting the module-unload operation to be completed.

Quick Quiz #1: Is there any other situation where rcu_barrier() might

.. _rcubarrier_quiz_1:

Quick Quiz #1:

	Is there any other situation where rcu_barrier() might

	be required?

:ref:`Answer to Quick Quiz #1 <answer_rcubarrier_quiz_1>`

Your module might have additional complications. For example, if your

module invokes call_rcu() from timers, you will need to first cancel all

the timers, and only then invoke rcu_barrier() to wait for any remaining

rcu_barrier() before unloading.  Similarly, if your module uses

call_srcu(), you will need to invoke srcu_barrier() before unloading,

and on the same srcu_struct structure.  If your module uses call_rcu()

-and- call_srcu(), then you will need to invoke rcu_barrier() -and-

**and** call_srcu(), then you will need to invoke rcu_barrier() **and**

srcu_barrier().

Implementing rcu_barrier()

--------------------------

Dipankar Sarma's implementation of rcu_barrier() makes use of the fact

that RCU callbacks are never reordered once queued on one of the per-CPU

callback queues, and then waits until they have all started executing, at

which point, all earlier RCU callbacks are guaranteed to have completed.

The original code for rcu_barrier() was as follows:

The original code for rcu_barrier() was as follows::

 1  void rcu_barrier(void)

 2  {

still gives the general idea.

The rcu_barrier_func() runs on each CPU, where it invokes call_rcu()

to post an RCU callback, as follows:

to post an RCU callback, as follows::

 1  static void rcu_barrier_func(void *notused)

 2  {

The rcu_barrier_callback() function simply atomically decrements the

rcu_barrier_cpu_count variable and finalizes the completion when it

reaches zero, as follows:

reaches zero, as follows::

 1 static void rcu_barrier_callback(struct rcu_head *notused)

 2 {

 4     complete(&rcu_barrier_completion);

 5 }

Quick Quiz #2: What happens if CPU 0's rcu_barrier_func() executes

.. _rcubarrier_quiz_2:

Quick Quiz #2:

	What happens if CPU 0's rcu_barrier_func() executes

	immediately (thus incrementing rcu_barrier_cpu_count to the

	value one), but the other CPU's rcu_barrier_func() invocations

	are delayed for a full grace period? Couldn't this result in

	rcu_barrier() returning prematurely?

:ref:`Answer to Quick Quiz #2 <answer_rcubarrier_quiz_2>`

The current rcu_barrier() implementation is more complex, due to the need

to avoid disturbing idle CPUs (especially on battery-powered systems)

and the need to minimally disturb non-idle CPUs in real-time systems.

rcu_barrier() Summary

---------------------

The rcu_barrier() primitive has seen relatively little use, since most

code using RCU is in the core kernel rather than in modules. However, if

Answers to Quick Quizzes

------------------------

Quick Quiz #1: Is there any other situation where rcu_barrier() might

.. _answer_rcubarrier_quiz_1:

Quick Quiz #1:

	Is there any other situation where rcu_barrier() might

	be required?

Answer: Interestingly enough, rcu_barrier() was not originally

	implementing rcutorture, and found that rcu_barrier() solves

	this problem as well.

Quick Quiz #2: What happens if CPU 0's rcu_barrier_func() executes

:ref:`Back to Quick Quiz #1 <rcubarrier_quiz_1>`

.. _answer_rcubarrier_quiz_2:

Quick Quiz #2:

	What happens if CPU 0's rcu_barrier_func() executes

	immediately (thus incrementing rcu_barrier_cpu_count to the

	value one), but the other CPU's rcu_barrier_func() invocations

	are delayed for a full grace period? Couldn't this result in

	is to add an rcu_read_lock() before line 8 of rcu_barrier()

	and an rcu_read_unlock() after line 8 of this same function. If

	you can think of a better change, please let me know!

:ref:`Back to Quick Quiz #2 <rcubarrier_quiz_2>`