Documentation/memory-barriers.txt: Add needed ACCESS_ONCE() calls to memory-barriers.txt

 (*) On any given CPU, dependent memory accesses will be issued in order, with

     respect to itself.  This means that for:

	Q = P; D = *Q;

	ACCESS_ONCE(Q) = P; smp_read_barrier_depends(); D = ACCESS_ONCE(*Q);

     the CPU will issue the following memory operations:

	Q = LOAD P, D = LOAD *Q

     and always in that order.

     and always in that order.  On most systems, smp_read_barrier_depends()

     does nothing, but it is required for DEC Alpha.  The ACCESS_ONCE()

     is required to prevent compiler mischief.  Please note that you

     should normally use something like rcu_dereference() instead of

     open-coding smp_read_barrier_depends().

 (*) Overlapping loads and stores within a particular CPU will appear to be

     ordered within that CPU.  This means that for:

	a = *X; *X = b;

	a = ACCESS_ONCE(*X); ACCESS_ONCE(*X) = b;

     the CPU will only issue the following sequence of memory operations:

     And for:

	*X = c; d = *X;

	ACCESS_ONCE(*X) = c; d = ACCESS_ONCE(*X);

     the CPU will only issue:

And there are a number of things that _must_ or _must_not_ be assumed:

 (*) It _must_not_ be assumed that the compiler will do what you want with

     memory references that are not protected by ACCESS_ONCE().  Without

     ACCESS_ONCE(), the compiler is within its rights to do all sorts

     of "creative" transformations:

     (-) Repeat the load, possibly getting a different value on the second

         and subsequent loads.  This is especially prone to happen when

	 register pressure is high.

     (-) Merge adjacent loads and stores to the same location.  The most

         familiar example is the transformation from:

		while (a)

			do_something();

         to something like:

		if (a)

			for (;;)

				do_something();

         Using ACCESS_ONCE() as follows prevents this sort of optimization:

		while (ACCESS_ONCE(a))

			do_something();

     (-) "Store tearing", where a single store in the source code is split

         into smaller stores in the object code.  Note that gcc really

	 will do this on some architectures when storing certain constants.

	 It can be cheaper to do a series of immediate stores than to

	 form the constant in a register and then to store that register.

     (-) "Load tearing", which splits loads in a manner analogous to

     	 store tearing.

 (*) It _must_not_ be assumed that independent loads and stores will be issued

     in the order given.  This means that for:

	{ A == 1, B == 2, C = 3, P == &A, Q == &C }

	B = 4;

	<write barrier>

	P = &B

			Q = P;

	ACCESS_ONCE(P) = &B

			      Q = ACCESS_ONCE(P);

			      D = *Q;

There's a clear data dependency here, and it would seem that by the end of the

	{ A == 1, B == 2, C = 3, P == &A, Q == &C }

	B = 4;

	<write barrier>

	P = &B

			Q = P;

	ACCESS_ONCE(P) = &B

			      Q = ACCESS_ONCE(P);

			      <data dependency barrier>

			      D = *Q;

	{ M[0] == 1, M[1] == 2, M[3] = 3, P == 0, Q == 3 }

	M[1] = 4;

	<write barrier>

	P = 1

			Q = P;

	ACCESS_ONCE(P) = 1

			      Q = ACCESS_ONCE(P);

			      <data dependency barrier>

			      D = M[Q];

The data dependency barrier is very important to the RCU system, for example.

See rcu_dereference() in include/linux/rcupdate.h.  This permits the current

target of an RCU'd pointer to be replaced with a new modified target, without

the replacement target appearing to be incompletely initialised.

The data dependency barrier is very important to the RCU system,

for example.  See rcu_assign_pointer() and rcu_dereference() in

include/linux/rcupdate.h.  This permits the current target of an RCU'd

pointer to be replaced with a new modified target, without the replacement

target appearing to be incompletely initialised.

See also the subsection on "Cache Coherency" for a more thorough example.

dependency barrier to make it work correctly.  Consider the following bit of

code:

	q = &a;

	q = ACCESS_ONCE(a);

	if (p) {

		<data dependency barrier>

		q = &b;

		q = ACCESS_ONCE(b);

	}

	x = *q;

This will not have the desired effect because there is no actual data

dependency, but rather a control dependency that the CPU may short-circuit by

attempting to predict the outcome in advance.  In such a case what's actually

required is:

dependency, but rather a control dependency that the CPU may short-circuit

by attempting to predict the outcome in advance, so that other CPUs see

the load from b as having happened before the load from a.  In such a

case what's actually required is:

	q = &a;

	q = ACCESS_ONCE(a);

	if (p) {

		<read barrier>

		q = &b;

		q = ACCESS_ONCE(b);

	}

	x = *q;

	CPU 1		      CPU 2

	===============	      ===============

	a = 1;

	ACCESS_ONCE(a) = 1;

	<write barrier>

	b = 2;		x = b;

	ACCESS_ONCE(b) = 2;   x = ACCESS_ONCE(b);

			      <read barrier>

			y = a;

			      y = ACCESS_ONCE(a);

Or:

	===============	      ===============================

	a = 1;

	<write barrier>

	b = &a;		x = b;

	ACCESS_ONCE(b) = &a;  x = ACCESS_ONCE(b);

			      <data dependency barrier>

			      y = *x;

versa:

	CPU 1                               CPU 2

	===============                 ===============

	a = 1;           }----   --->{  v = c

	b = 2;           }    \ /    {  w = d

	===================                 ===================

	ACCESS_ONCE(a) = 1;  }----   --->{  v = ACCESS_ONCE(c);

	ACCESS_ONCE(b) = 2;  }    \ /    {  w = ACCESS_ONCE(d);

	<write barrier>            \        <read barrier>

	c = 3;           }    / \    {  x = a;

	d = 4;           }----   --->{  y = b;

	ACCESS_ONCE(c) = 3;  }    / \    {  x = ACCESS_ONCE(a);

	ACCESS_ONCE(d) = 4;  }----   --->{  y = ACCESS_ONCE(b);

EXAMPLES OF MEMORY BARRIER SEQUENCES

	CPU 1				CPU 2

	===============================	===============================

	*A = a;				*E = e;

	ACCESS_ONCE(*A) = a;		ACCESS_ONCE(*E) = e;

	LOCK M				LOCK Q

	*B = b;				*F = f;

	*C = c;				*G = g;

	ACCESS_ONCE(*B) = b;		ACCESS_ONCE(*F) = f;

	ACCESS_ONCE(*C) = c;		ACCESS_ONCE(*G) = g;

	UNLOCK M			UNLOCK Q

	*D = d;				*H = h;

	ACCESS_ONCE(*D) = d;		ACCESS_ONCE(*H) = h;

Then there is no guarantee as to what order CPU 3 will see the accesses to *A

through *H occur in, other than the constraints imposed by the separate locks

	CPU 1				CPU 2

	===============================	===============================

	*A = a;

	ACCESS_ONCE(*A) = a;

	LOCK M		     [1]

	*B = b;

	*C = c;

	ACCESS_ONCE(*B) = b;

	ACCESS_ONCE(*C) = c;

	UNLOCK M	     [1]

	*D = d;				*E = e;

	ACCESS_ONCE(*D) = d;		ACCESS_ONCE(*E) = e;

					LOCK M		     [2]

					*F = f;

					*G = g;

					ACCESS_ONCE(*F) = f;

					ACCESS_ONCE(*G) = g;

					UNLOCK M	     [2]

					*H = h;

					ACCESS_ONCE(*H) = h;

CPU 3 might see:

operations in exactly the order specified, so that if the CPU is, for example,

given the following piece of code to execute:

	a = *A;

	*B = b;

	c = *C;

	d = *D;

	*E = e;

	a = ACCESS_ONCE(*A);

	ACCESS_ONCE(*B) = b;

	c = ACCESS_ONCE(*C);

	d = ACCESS_ONCE(*D);

	ACCESS_ONCE(*E) = e;

they would then expect that the CPU will complete the memory operation for each

instruction before moving on to the next one, leading to a definite sequence of

_own_ accesses appear to be correctly ordered, without the need for a memory

barrier.  For instance with the following code:

	U = *A;

	*A = V;

	*A = W;

	X = *A;

	*A = Y;

	Z = *A;

	U = ACCESS_ONCE(*A);

	ACCESS_ONCE(*A) = V;

	ACCESS_ONCE(*A) = W;

	X = ACCESS_ONCE(*A);

	ACCESS_ONCE(*A) = Y;

	Z = ACCESS_ONCE(*A);

and assuming no intervention by an external influence, it can be assumed that

the final result will appear to be:

in that order, but, without intervention, the sequence may have almost any

combination of elements combined or discarded, provided the program's view of

the world remains consistent.

the world remains consistent.  Note that ACCESS_ONCE() is -not- optional

in the above example, as there are architectures where a given CPU might

interchange successive loads to the same location.  On such architectures,

ACCESS_ONCE() does whatever is necessary to prevent this, for example, on

Itanium the volatile casts used by ACCESS_ONCE() cause GCC to emit the

special ld.acq and st.rel instructions that prevent such reordering.

The compiler may also combine, discard or defer elements of the sequence before

the CPU even sees them.

	*A = W;

since, without a write barrier, it can be assumed that the effect of the

storage of V to *A is lost.  Similarly:

since, without either a write barrier or an ACCESS_ONCE(), it can be

assumed that the effect of the storage of V to *A is lost.  Similarly:

	*A = Y;

	Z = *A;

may, without a memory barrier, be reduced to:

may, without a memory barrier or an ACCESS_ONCE(), be reduced to:

	*A = Y;

	Z = Y;