Commit c350c008 authored by Paul E. McKenney's avatar Paul E. McKenney
Browse files

srcu: Prevent sdp->srcu_gp_seq_needed counter wrap 



If a given CPU never happens to ever start an SRCU grace period, the
grace-period sequence counter might wrap.  If this CPU were to decide to
finally start a grace period, the state of its sdp->srcu_gp_seq_needed
might make it appear that it has already requested this grace period,
which would prevent starting the grace period.  If no other CPU ever started
a grace period again, this would look like a grace-period hang.  Even
if some other CPU took pity and started the needed grace period, the
leaf rcu_node structure's ->srcu_data_have_cbs field won't have record
of the fact that this CPU has a callback pending, which would look like
a very localized grace-period hang.

This might seem very unlikely, but SRCU grace periods can take less than
a microsecond on small systems, which means that overflow can happen
in much less than an hour on a 32-bit embedded system.  And embedded
systems are especially likely to have long-term idle CPUs.  Therefore,
it makes sense to prevent this scenario from happening.

This commit therefore scans each srcu_data structure occasionally,
with frequency controlled by the srcutree.counter_wrap_check kernel
boot parameter.  This parameter can be set to something like 255
in order to exercise the counter-wrap-prevention code.

Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
parent 71c40fd0

Documentation/admin-guide/kernel-parameters.txt

+9 −0
Original line number Diff line number Diff line
@@ -3810,6 +3810,15 @@ 
	spia_pedr=

	spia_peddr=



	srcutree.counter_wrap_check [KNL]

			Specifies how frequently to check for

			grace-period sequence counter wrap for the

			srcu_data structure's ->srcu_gp_seq_needed field.

			The greater the number of bits set in this kernel

			parameter, the less frequently counter wrap will

			be checked for.  Note that the bottom two bits

			are ignored.



	srcutree.exp_holdoff [KNL]

			Specifies how many nanoseconds must elapse

			since the end of the last SRCU grace period for

kernel/rcu/srcutree.c

+18 −0
Original line number Diff line number Diff line
@@ -45,6 +45,10 @@ 
static ulong exp_holdoff = DEFAULT_SRCU_EXP_HOLDOFF;

module_param(exp_holdoff, ulong, 0444);



/* Overflow-check frequency.  N bits roughly says every 2**N grace periods. */

static ulong counter_wrap_check = (ULONG_MAX >> 2);

module_param(counter_wrap_check, ulong, 0444);



static void srcu_invoke_callbacks(struct work_struct *work);

static void srcu_reschedule(struct srcu_struct *sp, unsigned long delay);
@@ -496,10 +500,13 @@ static void srcu_gp_end(struct srcu_struct *sp) 
{

	unsigned long cbdelay;

	bool cbs;

	int cpu;

	unsigned long flags;

	unsigned long gpseq;

	int idx;

	int idxnext;

	unsigned long mask;

	struct srcu_data *sdp;

	struct srcu_node *snp;



	/* Prevent more than one additional grace period. */
@@ -538,6 +545,17 @@ static void srcu_gp_end(struct srcu_struct *sp) 
			smp_mb(); /* GP end before CB invocation. */

			srcu_schedule_cbs_snp(sp, snp, mask, cbdelay);

		}



		/* Occasionally prevent srcu_data counter wrap. */

		if (!(gpseq & counter_wrap_check))

			for (cpu = snp->grplo; cpu <= snp->grphi; cpu++) {

				sdp = per_cpu_ptr(sp->sda, cpu);

				spin_lock_irqsave(&sdp->lock, flags);

				if (ULONG_CMP_GE(gpseq,

						 sdp->srcu_gp_seq_needed + 100))

					sdp->srcu_gp_seq_needed = gpseq;

				spin_unlock_irqrestore(&sdp->lock, flags);

			}

	}



	/* Callback initiation done, allow grace periods after next. */

CRA Git | Maintained and supported by SUSTech CRA and CCSE