stop_machine: make stop_machine safe and efficient to call early (189c3fd6) · Commits · 戴 / test

Make stop_machine() safe to call early in boot, before stop_machine() has been set up, by simply calling the callback function directly if there's only one CPU online. [ Fixes from AKPM: - add comment - local_irq_flags, not save_flags - also call hard_irq_disable() for systems which need it Tejun suggested using an explicit flag rather than just looking at the online cpu count. ] Signed-off-by:

Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> Acked-by:

Tejun Heo <tj@kernel.org> Acked-by:

Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Rusty Russell <rusty@rustcorp.com.au> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: H. Peter Anvin <hpa@linux.intel.com> Cc: Ingo Molnar <mingo@elte.hu> Cc: Steven Rostedt <rostedt@goodmis.org>

kernel/stop_machine.c

+22 −0

Original line number	Diff line number	Diff line
		@@ -41,6 +41,7 @@ struct cpu_stopper {
		};

		static DEFINE_PER_CPU(struct cpu_stopper, cpu_stopper);
		static bool stop_machine_initialized = false;

		static void cpu_stop_init_done(struct cpu_stop_done *done, unsigned int nr_todo)
		{
		@@ -386,6 +387,8 @@ static int __init cpu_stop_init(void)
		cpu_stop_cpu_callback(&cpu_stop_cpu_notifier, CPU_ONLINE, bcpu);
		register_cpu_notifier(&cpu_stop_cpu_notifier);

		stop_machine_initialized = true;

		return 0;
		}
		early_initcall(cpu_stop_init);
		@@ -485,6 +488,25 @@ int __stop_machine(int (fn)(void ), void data, const struct cpumask cpus)
		.num_threads = num_online_cpus(),
		.active_cpus = cpus };

		if (!stop_machine_initialized) {
		/*
		* Handle the case where stop_machine() is called
		* early in boot before stop_machine() has been
		* initialized.
		*/
		unsigned long flags;
		int ret;

		WARN_ON_ONCE(smdata.num_threads != 1);

		local_irq_save(flags);
		hard_irq_disable();
		ret = (*fn)(data);
		local_irq_restore(flags);

		return ret;
		}

		/* Set the initial state and stop all online cpus. */
		set_state(&smdata, STOPMACHINE_PREPARE);
		return stop_cpus(cpu_online_mask, stop_machine_cpu_stop, &smdata);

Admin message