sched: Fix ttwu() for __ARCH_WANT_INTERRUPTS_ON_CTXSW (d6aa8f85) · Commits · 戴 / test

Marc reported that (sched: Remove rq->lock from the first half of ttwu()) broke his ARM-SMP machine. Now ARM is one of the few __ARCH_WANT_INTERRUPTS_ON_CTXSW users, so that exception in the ttwu() code was suspect. Yong found that the interrupt could hit after context_switch() changes current but before it clears p->on_cpu, if that interrupt were to attempt a wake-up of p we would indeed find ourselves spinning in IRQ context. Fix this by reverting to the old behaviour for this situation and perform a full remote wake-up. Cc: Frank Rowand <frank.rowand@am.sony.com> Cc: Yong Zhang <yong.zhang0@gmail.com> Cc: Oleg Nesterov <oleg@redhat.com> Reported-by:

Marc Zyngier <Marc.Zyngier@arm.com> Tested-by:

Marc Zyngier <marc.zyngier@arm.com> Signed-off-by:

Peter Zijlstra <a.p.zijlstra@chello.nl> Signed-off-by:

Ingo Molnar <mingo@elte.hu>

kernel/sched.c

+28 −9

Original line number	Diff line number	Diff line
		@@ -2573,7 +2573,26 @@ static void ttwu_queue_remote(struct task_struct *p, int cpu)
		if (!next)
		smp_send_reschedule(cpu);
		}
		#endif

		#ifdef __ARCH_WANT_INTERRUPTS_ON_CTXSW
		static int ttwu_activate_remote(struct task_struct *p, int wake_flags)
		{
		struct rq *rq;
		int ret = 0;

		rq = __task_rq_lock(p);
		if (p->on_cpu) {
		ttwu_activate(rq, p, ENQUEUE_WAKEUP);
		ttwu_do_wakeup(rq, p, wake_flags);
		ret = 1;
		}
		__task_rq_unlock(rq);

		return ret;

		}
		#endif /* __ARCH_WANT_INTERRUPTS_ON_CTXSW */
		#endif /* CONFIG_SMP */

		static void ttwu_queue(struct task_struct *p, int cpu)
		{
		@@ -2631,17 +2650,17 @@ try_to_wake_up(struct task_struct *p, unsigned int state, int wake_flags)
		while (p->on_cpu) {
		#ifdef __ARCH_WANT_INTERRUPTS_ON_CTXSW
		/*
		* If called from interrupt context we could have landed in the
		* middle of schedule(), in this case we should take care not
		* to spin on ->on_cpu if p is current, since that would
		* deadlock.
		* In case the architecture enables interrupts in
		* context_switch(), we cannot busy wait, since that
		* would lead to deadlocks when an interrupt hits and
		* tries to wake up @prev. So bail and do a complete
		* remote wakeup.
		*/
		if (p == current) {
		ttwu_queue(p, cpu);
		if (ttwu_activate_remote(p, wake_flags))
		goto stat;
		}
		#endif
		#else
		cpu_relax();
		#endif
		}
		/*
		* Pairs with the smp_wmb() in finish_lock_switch().

Admin message