workqueue: UNBOUND -> REBIND morphing in rebind_workers() should be atomic (96e65306) · Commits · 戴 / test

The compiler may compile the following code into TWO write/modify instructions. worker->flags &= ~WORKER_UNBOUND; worker->flags |= WORKER_REBIND; so the other CPU may temporarily see worker->flags which doesn't have either WORKER_UNBOUND or WORKER_REBIND set and perform local wakeup prematurely. Fix it by using single explicit assignment via ACCESS_ONCE(). Because idle workers have another WORKER_NOT_RUNNING flag, this bug doesn't exist for them; however, update it to use the same pattern for consistency. tj: Applied the change to idle workers too and updated comments and patch description a bit. Signed-off-by:

Lai Jiangshan <laijs@cn.fujitsu.com> Signed-off-by:

Tejun Heo <tj@kernel.org> Cc: stable@vger.kernel.org

kernel/workqueue.c

+11 −6

Original line number	Diff line number	Diff line
		@@ -1396,12 +1396,15 @@ retry:
		/* set REBIND and kick idle ones, we'll wait for these later */
		for_each_worker_pool(pool, gcwq) {
		list_for_each_entry(worker, &pool->idle_list, entry) {
		unsigned long worker_flags = worker->flags;

		if (worker->flags & WORKER_REBIND)
		continue;

		/* morph UNBOUND to REBIND */
		worker->flags &= ~WORKER_UNBOUND;
		worker->flags \|= WORKER_REBIND;
		/* morph UNBOUND to REBIND atomically */
		worker_flags &= ~WORKER_UNBOUND;
		worker_flags \|= WORKER_REBIND;
		ACCESS_ONCE(worker->flags) = worker_flags;

		idle_rebind.cnt++;
		worker->idle_rebind = &idle_rebind;
		@@ -1434,10 +1437,12 @@ retry:
		/* rebind busy workers */
		for_each_busy_worker(worker, i, pos, gcwq) {
		struct work_struct *rebind_work = &worker->rebind_work;
		unsigned long worker_flags = worker->flags;

		/* morph UNBOUND to REBIND */
		worker->flags &= ~WORKER_UNBOUND;
		worker->flags \|= WORKER_REBIND;
		/* morph UNBOUND to REBIND atomically */
		worker_flags &= ~WORKER_UNBOUND;
		worker_flags \|= WORKER_REBIND;
		ACCESS_ONCE(worker->flags) = worker_flags;

		if (test_and_set_bit(WORK_STRUCT_PENDING_BIT,
		work_data_bits(rebind_work)))

Admin message