rwsem: wake queued readers when writer blocks on active read lock (424acaae) · Commits · 戴 / test

This change addresses the following situation: - Thread A acquires the rwsem for read - Thread B tries to acquire the rwsem for write, notices there is already an active owner for the rwsem. - Thread C tries to acquire the rwsem for read, notices that thread B already tried to acquire it. - Thread C grabs the spinlock and queues itself on the wait queue. - Thread B grabs the spinlock and queues itself behind C. At this point A is the only remaining active owner on the rwsem. In this situation thread B could notice that it was the last active writer on the rwsem, and decide to wake C to let it proceed in parallel with A since they both only want the rwsem for read. Signed-off-by:

Michel Lespinasse <walken@google.com> Acked-by:

David Howells <dhowells@redhat.com> Cc: Mike Waychison <mikew@google.com> Cc: Suleiman Souhlal <suleiman@google.com> Cc: Ying Han <yinghan@google.com> Cc: Ingo Molnar <mingo@elte.hu> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Signed-off-by:

Andrew Morton <akpm@linux-foundation.org> Signed-off-by:

Linus Torvalds <torvalds@linux-foundation.org>

lib/rwsem.c

+15 −4

Original line number	Diff line number	Diff line
		@@ -67,6 +67,9 @@ __rwsem_do_wake(struct rw_semaphore *sem, int wake_type)
		goto readers_only;

		if (wake_type == RWSEM_WAKE_READ_OWNED)
		/* Another active reader was observed, so wakeup is not
		* likely to succeed. Save the atomic op.
		*/
		goto out;

		/* There's a writer at the front of the queue - try to grant it the
		@@ -111,8 +114,8 @@ __rwsem_do_wake(struct rw_semaphore *sem, int wake_type)
		* count adjustment pretty soon.
		*/
		if (wake_type == RWSEM_WAKE_ANY &&
		(rwsem_atomic_update(0, sem) & RWSEM_ACTIVE_MASK))
		/* Someone grabbed the sem already */
		rwsem_atomic_update(0, sem) < RWSEM_WAITING_BIAS)
		/* Someone grabbed the sem for write already */
		goto out;

		/* Grant an infinite number of read locks to the readers at the front
		@@ -187,9 +190,17 @@ rwsem_down_failed_common(struct rw_semaphore *sem,
		/* we're now waiting on the lock, but no longer actively locking */
		count = rwsem_atomic_update(adjustment, sem);

		/* if there are no active locks, wake the front queued process(es) up */
		if (!(count & RWSEM_ACTIVE_MASK))
		/* If there are no active locks, wake the front queued process(es) up.
		*
		* Alternatively, if we're called from a failed down_write(), there
		* were already threads queued before us and there are no active
		* writers, the lock must be read owned; so we try to wake any read
		* locks that were queued ahead of us. */
		if (count == RWSEM_WAITING_BIAS)
		sem = __rwsem_do_wake(sem, RWSEM_WAKE_NO_ACTIVE);
		else if (count > RWSEM_WAITING_BIAS &&
		adjustment == -RWSEM_ACTIVE_WRITE_BIAS)
		sem = __rwsem_do_wake(sem, RWSEM_WAKE_READ_OWNED);

		spin_unlock_irq(&sem->wait_lock);

Admin message