Commit fa004ab7 authored Dec 28, 2018 by Wei Yang Committed by Linus Torvalds Dec 28, 2018

mm, hotplug: move init_currently_empty_zone() under zone_span_lock protection

During online_pages phase, pgdat->nr_zones will be updated in case this
zone is empty.

Currently the online_pages phase is protected by the global locks
(device_device_hotplug_lock and mem_hotplug_lock), which ensures there is
no contention during the update of nr_zones.

These global locks introduces scalability issues (especially the second
one), which slow down code relying on get_online_mems().  This is also a
preparation for not having to rely on get_online_mems() but instead some
more fine grained locks.

The patch moves init_currently_empty_zone under both zone_span_writelock
and pgdat_resize_lock because both the pgdat state is changed (nr_zones)
and the zone's start_pfn.  Also this patch changes the documentation of
node_size_lock to include the protection of nr_zones.

Link: http://lkml.kernel.org/r/20181203205016.14123-1-richard.weiyang@gmail.com


Signed-off-by: Wei Yang <richard.weiyang@gmail.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Reviewed-by: Oscar Salvador <osalvador@suse.de>
Cc: David Hildenbrand <david@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

parent 4e0d2e7e

include/linux/mmzone.h

+2 −2

Original line number	Diff line number	Diff line
		@@ -636,8 +636,8 @@ typedef struct pglist_data {
		#endif
		#if defined(CONFIG_MEMORY_HOTPLUG) \|\| defined(CONFIG_DEFERRED_STRUCT_PAGE_INIT)
		/*
		* Must be held any time you expect node_start_pfn, node_present_pages
		* or node_spanned_pages stay constant.
		* Must be held any time you expect node_start_pfn,
		* node_present_pages, node_spanned_pages or nr_zones to stay constant.
		*
		* pgdat_resize_lock() and pgdat_resize_unlock() are provided to
		* manipulate node_size_lock without checking for CONFIG_MEMORY_HOTPLUG

mm/memory_hotplug.c

+2 −3

Original line number	Diff line number	Diff line
		@@ -743,14 +743,13 @@ void __ref move_pfn_range_to_zone(struct zone *zone, unsigned long start_pfn,
		int nid = pgdat->node_id;
		unsigned long flags;

		if (zone_is_empty(zone))
		init_currently_empty_zone(zone, start_pfn, nr_pages);

		clear_zone_contiguous(zone);

		/* TODO Huh pgdat is irqsave while zone is not. It used to be like that before */
		pgdat_resize_lock(pgdat, &flags);
		zone_span_writelock(zone);
		if (zone_is_empty(zone))
		init_currently_empty_zone(zone, start_pfn, nr_pages);
		resize_zone_range(zone, start_pfn, nr_pages);
		zone_span_writeunlock(zone);
		resize_pgdat_range(pgdat, start_pfn, nr_pages);

Admin message