mm: page_alloc: exempt GFP_THISNODE allocations from zone fairness (27329369) · Commits · 戴 / test

Jan Stancek reports manual page migration encountering allocation failures after some pages when there is still plenty of memory free, and bisected the problem down to commit ("mm: page_alloc: fair zone allocator policy"). The problem is that GFP_THISNODE obeys the zone fairness allocation batches on one hand, but doesn't reset them and wake kswapd on the other hand. After a few of those allocations, the batches are exhausted and the allocations fail. Fixing this means either having GFP_THISNODE wake up kswapd, or GFP_THISNODE not participating in zone fairness at all. The latter seems safer as an acute bugfix, we can clean up later. Reported-by:

Jan Stancek <jstancek@redhat.com> Signed-off-by:

Johannes Weiner <hannes@cmpxchg.org> Acked-by:

Rik van Riel <riel@redhat.com> Acked-by:

Mel Gorman <mgorman@suse.de> Cc: <stable@kernel.org> [3.12+] Signed-off-by:

Andrew Morton <akpm@linux-foundation.org> Signed-off-by:

Linus Torvalds <torvalds@linux-foundation.org>

mm/page_alloc.c

+22 −4

Original line number	Diff line number	Diff line
		@@ -1238,6 +1238,15 @@ void drain_zone_pages(struct zone zone, struct per_cpu_pages pcp)
		}
		local_irq_restore(flags);
		}
		static bool gfp_thisnode_allocation(gfp_t gfp_mask)
		{
		return (gfp_mask & GFP_THISNODE) == GFP_THISNODE;
		}
		#else
		static bool gfp_thisnode_allocation(gfp_t gfp_mask)
		{
		return false;
		}
		#endif

		/*
		@@ -1574,7 +1583,13 @@ again:
		get_pageblock_migratetype(page));
		}

		/*
		* NOTE: GFP_THISNODE allocations do not partake in the kswapd
		* aging protocol, so they can't be fair.
		*/
		if (!gfp_thisnode_allocation(gfp_flags))
		__mod_zone_page_state(zone, NR_ALLOC_BATCH, -(1 << order));

		__count_zone_vm_events(PGALLOC, zone, 1 << order);
		zone_statistics(preferred_zone, zone, gfp_flags);
		local_irq_restore(flags);
		@@ -1946,8 +1961,12 @@ zonelist_scan:
		* ultimately fall back to remote zones that do not
		* partake in the fairness round-robin cycle of this
		* zonelist.
		*
		* NOTE: GFP_THISNODE allocations do not partake in
		* the kswapd aging protocol, so they can't be fair.
		*/
		if (alloc_flags & ALLOC_WMARK_LOW) {
		if ((alloc_flags & ALLOC_WMARK_LOW) &&
		!gfp_thisnode_allocation(gfp_mask)) {
		if (zone_page_state(zone, NR_ALLOC_BATCH) <= 0)
		continue;
		if (!zone_local(preferred_zone, zone))
		@@ -2503,8 +2522,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
		* allowed per node queues are empty and that nodes are
		* over allocated.
		*/
		if (IS_ENABLED(CONFIG_NUMA) &&
		(gfp_mask & GFP_THISNODE) == GFP_THISNODE)
		if (gfp_thisnode_allocation(gfp_mask))
		goto nopage;

		restart:

Admin message