mm: memcontrol: rearrange charging fast path (06b078fc) · Commits · 戴 / test

The charging path currently starts out with OOM condition checks when OOM is the rarest possible case. Rearrange this code to run OOM/task dying checks only after trying the percpu charge and the res_counter charge and bail out before entering reclaim. Attempting a charge does not hurt an (oom-)killed task as much as every charge attempt having to check OOM conditions. Also, only check __GFP_NOFAIL when the charge would actually fail. Signed-off-by:

Johannes Weiner <hannes@cmpxchg.org> Acked-by:

Michal Hocko <mhocko@suse.cz> Cc: Hugh Dickins <hughd@google.com> Cc: Tejun Heo <tj@kernel.org> Cc: Vladimir Davydov <vdavydov@parallels.com> Signed-off-by:

Andrew Morton <akpm@linux-foundation.org> Signed-off-by:

Linus Torvalds <torvalds@linux-foundation.org>

mm/memcontrol.c

+17 −16

Original line number	Diff line number	Diff line
		@@ -2575,22 +2575,6 @@ static int mem_cgroup_try_charge(struct mem_cgroup *memcg,

		if (mem_cgroup_is_root(memcg))
		goto done;
		/*
		* Unlike in global OOM situations, memcg is not in a physical
		* memory shortage. Allow dying and OOM-killed tasks to
		* bypass the last charges so that they can exit quickly and
		* free their memory.
		*/
		if (unlikely(test_thread_flag(TIF_MEMDIE) \|\|
		fatal_signal_pending(current) \|\|
		current->flags & PF_EXITING))
		goto bypass;

		if (unlikely(task_in_memcg_oom(current)))
		goto nomem;

		if (gfp_mask & __GFP_NOFAIL)
		oom = false;
		retry:
		if (consume_stock(memcg, nr_pages))
		goto done;
		@@ -2612,6 +2596,20 @@ retry:
		goto retry;
		}

		/*
		* Unlike in global OOM situations, memcg is not in a physical
		* memory shortage. Allow dying and OOM-killed tasks to
		* bypass the last charges so that they can exit quickly and
		* free their memory.
		*/
		if (unlikely(test_thread_flag(TIF_MEMDIE) \|\|
		fatal_signal_pending(current) \|\|
		current->flags & PF_EXITING))
		goto bypass;

		if (unlikely(task_in_memcg_oom(current)))
		goto nomem;

		if (!(gfp_mask & __GFP_WAIT))
		goto nomem;

		@@ -2640,6 +2638,9 @@ retry:
		if (mem_cgroup_wait_acct_move(mem_over_limit))
		goto retry;

		if (gfp_mask & __GFP_NOFAIL)
		goto bypass;

		if (fatal_signal_pending(current))
		goto bypass;

Admin message