mm/lru: revise the comments of lru_lock

8. LRU

======

        Each memcg has its own private LRU. Now, its handling is under global

	VM's control (means that it's handled under global pgdat->lru_lock).

	Almost all routines around memcg's LRU is called by global LRU's

	list management functions under pgdat->lru_lock.

	A special function is mem_cgroup_isolate_pages(). This scans

	memcg's private LRU and call __isolate_lru_page() to extract a page

	from LRU.

	(By __isolate_lru_page(), the page is removed from both of global and

	private LRU.)

	Each memcg has its own vector of LRUs (inactive anon, active anon,

	inactive file, active file, unevictable) of pages from each node,

	each LRU handled under a single lru_lock for that memcg and node.

9. Typical Tests.

=================

2.6 Locking

-----------

   lock_page_cgroup()/unlock_page_cgroup() should not be called under

   the i_pages lock.

Lock order is as follows:

   Other lock order is following:

  Page lock (PG_locked bit of page->flags)

    mm->page_table_lock or split pte_lock

      lock_page_memcg (memcg->move_lock)

        mapping->i_pages lock

          lruvec->lru_lock.

   PG_locked.

     mm->page_table_lock

         pgdat->lru_lock

	   lock_page_cgroup.

  In many cases, just lock_page_cgroup() is called.

  per-zone-per-cgroup LRU (cgroup's private LRU) is just guarded by

  pgdat->lru_lock, it has no lock of its own.

Per-node-per-memcgroup LRU (cgroup's private LRU) is guarded by

lruvec->lru_lock; PG_lru bit of page->flags is cleared before

isolating a page from its LRU under lruvec->lru_lock.

2.7 Kernel Memory Extension (CONFIG_MEMCG_KMEM)

-----------------------------------------------

Broadly speaking, pages are taken off the LRU lock in bulk and

freed in batch with a page list. Significant amounts of activity here could

indicate that the system is under memory pressure and can also indicate

contention on the zone->lru_lock.

contention on the lruvec->lru_lock.

4. Per-CPU Allocator Activity

=============================

memory x86_64 systems.

To illustrate this with an example, a non-NUMA x86_64 platform with 128GB of

main memory will have over 32 million 4k pages in a single zone.  When a large

main memory will have over 32 million 4k pages in a single node.  When a large

fraction of these pages are not evictable for any reason [see below], vmscan

will spend a lot of time scanning the LRU lists looking for the small fraction

of pages that are evictable.  This can result in a situation where all CPUs are

The Unevictable Page List

-------------------------

The Unevictable LRU infrastructure consists of an additional, per-zone, LRU list

The Unevictable LRU infrastructure consists of an additional, per-node, LRU list

called the "unevictable" list and an associated page flag, PG_unevictable, to

indicate that the page is being managed on the unevictable list.

swap-backed pages.  This differentiation is only important while the pages are,

in fact, evictable.

The unevictable list benefits from the "arrayification" of the per-zone LRU

The unevictable list benefits from the "arrayification" of the per-node LRU

lists and statistics originally proposed and posted by Christoph Lameter.

The unevictable list does not use the LRU pagevec mechanism. Rather,

unevictable pages are placed directly on the page's zone's unevictable list

under the zone lru_lock.  This allows us to prevent the stranding of pages on

the unevictable list when one task has the page isolated from the LRU and other

tasks are changing the "evictability" state of the page.

Memory Control Group Interaction

--------------------------------

memory controller; see Documentation/admin-guide/cgroup-v1/memory.rst] by extending the

lru_list enum.

The memory controller data structure automatically gets a per-zone unevictable

list as a result of the "arrayification" of the per-zone LRU lists (one per

The memory controller data structure automatically gets a per-node unevictable

list as a result of the "arrayification" of the per-node LRU lists (one per

lru_list enum element).  The memory controller tracks the movement of pages to

and from the unevictable list.

active/inactive LRU lists for vmscan to deal with.  vmscan checks for such

pages in all of the shrink_{active|inactive|page}_list() functions and will

"cull" such pages that it encounters: that is, it diverts those pages to the

unevictable list for the zone being scanned.

unevictable list for the node being scanned.

There may be situations where a page is mapped into a VM_LOCKED VMA, but the

page is not marked as PG_mlocked.  Such pages will make it all the way to

page from the LRU, as it is likely on the appropriate active or inactive list

at that time.  If the isolate_lru_page() succeeds, mlock_vma_page() will put

back the page - by calling putback_lru_page() - which will notice that the page

is now mlocked and divert the page to the zone's unevictable list.  If

is now mlocked and divert the page to the node's unevictable list.  If

mlock_vma_page() is unable to isolate the page from the LRU, vmscan will handle

it later if and when it attempts to reclaim the page.

     unevictable list in mlock_vma_page().

shrink_inactive_list() also diverts any unevictable pages that it finds on the

inactive lists to the appropriate zone's unevictable list.

inactive lists to the appropriate node's unevictable list.

shrink_inactive_list() should only see SHM_LOCK'd pages that became SHM_LOCK'd

after shrink_active_list() had moved them to the inactive list, or pages mapped

		struct {	/* Page cache and anonymous pages */

			/**

			 * @lru: Pageout list, eg. active_list protected by

			 * pgdat->lru_lock.  Sometimes used as a generic list

			 * lruvec->lru_lock.  Sometimes used as a generic list

			 * by the page owner.

			 */

			struct list_head lru;