Commit 41d78ba5 authored by Hugh Dickins's avatar Hugh Dickins Committed by Linus Torvalds
Browse files

[PATCH] compound page: use page[1].lru 



If a compound page has its own put_page_testzero destructor (the only current
example is free_huge_page), that is noted in page[1].mapping of the compound
page.  But that's rather a poor place to keep it: functions which call
set_page_dirty_lock after get_user_pages (e.g.  Infiniband's
__ib_umem_release) ought to be checking first, otherwise set_page_dirty is
liable to crash on what's not the address of a struct address_space.

And now I'm about to make that worse: it turns out that every compound page
needs a destructor, so we can no longer rely on hugetlb pages going their own
special way, to avoid further problems of page->mapping reuse.  For example,
not many people know that: on 50% of i386 -Os builds, the first tail page of a
compound page purports to be PageAnon (when its destructor has an odd
address), which surprises page_add_file_rmap.

Keep the compound page destructor in page[1].lru.next instead.  And to free up
the common pairing of mapping and index, also move compound page order from
index to lru.prev.  Slab reuses page->lru too: but if we ever need slab to use
compound pages, it can easily stack its use above this.

(akpm: decoded version of the above: the tail pages of a compound page now
have ->mapping==NULL, so there's no need for the set_page_dirty[_lock]()
caller to check that they're not compund pages before doing the dirty).

Signed-off-by: default avatarHugh Dickins <hugh@veritas.com>
Signed-off-by: default avatarAndrew Morton <akpm@osdl.org>
Signed-off-by: default avatarLinus Torvalds <torvalds@osdl.org>
parent 72772323

mm/hugetlb.c

+2 −2
Original line number Diff line number Diff line
@@ -85,7 +85,7 @@ void free_huge_page(struct page *page) 
	BUG_ON(page_count(page));



	INIT_LIST_HEAD(&page->lru);

	page[1].mapping = NULL;

	page[1].lru.next = NULL;			/* reset dtor */



	spin_lock(&hugetlb_lock);

	enqueue_huge_page(page);
@@ -105,7 +105,7 @@ struct page *alloc_huge_page(struct vm_area_struct *vma, unsigned long addr) 
	}

	spin_unlock(&hugetlb_lock);

	set_page_count(page, 1);

	page[1].mapping = (void *)free_huge_page;

	page[1].lru.next = (void *)free_huge_page;	/* set dtor */

	for (i = 0; i < (HPAGE_SIZE/PAGE_SIZE); ++i)

		clear_user_highpage(&page[i], addr);

	return page;

mm/page_alloc.c

+6 −9
Original line number Diff line number Diff line
@@ -169,20 +169,17 @@ static void bad_page(struct page *page) 
 * All pages have PG_compound set.  All pages have their ->private pointing at

 * the head page (even the head page has this).

 *

 * The first tail page's ->mapping, if non-zero, holds the address of the

 * compound page's put_page() function.

 *

 * The order of the allocation is stored in the first tail page's ->index

 * This is only for debug at present.  This usage means that zero-order pages

 * may not be compound.

 * The first tail page's ->lru.next holds the address of the compound page's

 * put_page() function.  Its ->lru.prev holds the order of allocation.

 * This usage means that zero-order pages may not be compound.

 */

static void prep_compound_page(struct page *page, unsigned long order)

{

	int i;

	int nr_pages = 1 << order;



	page[1].mapping = NULL;

	page[1].index = order;

	page[1].lru.next = NULL;			/* set dtor */

	page[1].lru.prev = (void *)order;

	for (i = 0; i < nr_pages; i++) {

		struct page *p = page + i;
@@ -196,7 +193,7 @@ static void destroy_compound_page(struct page *page, unsigned long order) 
	int i;

	int nr_pages = 1 << order;



	if (unlikely(page[1].index != order))

	if (unlikely((unsigned long)page[1].lru.prev != order))

		bad_page(page);



	for (i = 0; i < nr_pages; i++) {

mm/swap.c

+1 −1
Original line number Diff line number Diff line
@@ -40,7 +40,7 @@ static void put_compound_page(struct page *page) 
	if (put_page_testzero(page)) {

		void (*dtor)(struct page *page);



		dtor = (void (*)(struct page *))page[1].mapping;

		dtor = (void (*)(struct page *))page[1].lru.next;

		(*dtor)(page);

	}

}

CRA Git | Maintained and supported by SUSTech CRA and CCSE