system/linux
system/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux synced 2024-07-21 10:41:44 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

mm: allow anon exclusive check over hugetlb tail pages

Browse source 
PageAnonExclusive() used to forbid tail pages for hugetlbfs, as that used
to be called mostly in hugetlb specific paths and the head page was
guaranteed.

As we move forward towards merging hugetlb paths into generic mm, we may
start to pass in tail hugetlb pages (when with cont-pte/cont-pmd huge
pages) for such check.  Allow it to properly fetch the head, in which case
the anon-exclusiveness of the head will always represents the tail page.

There's already a sign of it when we look at the GUP-fast which already
contain the hugetlb processing altogether: we used to have a specific
commit 5805192c7b ("mm/gup: handle cont-PTE hugetlb pages correctly in
gup_must_unshare() via GUP-fast") covering that area.  Now with this more
generic change, that can also go away.

[akpm@linux-foundation.org: simplify PageAnonExclusive(), per Matthew]
  Link: https://lkml.kernel.org/r/Zg3u5Sh9EbbYPhaI@casper.infradead.org
Link: https://lkml.kernel.org/r/20240403013249.1418299-2-peterx@redhat.com
Signed-off-by: Peter Xu <peterx@redhat.com>
Acked-by: David Hildenbrand <david@redhat.com>
Cc: Huacai Chen <chenhuacai@kernel.org>
Cc: Jason Gunthorpe <jgg@nvidia.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Nathan Chancellor <nathan@kernel.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: WANG Xuerui <kernel@xen0n.name>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
This commit is contained in:
Peter Xu 2024-04-02 21:32:47 -04:00 committed by Andrew Morton
parent 9cb28da546
commit c0bff412e6
2 changed files with 6 additions and 11 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
include/linux/page-flags.h
View file

@ -1095,7 +1095,12 @@ PAGEFLAG(Isolated, isolated, PF_ANY);
static __always_inline int PageAnonExclusive(const struct page *page)
{
VM_BUG_ON_PGFLAGS(!PageAnon(page), page);
VM_BUG_ON_PGFLAGS(PageHuge(page) && !PageHead(page), page);
/*
* HugeTLB stores this information on the head page; THP keeps it per
* page
*/
if (PageHuge(page))
page = compound_head(page);
return test_bit(PG_anon_exclusive, &PF_ANY(page, 1)->flags);
}

10
mm/internal.h
View file

@ -1203,16 +1203,6 @@ static inline bool gup_must_unshare(struct vm_area_struct *vma,
if (IS_ENABLED(CONFIG_HAVE_FAST_GUP))
smp_rmb();
/*
* During GUP-fast we might not get called on the head page for a
* hugetlb page that is mapped using cont-PTE, because GUP-fast does
* not work with the abstracted hugetlb PTEs that always point at the
* head page. For hugetlb, PageAnonExclusive only applies on the head
* page (as it cannot be partially COW-shared), so lookup the head page.
*/
if (unlikely(!PageHead(page) && PageHuge(page)))
page = compound_head(page);
/*
* Note that PageKsm() pages cannot be exclusive, and consequently,
* cannot get pinned.