tools/mm: Add thpmaps script to dump THP usage info
With the proliferation of large folios for file-backed memory, and more recently the introduction of multi-size THP for anonymous memory, it is becoming useful to be able to see exactly how large folios are mapped into processes. For some architectures (e.g. arm64), if most memory is mapped using contpte-sized and -aligned blocks, TLB usage can be optimized so it's useful to see where these requirements are and are not being met. thpmaps is a Python utility that reads /proc/<pid>/smaps, /proc/<pid>/pagemap and /proc/kpageflags to print information about how transparent huge pages (both file and anon) are mapped to a specified process or cgroup. It aims to help users debug and optimize their workloads. In future we may wish to introduce stats directly into the kernel (e.g. smaps or similar), but for now this provides a short term solution without the need to introduce any new ABI. Run with help option for a full listing of the arguments: # ./thpmaps --help --8<-- usage: thpmaps [-h] [--pid pid] [--cgroup path] [--summary] [--cont size[KMG]] [--inc-smaps] [--inc-empty] [--periodic sleep_ms] Prints information about how transparent huge pages are mapped to a specified process or cgroup. Shows statistics for fully-mapped THPs of every size, mapped both naturally aligned and unaligned for both file and anonymous memory. See [anon|file]-thp-[aligned|unaligned]-<size>kB keys. Shows statistics for mapped pages that belong to a THP but which are not fully mapped. See [anon|file]-thp-partial keys. Optionally shows statistics for naturally aligned, contiguous blocks of memory of a specified size (when --cont is provided). See [anon|file]-cont- aligned-<size>kB keys. Statistics are shown in kB and as a percentage of either total anon or file memory as appropriate. options: -h, --help show this help message and exit --pid pid Process id of the target process. Exactly one of --pid and --cgroup must be provided. --cgroup path Path to the target cgroup in sysfs. Iterates over every pid in the cgroup and its children. Get global stats by passing in the root cgroup (e.g. /sys/fs/cgroup for cgroup-v2 or /sys/fs/cgroup/pids for cgroup-v1). Exactly one of --pid and --cgroup must be provided. --summary Sum the per-vma statistics to provide a summary over the whole process or cgroup. --cont size[KMG] Adds anon and file stats for naturally aligned, contiguously mapped blocks of the specified size. May be issued multiple times to track multiple sized blocks. Useful to infer e.g. arm64 contpte and hpa mappings. Size must be a power-of-2 number of pages. --inc-smaps Include all numerical, additive /proc/<pid>/smaps stats in the output. --inc-empty Show all statistics including those whose value is 0. --periodic sleep_ms Run in a loop, polling every sleep_ms milliseconds. Requires root privilege to access pagemap and kpageflags. --8<-- Example command to summarise fully and partially mapped THPs and 64K contiguous blocks over all VMAs in a single process (--inc-empty forces printing stats that are 0): # ./thpmaps --pid 10837 --cont 64K --summary --inc-empty --8<-- anon-thp-aligned-16kB: 16 kB ( 0%) anon-thp-aligned-32kB: 0 kB ( 0%) anon-thp-aligned-64kB: 4194304 kB (100%) anon-thp-aligned-128kB: 0 kB ( 0%) anon-thp-aligned-256kB: 0 kB ( 0%) anon-thp-aligned-512kB: 0 kB ( 0%) anon-thp-aligned-1024kB: 0 kB ( 0%) anon-thp-aligned-2048kB: 0 kB ( 0%) anon-thp-unaligned-16kB: 0 kB ( 0%) anon-thp-unaligned-32kB: 0 kB ( 0%) anon-thp-unaligned-64kB: 0 kB ( 0%) anon-thp-unaligned-128kB: 0 kB ( 0%) anon-thp-unaligned-256kB: 0 kB ( 0%) anon-thp-unaligned-512kB: 0 kB ( 0%) anon-thp-unaligned-1024kB: 0 kB ( 0%) anon-thp-unaligned-2048kB: 0 kB ( 0%) anon-thp-partial: 0 kB ( 0%) file-thp-aligned-16kB: 16 kB ( 1%) file-thp-aligned-32kB: 64 kB ( 5%) file-thp-aligned-64kB: 640 kB (50%) file-thp-aligned-128kB: 128 kB (10%) file-thp-aligned-256kB: 0 kB ( 0%) file-thp-aligned-512kB: 0 kB ( 0%) file-thp-aligned-1024kB: 0 kB ( 0%) file-thp-aligned-2048kB: 0 kB ( 0%) file-thp-unaligned-16kB: 16 kB ( 1%) file-thp-unaligned-32kB: 32 kB ( 3%) file-thp-unaligned-64kB: 64 kB ( 5%) file-thp-unaligned-128kB: 0 kB ( 0%) file-thp-unaligned-256kB: 0 kB ( 0%) file-thp-unaligned-512kB: 0 kB ( 0%) file-thp-unaligned-1024kB: 0 kB ( 0%) file-thp-unaligned-2048kB: 0 kB ( 0%) file-thp-partial: 12 kB ( 1%) anon-cont-aligned-64kB: 4194304 kB (100%) file-cont-aligned-64kB: 768 kB (61%) --8<-- Tested-by:Barry Song <v-songbaohua@oppo.com> Signed-off-by:
Ryan Roberts <ryan.roberts@arm.com>
Loading
Please register or sign in to comment