1================================================== 2page owner: Tracking about who allocated each page 3================================================== 4 5Introduction 6============ 7 8page owner is for the tracking about who allocated each page. 9It can be used to debug memory leak or to find a memory hogger. 10When allocation happens, information about allocation such as call stack 11and order of pages is stored into certain storage for each page. 12When we need to know about status of all pages, we can get and analyze 13this information. 14 15Although we already have tracepoint for tracing page allocation/free, 16using it for analyzing who allocate each page is rather complex. We need 17to enlarge the trace buffer for preventing overlapping until userspace 18program launched. And, launched program continually dump out the trace 19buffer for later analysis and it would change system behaviour with more 20possibility rather than just keeping it in memory, so bad for debugging. 21 22page owner can also be used for various purposes. For example, accurate 23fragmentation statistics can be obtained through gfp flag information of 24each page. It is already implemented and activated if page owner is 25enabled. Other usages are more than welcome. 26 27It can also be used to show all the stacks and their current number of 28allocated base pages, which gives us a quick overview of where the memory 29is going without the need to screen through all the pages and match the 30allocation and free operation. 31 32page owner is disabled by default. So, if you'd like to use it, you need 33to add "page_owner=on" to your boot cmdline. If the kernel is built 34with page owner and page owner is disabled in runtime due to not enabling 35boot option, runtime overhead is marginal. If disabled in runtime, it 36doesn't require memory to store owner information, so there is no runtime 37memory overhead. And, page owner inserts just two unlikely branches into 38the page allocator hotpath and if not enabled, then allocation is done 39like as the kernel without page owner. These two unlikely branches should 40not affect to allocation performance, especially if the static keys jump 41label patching functionality is available. Following is the kernel's code 42size change due to this facility. 43 44Although enabling page owner increases kernel size by several kilobytes, 45most of this code is outside page allocator and its hot path. Building 46the kernel with page owner and turning it on if needed would be great 47option to debug kernel memory problem. 48 49There is one notice that is caused by implementation detail. page owner 50stores information into the memory from struct page extension. This memory 51is initialized some time later than that page allocator starts in sparse 52memory system, so, until initialization, many pages can be allocated and 53they would have no owner information. To fix it up, these early allocated 54pages are investigated and marked as allocated in initialization phase. 55Although it doesn't mean that they have the right owner information, 56at least, we can tell whether the page is allocated or not, 57more accurately. On 2GB memory x86-64 VM box, 13343 early allocated pages 58are caught and marked, although they are mostly allocated from struct 59page extension feature. Anyway, after that, no page is left in 60un-tracking state. 61 62Usage 63===== 64 651) Build user-space helper:: 66 67 cd tools/mm 68 make page_owner_sort 69 702) Enable page owner: add "page_owner=on" to boot cmdline. 71 723) Do the job that you want to debug. 73 744) Analyze information from page owner:: 75 76 cat /sys/kernel/debug/page_owner_stacks/show_stacks > stacks.txt 77 cat stacks.txt 78 post_alloc_hook+0x177/0x1a0 79 get_page_from_freelist+0xd01/0xd80 80 __alloc_pages+0x39e/0x7e0 81 allocate_slab+0xbc/0x3f0 82 ___slab_alloc+0x528/0x8a0 83 kmem_cache_alloc+0x224/0x3b0 84 sk_prot_alloc+0x58/0x1a0 85 sk_alloc+0x32/0x4f0 86 inet_create+0x427/0xb50 87 __sock_create+0x2e4/0x650 88 inet_ctl_sock_create+0x30/0x180 89 igmp_net_init+0xc1/0x130 90 ops_init+0x167/0x410 91 setup_net+0x304/0xa60 92 copy_net_ns+0x29b/0x4a0 93 create_new_namespaces+0x4a1/0x820 94 nr_base_pages: 16 95 ... 96 ... 97 echo 7000 > /sys/kernel/debug/page_owner_stacks/count_threshold 98 cat /sys/kernel/debug/page_owner_stacks/show_stacks> stacks_7000.txt 99 cat stacks_7000.txt 100 post_alloc_hook+0x177/0x1a0 101 get_page_from_freelist+0xd01/0xd80 102 __alloc_pages+0x39e/0x7e0 103 alloc_pages_mpol+0x22e/0x490 104 folio_alloc+0xd5/0x110 105 filemap_alloc_folio+0x78/0x230 106 page_cache_ra_order+0x287/0x6f0 107 filemap_get_pages+0x517/0x1160 108 filemap_read+0x304/0x9f0 109 xfs_file_buffered_read+0xe6/0x1d0 [xfs] 110 xfs_file_read_iter+0x1f0/0x380 [xfs] 111 __kernel_read+0x3b9/0x730 112 kernel_read_file+0x309/0x4d0 113 __do_sys_finit_module+0x381/0x730 114 do_syscall_64+0x8d/0x150 115 entry_SYSCALL_64_after_hwframe+0x62/0x6a 116 nr_base_pages: 20824 117 ... 118 119 cat /sys/kernel/debug/page_owner > page_owner_full.txt 120 ./page_owner_sort page_owner_full.txt sorted_page_owner.txt 121 122 The general output of ``page_owner_full.txt`` is as follows:: 123 124 Page allocated via order XXX, ... 125 PFN XXX ... 126 // Detailed stack 127 128 Page allocated via order XXX, ... 129 PFN XXX ... 130 // Detailed stack 131 By default, it will do full pfn dump, to start with a given pfn, 132 page_owner supports fseek. 133 134 FILE *fp = fopen("/sys/kernel/debug/page_owner", "r"); 135 fseek(fp, pfn_start, SEEK_SET); 136 137 The ``page_owner_sort`` tool ignores ``PFN`` rows, puts the remaining rows 138 in buf, uses regexp to extract the page order value, counts the times 139 and pages of buf, and finally sorts them according to the parameter(s). 140 141 See the result about who allocated each page 142 in the ``sorted_page_owner.txt``. General output:: 143 144 XXX times, XXX pages: 145 Page allocated via order XXX, ... 146 // Detailed stack 147 148 By default, ``page_owner_sort`` is sorted according to the times of buf. 149 If you want to sort by the page nums of buf, use the ``-m`` parameter. 150 The detailed parameters are: 151 152 fundamental function:: 153 154 Sort: 155 -a Sort by memory allocation time. 156 -m Sort by total memory. 157 -p Sort by pid. 158 -P Sort by tgid. 159 -n Sort by task command name. 160 -r Sort by memory release time. 161 -s Sort by stack trace. 162 -t Sort by times (default). 163 --sort <order> Specify sorting order. Sorting syntax is [+|-]key[,[+|-]key[,...]]. 164 Choose a key from the **STANDARD FORMAT SPECIFIERS** section. The "+" is 165 optional since default direction is increasing numerical or lexicographic 166 order. Mixed use of abbreviated and complete-form of keys is allowed. 167 168 Examples: 169 ./page_owner_sort <input> <output> --sort=n,+pid,-tgid 170 ./page_owner_sort <input> <output> --sort=at 171 172 additional function:: 173 174 Cull: 175 --cull <rules> 176 Specify culling rules.Culling syntax is key[,key[,...]].Choose a 177 multi-letter key from the **STANDARD FORMAT SPECIFIERS** section. 178 179 <rules> is a single argument in the form of a comma-separated list, 180 which offers a way to specify individual culling rules. The recognized 181 keywords are described in the **STANDARD FORMAT SPECIFIERS** section below. 182 <rules> can be specified by the sequence of keys k1,k2, ..., as described in 183 the STANDARD SORT KEYS section below. Mixed use of abbreviated and 184 complete-form of keys is allowed. 185 186 Examples: 187 ./page_owner_sort <input> <output> --cull=stacktrace 188 ./page_owner_sort <input> <output> --cull=st,pid,name 189 ./page_owner_sort <input> <output> --cull=n,f 190 191 Filter: 192 -f Filter out the information of blocks whose memory has been released. 193 194 Select: 195 --pid <pidlist> Select by pid. This selects the blocks whose process ID 196 numbers appear in <pidlist>. 197 --tgid <tgidlist> Select by tgid. This selects the blocks whose thread 198 group ID numbers appear in <tgidlist>. 199 --name <cmdlist> Select by task command name. This selects the blocks whose 200 task command name appear in <cmdlist>. 201 202 <pidlist>, <tgidlist>, <cmdlist> are single arguments in the form of a comma-separated list, 203 which offers a way to specify individual selecting rules. 204 205 206 Examples: 207 ./page_owner_sort <input> <output> --pid=1 208 ./page_owner_sort <input> <output> --tgid=1,2,3 209 ./page_owner_sort <input> <output> --name name1,name2 210 211STANDARD FORMAT SPECIFIERS 212========================== 213:: 214 215 For --sort option: 216 217 KEY LONG DESCRIPTION 218 p pid process ID 219 tg tgid thread group ID 220 n name task command name 221 st stacktrace stack trace of the page allocation 222 T txt full text of block 223 ft free_ts timestamp of the page when it was released 224 at alloc_ts timestamp of the page when it was allocated 225 ator allocator memory allocator for pages 226 227 For --cull option: 228 229 KEY LONG DESCRIPTION 230 p pid process ID 231 tg tgid thread group ID 232 n name task command name 233 f free whether the page has been released or not 234 st stacktrace stack trace of the page allocation 235 ator allocator memory allocator for pages 236