linux/lib
Marco Elver 31639fd6ce stackdepot: use variable size records for non-evictable entries
With the introduction of stack depot evictions, each stack record is now
fixed size, so that future reuse after an eviction can safely store
differently sized stack traces.  In all cases that do not make use of
evictions, this wastes lots of space.

Fix it by re-introducing variable size stack records (up to the max
allowed size) for entries that will never be evicted.  We know if an entry
will never be evicted if the flag STACK_DEPOT_FLAG_GET is not provided,
since a later stack_depot_put() attempt is undefined behavior.

With my current kernel config that enables KASAN and also SLUB owner
tracking, I observe (after a kernel boot) a whopping reduction of 296
stack depot pools, which translates into 4736 KiB saved.  The savings here
are from SLUB owner tracking only, because KASAN generic mode still uses
refcounting.

Before:

  pools: 893
  allocations: 29841
  frees: 6524
  in_use: 23317
  freelist_size: 3454

After:

  pools: 597
  refcounted_allocations: 17547
  refcounted_frees: 6477
  refcounted_in_use: 11070
  freelist_size: 3497
  persistent_count: 12163
  persistent_bytes: 1717008

[elver@google.com: fix -Wstringop-overflow warning]
  Link: https://lore.kernel.org/all/20240201135747.18eca98e@canb.auug.org.au/
  Link: https://lkml.kernel.org/r/20240201090434.1762340-1-elver@google.com
  Link: https://lore.kernel.org/all/CABXGCsOzpRPZGg23QqJAzKnqkZPKzvieeg=W7sgjgi3q0pBo0g@mail.gmail.com/
Link: https://lkml.kernel.org/r/20240129100708.39460-1-elver@google.com
Link: https://lore.kernel.org/all/CABXGCsOzpRPZGg23QqJAzKnqkZPKzvieeg=W7sgjgi3q0pBo0g@mail.gmail.com/
Fixes: 108be8def4 ("lib/stackdepot: allow users to evict stack traces")
Signed-off-by: Marco Elver <elver@google.com>
Reviewed-by: Andrey Konovalov <andreyknvl@gmail.com>
Tested-by: Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2024-02-23 17:27:12 -08:00
..
842
crypto crypto: lib/mpi - Fix unexpected pointer access in mpi_ec_init 2023-12-22 12:30:19 +08:00
dim linux/dim: Do nothing if no time delta between samples 2023-05-09 11:06:45 +02:00
fonts powerpc: Use shared font data 2023-10-01 23:09:02 +11:00
kunit kunit: device: Unregister the kunit_bus on shutdown 2024-02-06 17:07:37 -07:00
livepatch
lz4
lzo
math lib/math/int_log: Replace LGPL-2.1-or-later boilerplate with SPDX identifier 2023-07-09 22:47:50 +01:00
pldmfw lib: remove MODULE_LICENSE in non-modules 2023-04-13 13:13:53 -07:00
raid6 s390/fpu: get rid of MACHINE_HAS_VX 2023-12-11 14:33:07 +01:00
reed_solomon
test_fortify string: Remove strlcpy() 2024-01-19 11:59:11 -08:00
vdso vdso: Improve cmd_vdso_check to check all dynamic relocations 2023-03-21 21:15:34 +01:00
xz arch: Remove Itanium (IA-64) architecture 2023-09-11 08:13:17 +00:00
zlib_deflate lib/zlib: DFLTCC deflate does not write all available bits for Z_NO_FLUSH 2023-02-27 17:00:14 -08:00
zlib_dfltcc lib/zlib: remove redundation assignement of avail_in dfltcc_gdht() 2023-02-02 22:50:10 -08:00
zlib_inflate lib/zlib: Split deflate and inflate states for DFLTCC 2023-02-02 22:50:09 -08:00
zstd zstd: Fix array-index-out-of-bounds UBSAN warning 2023-11-14 17:12:52 -08:00
.gitignore
argv_split.c argv_split: fix kernel-doc warnings 2023-09-19 13:21:33 -07:00
ashldi3.c
ashrdi3.c
asn1_decoder.c
asn1_encoder.c
assoc_array.c
atomic64.c
atomic64_test.c
audit.c
base64.c
bcd.c
bch.c lib/bch.c: use bitrev instead of internal logic 2023-08-18 10:18:58 -07:00
bitfield_kunit.c
bitmap-str.c lib/bitmap: split-out string-related operations to a separate files 2023-10-14 20:25:22 -07:00
bitmap.c bitmap: move bitmap_*_region() functions to bitmap.h 2023-10-16 16:14:45 -07:00
bitrev.c
bootconfig-data.S
bootconfig.c
bsearch.c
btree.c btree: remove MODULE_LICENSE in non-modules 2023-04-13 13:13:54 -07:00
bucket_locks.c
bug.c
build_OID_registry
buildid.c ELF: fix all "Elf" typos 2023-04-08 13:45:37 -07:00
bust_spinlocks.c
check_signature.c
checksum.c
checksum_kunit.c lib: checksum: Fix build with CONFIG_NET=n 2024-01-19 08:12:38 -08:00
closure.c closures: CLOSURE_CALLBACK() to fix type punning 2023-11-24 00:29:58 -05:00
clz_ctz.c lib/clz_ctz.c: Fix __clzdi2() and __ctzdi2() for 32-bit kernels 2023-08-25 13:22:10 -07:00
clz_tab.c
cmdline.c
cmdline_kunit.c
cmpdi2.c
compat_audit.c
cpu_rmap.c lib: cpu_rmap: Fix potential use-after-free in irq_cpu_rmap_release() 2023-06-07 21:25:00 -07:00
cpumask.c bitmap patches for v6.7 2023-11-03 07:08:36 -10:00
cpumask_kunit.c cpumask: re-introduce constant-sized cpumask optimizations 2023-03-05 14:30:34 -08:00
crc-ccitt.c lib: crc_ccitt_false() is identical to crc_itu_t() 2023-12-29 12:22:26 -08:00
crc-itu-t.c
crc-t10dif.c
crc4.c
crc7.c
crc8.c
crc16.c
crc32.c
crc32defs.h
crc32test.c
crc64-rocksoft.c
crc64.c
ctype.c
debug_info.c
debug_locks.c
debugobjects.c debugobjects: Stop accessing objects after releasing hash bucket lock 2023-11-22 10:41:46 +01:00
dec_and_lock.c perf: Fix perf_event_pmu_context serialization 2023-01-31 20:37:18 +01:00
decompress.c
decompress_bunzip2.c
decompress_inflate.c decompressor: provide missing prototypes 2023-06-09 17:44:17 -07:00
decompress_unlz4.c
decompress_unlzma.c
decompress_unlzo.c
decompress_unxz.c arch: Remove Itanium (IA-64) architecture 2023-09-11 08:13:17 +00:00
decompress_unzstd.c decompressor: provide missing prototypes 2023-06-09 17:44:17 -07:00
devmem_is_allowed.c lib: devmem_is_allowed: include linux/io.h 2023-06-09 17:44:15 -07:00
devres.c devres: show which resource was invalid in __devm_ioremap_resource() 2023-06-23 17:04:04 -07:00
dhry.h lib: add Dhrystone benchmark test 2023-02-02 22:50:01 -08:00
dhry_1.c lib: dhry: fix sleeping allocations inside non-preemptable section 2023-07-08 09:29:32 -07:00
dhry_2.c lib: add Dhrystone benchmark test 2023-02-02 22:50:01 -08:00
dhry_run.c lib: dhry: fix unstable smp_processor_id(_) usage 2023-03-23 17:18:35 -07:00
digsig.c
dump_stack.c
dynamic_debug.c dyndbg: add source filename to prefix 2023-08-04 15:28:41 +02:00
dynamic_queue_limits.c
earlycpio.c
errname.c parisc: Drop the HP-UX ENOSYM and EREMOTERELEASE error codes 2023-11-25 09:43:18 +01:00
error-inject.c lib: error-inject: remove error checking for debugfs_create_dir() 2023-08-18 10:18:55 -07:00
errseq.c
extable.c
fault-inject-usercopy.c
fault-inject.c fault-inject: allow configuration via configfs 2023-04-13 07:38:54 -06:00
fdt.c
fdt_addresses.c
fdt_empty_tree.c
fdt_ro.c
fdt_rw.c
fdt_strerror.c
fdt_sw.c
fdt_wip.c
find_bit.c cpumask: introduce for_each_cpu_or 2023-03-19 10:02:04 -07:00
find_bit_benchmark.c
flex_proportions.c
fortify_kunit.c fortify: test: Use kunit_device 2023-12-18 13:28:08 -07:00
fw_table.c cxl for v6.8 2024-01-18 16:22:43 -08:00
gen_crc32table.c
gen_crc64table.c
genalloc.c Devicetree include cleanups for v6.6: 2023-08-30 17:04:28 -07:00
generic-radix-tree.c lib/generic-radix-tree.c: Add peek_prev() 2023-10-19 14:47:33 -04:00
glob.c
globtest.c
group_cpus.c lib/group_cpus.c: avoid acquiring cpu hotplug lock in group_cpus_evenly 2023-12-06 16:12:46 -08:00
hashtable_test.c lib/hashtable_test.c: add test for the hashtable structure 2023-02-08 14:28:17 -07:00
hexdump.c
hweight.c
idr.c ida: Fix crash in ida_free when the bitmap is empty 2023-12-21 10:02:28 -08:00
inflate.c
interval_tree.c
interval_tree_test.c
iomap.c
iomap_copy.c
iommu-helper.c
iov_iter.c vfs-6.8.iov_iter 2024-01-08 11:43:04 -08:00
irq_poll.c
irq_regs.c
is_signed_type_kunit.c
is_single_threaded.c
kasprintf.c
Kconfig lib/stackdepot: use fixed-sized slots for stack records 2023-12-10 16:51:44 -08:00
Kconfig.debug lib/Kconfig.debug: TEST_IOV_ITER depends on MMU 2024-02-20 14:20:48 -08:00
Kconfig.kasan Many singleton patches against the MM code. The patch series which 2024-01-09 11:18:47 -08:00
Kconfig.kcsan Kernel concurrency sanitizer (KCSAN) updates for v6.3 2023-02-25 13:02:20 -08:00
Kconfig.kfence mm/slab: remove CONFIG_SLAB from all Kconfig and Makefile 2023-12-05 11:14:40 +01:00
Kconfig.kgdb
Kconfig.kmsan mm/slab: remove CONFIG_SLAB from all Kconfig and Makefile 2023-12-05 11:14:40 +01:00
Kconfig.ubsan ubsan: Clarify Kconfig text for CONFIG_UBSAN_TRAP 2023-07-17 16:05:19 -07:00
kfifo.c
klist.c
kobject.c Revert "kobject: Remove redundant checks for whether ktype is NULL" 2024-02-08 16:39:25 +00:00
kobject_uevent.c kobject: Replace strlcpy with strscpy 2023-09-22 09:50:56 -07:00
kstrtox.c kstrtox: consistently use _tolower() 2023-08-21 13:46:25 -07:00
kstrtox.h
kunit_iov_iter.c iov_iter: Kunit tests for page extraction 2023-09-09 15:11:49 -07:00
libcrc32c.c libcrc32c: remove crc32c_impl 2023-04-17 18:01:23 +02:00
linear_ranges.c
list-test.c list: test: Test the klist structure 2023-03-31 09:21:35 -06:00
list_debug.c list: Introduce CONFIG_LIST_HARDENED 2023-08-15 14:57:25 -07:00
list_sort.c
llist.c llist: add llist_del_first_this() 2023-10-16 12:44:06 -04:00
locking-selftest-hardirq.h
locking-selftest-mutex.h
locking-selftest-rlock-hardirq.h
locking-selftest-rlock-softirq.h
locking-selftest-rlock.h
locking-selftest-rsem.h
locking-selftest-rtmutex.h
locking-selftest-softirq.h
locking-selftest-spin-hardirq.h
locking-selftest-spin-softirq.h
locking-selftest-spin.h
locking-selftest-wlock-hardirq.h
locking-selftest-wlock-softirq.h
locking-selftest-wlock.h
locking-selftest-wsem.h
locking-selftest.c lockdep/selftests: Use SBRM APIs for wait context tests 2023-07-26 12:29:13 +02:00
lockref.c
logic_iomem.c
logic_pio.c minmax: add in_range() macro 2023-08-24 16:20:18 -07:00
lru_cache.c
lshrdi3.c
lwq.c lib: add light-weight queuing mechanism. 2023-10-16 12:44:06 -04:00
Makefile cxl for v6.7 2023-11-04 16:20:36 -10:00
maple_tree.c maple_tree: avoid checking other gaps after getting the largest gap 2023-12-20 14:48:14 -08:00
memcat_p.c
memcpy_kunit.c kunit: memcpy: Mark tests as slow using test attributes 2023-07-26 13:29:28 -06:00
memory-notifier-error-inject.c
memregion.c
memweight.c
muldi3.c
net_utils.c mac_pton: Clean up the header inclusions 2023-06-06 13:18:32 +02:00
netdev-notifier-error-inject.c
nlattr.c string: Remove strlcpy() 2024-01-19 11:59:11 -08:00
nmi_backtrace.c nmi_backtrace: allow excluding an arbitrary CPU 2023-08-18 10:19:00 -07:00
notifier-error-inject.c lib: remove error checking for debugfs_create_dir() 2023-08-18 10:18:55 -07:00
notifier-error-inject.h
objagg.c
objpool.c lib: objpool: fix head overrun on RK3588 SBC 2023-12-01 14:53:55 +09:00
of-reconfig-notifier-error-inject.c
oid_registry.c
once.c
overflow_kunit.c overflow: Replace fake root_device with kunit_device 2023-12-18 13:28:08 -07:00
packing.c lib: packing: remove MODULE_LICENSE in non-modules 2023-03-09 23:08:04 -08:00
parman.c
parser.c lib: parser: update documentation for match_NUMBER functions 2023-03-02 21:54:22 -08:00
pci_iomap.c
percpu-refcount.c
percpu_counter.c percpu_counter: extend _limited_add() to negative amounts 2023-10-18 14:34:14 -07:00
percpu_test.c
plist.c
pm-notifier-error-inject.c
polynomial.c
radix-tree.c radix tree: remove unused variable 2023-08-21 13:07:22 -07:00
radix-tree.h radix-tree: move declarations to header 2023-06-12 11:31:50 -07:00
random32.c
ratelimit.c
rbtree.c lib/rbtree: use '+' instead of '|' for setting color. 2023-04-18 16:39:33 -07:00
rbtree_test.c
rcuref.c locking/atomics: Use atomic_try_cmpxchg_release() to micro-optimize rcuref_put_slowpath() 2023-10-10 10:14:27 +02:00
ref_tracker.c lib/ref_tracker: remove warnings in case of allocation failure 2023-06-05 15:28:42 -07:00
refcount.c
rhashtable.c
sbitmap.c sbitmap: remove stale comment in sbq_calc_wake_batch 2024-01-15 07:23:50 -07:00
scatterlist.c scatterlist: add missing function params to kernel-doc 2023-09-19 13:21:33 -07:00
seq_buf.c seq_buf: Fix kernel documentation 2024-02-15 12:17:28 -05:00
sg_pool.c
sg_split.c
siphash.c
siphash_kunit.c
slub_kunit.c
smp_processor_id.c
sort.c
stackdepot.c stackdepot: use variable size records for non-evictable entries 2024-02-23 17:27:12 -08:00
stackinit_kunit.c kernel/range: Uplevel the cxl subsystem's range_contains() helper 2023-02-10 17:32:37 -08:00
stmp_device.c
strcat_kunit.c string: Add Kunit tests for strcat() family 2023-05-16 14:08:02 -07:00
string.c string: Remove strlcpy() 2024-01-19 11:59:11 -08:00
string_helpers.c lib/string_helpers: string_get_size() now returns characters wrote 2023-09-11 23:59:47 -04:00
strncpy_from_user.c
strnlen_user.c
strscpy_kunit.c
syscall.c
test-kstrtox.c
test-string_helpers.c lib/test-string_helpers: replace UNESCAPE_ANY by UNESCAPE_ALL_MASK 2023-04-08 13:45:39 -07:00
test_bitmap.c bitmap: add test for bitmap_*_region() functions 2023-10-14 20:25:22 -07:00
test_bitops.c
test_bits.c
test_blackhole_dev.c
test_bpf.c test_bpf: Rename second ALU64_SMOD_X to ALU64_SMOD_K 2023-12-09 21:27:54 -08:00
test_debug_virtual.c
test_dynamic_debug.c
test_firmware.c firmware_loader: Expand Firmware upload error codes with firmware invalid error 2023-11-24 18:09:19 -08:00
test_fprobe.c fprobe: Pass return address to the handlers 2023-06-06 21:39:55 +09:00
test_fpu.c
test_free_pages.c
test_hash.c
test_hexdump.c
test_hmm.c lib: replace kmap() with kmap_local_page() 2023-08-18 10:18:50 -07:00
test_hmm_uapi.h
test_ida.c Quite a lot of kexec work this time around. Many singleton patches in 2024-01-09 11:46:20 -08:00
test_kmod.c
test_kprobes.c test_kprobes: Add recursed kprobe test case 2023-02-21 08:52:42 +09:00
test_linear_ranges.c
test_list_sort.c
test_lockup.c
test_maple_tree.c maple_tree: don't find node end in mtree_lookup_walk() 2023-12-12 10:56:59 -08:00
test_memcat_p.c
test_meminit.c mm, treewide: introduce NR_PAGE_ORDERS 2024-01-08 15:27:15 -08:00
test_min_heap.c
test_module.c
test_objagg.c
test_objpool.c lib: test_objpool: make global variables static 2023-11-10 19:59:04 +09:00
test_parman.c
test_printf.c lib/vsprintf: declare no_hash_pointers in sprintf.h 2023-08-21 13:46:24 -07:00
test_ref_tracker.c lib/ref_tracker: improve printing stats 2023-06-05 15:28:42 -07:00
test_rhashtable.c Kill sched.h dependency on rcupdate.h 2023-12-27 11:50:20 -05:00
test_scanf.c lib: test_scanf: Add explicit type cast to result initialization in test_number_prefix() 2023-08-16 11:47:29 +02:00
test_sort.c
test_static_key_base.c
test_static_keys.c
test_string.c
test_sysctl.c sysctl: Remove the now superfluous sentinel elements from ctl_table array 2023-12-28 04:57:57 -08:00
test_ubsan.c
test_user_copy.c
test_uuid.c
test_vmalloc.c lib/test_vmalloc.c: avoid garbage in page array 2023-06-12 11:31:51 -07:00
test_xarray.c
textsearch.c
timerqueue.c
trace_readwrite.c lib/trace_readwrite.c:: replace asm-generic/io with linux/io 2023-12-29 12:22:29 -08:00
ts_bm.c lib/ts_bm: add helper to reduce indentation and improve readability 2023-07-27 13:45:51 +02:00
ts_fsm.c
ts_kmp.c
ubsan.c UBSAN: use the kernel panic message markers 2023-12-20 14:48:14 -08:00
ubsan.h ubsan: add prototypes for internal functions 2023-05-30 16:42:01 -07:00
ucmpdi2.c
ucs2_string.c lib/ucs2_string: Add UCS-2 strscpy function 2023-09-13 10:18:42 -07:00
usercopy.c uaccess: Add speculation barrier to copy_from_user() 2023-02-21 14:45:22 -08:00
uuid.c
vsprintf.c lib/vsprintf: Fix %pfwf when current node refcount == 0 2023-12-06 11:06:59 +01:00
win_minmax.c
xarray.c xarray: Document necessary flag in alloc functions 2023-09-05 19:01:38 -04:00
xxhash.c