freebsd-src/sys
Mark Johnston b16b4c22d2 vm_page: Implement lazy page initialization
FreeBSD's boot times have decreased to the point where vm_page array
initialization represents a significant fraction of the total boot time.
For example, when booting FreeBSD in Firecracker (a VMM designed to
support lightweight VMs) with 128MB and 1GB of RAM, vm_page
initialization consumes 9% (3ms) and 37% (21.5ms) of the kernel boot
time, respectively.  This is generally relevant in cloud environments,
where one wants to be able to spin up VMs as quickly as possible.

This patch implements lazy initialization of (most) page structures,
following a suggestion from cperciva@.  The idea is to introduce a new
free pool, VM_FREEPOOL_LAZYINIT, into which all vm_page structures are
initially placed.  For this to work, we need only initialize the first
free page of each chunk placed into the buddy allocator.  Then, early
page allocations draw from the lazy init pool and initialize vm_page
chunks (up to 16MB, 4096 pages) on demand.  Once APs are started, an
idle-priority thread drains the lazy init pool in the background to
avoid introducing extra latency in the allocator.  With this scheme,
almost all of the initialization work is moved out of the critical path.

A couple of vm_phys operations require the pool to be drained before
they can run: vm_phys_find_range() and vm_phys_unfree_page().  However,
these are rare operations.  I believe that
vm_phys_find_freelist_contig() does not require any special treatment,
as it only ever accesses the first page in a power-of-2-sized free page
chunk, which is always initialized.

For now the new pool is only used on amd64 and arm64, since that's where
I can easily test and those platforms would get the most benefit.

Reviewed by:	alc, kib
Differential Revision:	https://reviews.freebsd.org/D40403
2024-06-13 21:19:00 -04:00
..
amd64 vm_page: Implement lazy page initialization 2024-06-13 21:19:00 -04:00
arm Remove the arm FIQ support 2024-06-10 15:16:10 +00:00
arm64 vm_page: Implement lazy page initialization 2024-06-13 21:19:00 -04:00
bsm
cam ctladm: don't require the use of "-p" with "port -r" 2024-06-10 10:01:25 -06:00
cddl arm64: Support BTI checking in most of the kernel 2024-06-05 09:23:40 +00:00
compat LinuxKPI: 802.11: use net80211 IEEE80211_HTCAP_* definitions 2024-06-07 23:57:04 +00:00
conf Remove the arm FIQ support 2024-06-10 15:16:10 +00:00
contrib openzfs: Fix mismerge breaking macOS bootstrap 2024-05-31 22:12:18 +01:00
crypto
ddb sys/ddb: Add hardware breakpoint support to ddb 2024-04-12 09:32:41 +00:00
dev powerof2: replace loops with fls or ilog2 2024-06-12 05:00:48 -05:00
dts dts: Bump the freebsd branding version to 6.8 2024-05-06 10:47:26 +02:00
fs nfscl: Add support for read delegations and atomic upgrade 2024-06-12 16:41:12 -07:00
gdb sys/gdb: Support hardware breakpoints 2024-04-12 09:32:55 +00:00
geom geom_io: Shift to pause_sbt to eliminate bogus min and update comment. 2024-05-24 08:31:55 -06:00
gnu
i386 pmap: Skip some superpage promotion attempts that will fail 2024-06-04 00:38:05 -05:00
isa new-bus: Remove the 'rid' and 'type' arguments from BUS_RELEASE_RESOURCE 2024-03-13 15:05:54 -07:00
kern subr_pctrie: add a word to a comment 2024-06-13 15:28:15 -05:00
kgssapi
libkern
modules bnxt: Use a simpler test for 32-bit platforms 2024-06-13 21:18:26 -04:00
net pf: convert DIOCBEGINADDRS to netlink 2024-06-08 04:46:43 +02:00
net80211 net80211: fix IEEE80211_FHT_BITS 2024-06-07 21:15:04 +00:00
netgraph netgraph: provide separate malloc type for nodes that are missing it 2024-05-31 09:19:54 -07:00
netinet pf: simplify pf_addrcpy() and pf_match_addr() 2024-06-06 15:45:31 +02:00
netinet6 in6_rmx: remove unnecessary socketvar.h 2024-05-07 14:15:56 -07:00
netipsec libipsec: make const-correct 2024-04-22 22:36:34 -06:00
netlink netlink: pass the correct arguments for SIOCDIFADDR and SIOCDIFADDR_IN6 2024-06-06 15:45:30 +02:00
netpfil pf: make TCP sequence number tracking less strict by one octet for FIN packets 2024-06-12 23:33:11 +02:00
netsmb
nfs
nfsclient
nfsserver
nlm
ofed ibcore: Mark write-only variables 2024-06-12 14:04:45 +01:00
opencrypto
powerpc pmap: Skip some superpage promotion attempts that will fail 2024-06-04 00:38:05 -05:00
riscv riscv: include ahci device to GENERIC. 2024-06-12 13:40:50 +01:00
rpc svc.c: Check for a non-NULL xp_socket 2024-05-27 19:22:04 -07:00
security MAC/do: allow to call setuid if real user id is 0 2024-05-23 12:09:11 +02:00
sys subr_pctrie: add leaf callbacks to pctrie_reclaim 2024-06-13 11:48:09 -05:00
teken
tests callout: retire callout_async_drain() 2024-01-24 09:33:27 -08:00
tools arm_kernel_bothdr.awk: Update to latest ota 2024-04-16 21:30:17 -06:00
ufs uio: Use switch statements when handling UIO_READ vs UIO_WRITE 2024-05-10 13:43:36 -07:00
vm vm_page: Implement lazy page initialization 2024-06-13 21:19:00 -04:00
x86 x86: simplify ceil(log2(x)) function 2024-06-04 13:00:25 -05:00
xdr
xen x86/xen: introduce non-hypercall based emergency print 2024-02-22 11:08:03 +01:00
Makefile
README.md

FreeBSD Kernel Source:

This directory contains the source files and build glue that make up the FreeBSD kernel and its modules, including both original and contributed software.

Kernel configuration files are located in the conf/ subdirectory of each architecture. GENERIC is the configuration used in release builds. NOTES contains documentation of all possible entries. LINT is a compile-only configuration used to maximize build coverage and detect regressions.

Documentation:

Source code documentation is maintained in a set of man pages, under section 9. These pages are located in share/man/man9, from the top-level of the src tree. Consult intro(9) for an overview of existing pages.

Some additional high-level documentation of the kernel is maintained in the Architecture Handbook.

Source Roadmap:

Directory Description
amd64 AMD64 (64-bit x86) architecture support
arm 32-bit ARM architecture support
arm64 64-bit ARM (AArch64) architecture support
cam Common Access Method storage subsystem - cam(4) and ctl(4)
cddl CDDL-licensed optional sources such as DTrace
conf kernel build glue
compat Linux compatibility layer, FreeBSD 32-bit compatibility
contrib 3rd-party imported software such as OpenZFS
crypto crypto drivers
ddb interactive kernel debugger - ddb(4)
fs most filesystems, excluding UFS, NFS, and ZFS
dev device drivers and other arch independent code
gdb kernel remote GDB stub - gdb(4)
geom GEOM framework - geom(4)
i386 i386 (32-bit x86) architecture support
kern main part of the kernel
libkern libc-like and other support functions for kernel use
modules kernel module infrastructure
net core networking code
net80211 wireless networking (IEEE 802.11) - net80211(4)
netgraph graph-based networking subsystem - netgraph(4)
netinet IPv4 protocol implementation - inet(4)
netinet6 IPv6 protocol implementation - inet6(4)
netipsec IPsec protocol implementation - ipsec(4)
netpfil packet filters - ipfw(4), pf(4), and ipfilter(4)
opencrypto OpenCrypto framework - crypto(7)
powerpc PowerPC/POWER (32 and 64-bit) architecture support
riscv 64-bit RISC-V architecture support
security security facilities - audit(4) and mac(4)
sys kernel headers
tests kernel unit tests
ufs Unix File System - ffs(7)
vm virtual memory system
x86 code shared by AMD64 and i386 architectures