When a vcpu sees that a rendezvous is in progress, it exits and tries to
handle the rendezvous. The vcpu doesn't check if it's part of the
rendezvous or not. If the vcpu isn't part of the rendezvous, the
rendezvous could be done before it reaches the assertion. This will
cause a panic.
The assertion isn't needed at all because vm_handle_rendezvous properly
handles a spurious rendezvous. So, we can just remove it.
PR: 267779
Reviewed by: jhb, markj
Tested by: bz
Approved by: manu (mentor)
MFC after: 1 week
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D37417
32-bit powerpc doesn't have a fuword64, so only use it on powerpc64.
This could also be done elsewhere that splits 32-bit and 64-bit
accesses, but adding ifdefs everywhere would just clutter up the source
for little benefit. This is the only usage of fuword64 directly; other
archs have a dtrace_fuword64_nocheck(), but powerpc does not, and I
don't see a need to add it currently.
MFC after: 1 week
of various keyboard drivers.
EVIOCGRAB ioctl execution on /dev/input/event# device node gains
exclusive access to this device to caller. It is used mostly for
development purposes and remote control software. See e.g.
https://reviews.freebsd.org/D30020 which is the reason of creation
of this change.
Keyboard grabbing is disabled in KDB and during panics.
MFC with: 4a0db5e292
Tested by: corvink
Differential revision: https://reviews.freebsd.org/D30542
The default is now the number of physical CPUs in the system rather
than 16.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37175
Convert the vcpu[] array in struct vm to an array of pointers and
allocate vCPUs on first use. This avoids always allocating VM_MAXCPU
vCPUs for each VM, but instead only allocates the vCPUs in use. A new
per-VM sx lock is added to serialize attempts to allocate vCPUs on
first use. However, a given vCPU is never freed while the VM is
active, so the pointer is read via an unlocked read first to avoid the
need for the lock in the common case once the vCPU has been created.
Some ioctls need to lock all vCPUs. To prevent races with ioctls that
want to allocate a new vCPU, these ioctls also lock the sx lock that
protects vCPU creation.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37174
Retire the boot_state member of struct vlapic and instead use a cpuset
in the VM to track vCPUs waiting for STARTUP IPIs. INIT IPIs add
vCPUs to this set, and STARTUP IPIs remove vCPUs from the set.
STARTUP IPIs are only reported to userland for vCPUs that were removed
from the set.
In particular, this permits a subsequent change to allocate vCPUs on
demand when the vCPU may not be allocated until after a STARTUP IPI is
reported to userland.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37173
Previously bhyve obtained a "read lock" on the memory map for ioctls
needing to read the map by locking the last vCPU. This is now
replaced by a new per-VM sx lock. Modifying the map requires
exclusively locking the sx lock as well as locking all existing vCPUs.
Reading the map requires either locking one vCPU or the sx lock.
This permits safely modifying or querying the memory map while some
vCPUs do not exist which will be true in a future commit.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37172
Compared to the previous version this does mean that if the system as
a whole runs out of dedicated vPIDs you might end up with some vCPUs
within a single VM using dedicated vPIDs and others using shared
vPIDs, but this should not break anything.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37169
Centralize mapping vCPU IDs to struct vcpu objects in vmmdev_ioctl and
pass vcpu pointers to the routines in vmm.c. For operations that want
to perform an action on all vCPUs or on a single vCPU, pass pointers
to both the VM and the vCPU using a NULL vCPU pointer to request
global actions.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37168
No I/O ports are vCPU-specific (unlike memory which does have
vCPU-specific ranges such as the local APIC).
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37163
This passes struct vcpu down in place of struct vm and and integer
vcpu index through the in-kernel instruction emulation code. To
minimize userland disruption, helper macros are used for the vCPU
arguments passed into and through the shared instruction emulation
code.
A few other APIs used by the instruction emulation code have also been
updated to accept struct vcpu in the kernel including
vm_get/set_register and vm_inject_fault.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37161
This handles the case that guest pages are being held not on behalf of
a virtual CPU but globally. Previously this was handled by passing a
vcpuid of -1 to vm_gpa_hold, but that will not work in the future when
vm_gpa_hold is changed to accept a struct vcpu pointer.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37160
The arguments identifying the VM and vCPU are only needed for
vm_copy_setup.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37158
Pass a pointer to the current struct vcpu to the vcpu_init callback
and save this pointer in the CPU-specific vcpu structures.
Add routines to fetch a struct vcpu by index from a VM and to query
the VM and vcpuid from a struct vcpu.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37156
These macros are similar to VCPU_CTR* but accept a single vmx_vcpu
pointer as the first argument instead of separate vm and vcpuid.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37154
These macros are similar to VCPU_CTR* but accept a single svm_vcpu
pointer as the first argument instead of separate vm and vcpuid.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37153
This requires storing a reference to the per-vm cookie in the
CPU-specific vCPU structure. Take advantage of this new field to
remove no-longer-needed function arguments in the CPU-specific
backends. In particular, stop passing the per-vm cookie to functions
that either don't use it or only use it for KTR traces.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37152
Rather than storing static arrays of per-vCPU data in the CPU-specific
per-VM structure, adopt a more dynamic model similar to that used to
manage CPU-specific per-VM data.
That is, add new vmmops methods to init and cleanup a single vCPU.
The init method returns a pointer that is stored in 'struct vcpu' as a
cookie pointer. This cookie pointer is now passed to other vmmops
callbacks in place of the integer index. The index is now only used
in KTR traces and when calling back into the CPU-independent layer.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37151
A future commit will remove direct access to vCPU structures from
struct vmx, so add a dedicated boolean for this rather than checking
the capabilities for vCPU 0.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37269
Previously some per-vCPU state was saved in vmmops_snapshot and other
state was saved in vmmops_vcmx_snapshot. Consolidate all per-vCPU
state into the latter routine and rename the hook to the more generic
'vcpu_snapshot'. Note that the CPU-independent per-vCPU data is still
stored in a separate blob as well as the per-vCPU local APIC data.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37150
Mark Johnston noticed that this was missing VMCB_CACHE_LBR. Just set
all the bits as is done in svm_run() rather than trying to clear
individual bits.
Reported by: markj
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37259
Add a struct vmx_vcpu to hold per-vCPU data specific to VT-x and
move parallel arrays out of struct vmx into a single array of
this structure.
While here, dynamically allocate the VMCS, APIC page and PIR
descriptors for each vCPU rather than embedding them.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37149
- Allocate VMCBs separately to avoid excessive padding in struct
svm_vcpu.
- Allocate APIC pages dynamically directly in struct vlapic.
- Move vm_mtrr into struct svm_vcpu rather than using a separate
parallel array.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37148
Mostly these are loops that iterate over all possible vCPU IDs for a
specific virtual machine.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37147
Read the current "now" TSC value and use it to compute absolute time
saved value in vm_snapshot_vcpus rather than iterating over vCPUs
multiple times in vm_snapshot_vm.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37146
When PV_STATS is defined, freed is used. Otherwise it isn't. Mark it as
__pvused and define __pvused appropriately.
Sponsored by: Netflix
Reviewed by: tsoome, rpokala, andrew
Differential Revision: https://reviews.freebsd.org/D37438
Further updates adding casts to avoid 32-bit multiplication overflow
inspired by fixes in commit 017367c114.
No legitimate superblocks should fail as a result of these changes.
Sponsored by: The FreeBSD Foundation
of psm(4), ums(4) and sysmouse(4) drivers.
EVIOCGRAB ioctl execution on /dev/input/event# device node gains
exclusive access to this device to caller. It is used mostly for
development purposes and remote control software. See e.g.
https://reviews.freebsd.org/D30020 which is the reason of creation
of this change.
MFC after: 2 weeks
Tested by: corvink
Differential revision: https://reviews.freebsd.org/D30542
As with amd64 pmap introduce per-superpage locks backed by pages
allocated by their respective domains.
This significiantly reduces lock contantion from pmap when running
poudriere on a 160 core Ampere Altra server
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D36310
There is nothing wrong with declaring an array as CTLTYPE_U64. Note
that sysctl(8) is able to print such arrays and doesn't need any change.
Fixes: b5b7b142a7
Since ZFS reports _PC_MIN_HOLE_SIZE as 512 (although it
appears that an unwritten region must be at least f_iosize
to remain unallocated), vn_generic_copy_file_range()
uses 4096 for the copy blksize for ZFS, reulting in slow copies.
For most other file systems, _PC_MIN_HOLE_SIZE and f_iosize
are the same value, so this patch modifies the code to
use f_iosize for most cases. It also documents in comments
why the blksize is being set a certain way, so that the code
does not appear to be doing "magic math".
Reported by: allanjude
Reviewed by: allanjude, asomers
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D37076
The import of OpenZFS PR 13758 causes a panic in zfsctl_is_node()
if ZFS is mounting as root filesystem. This implements a workaround
until the issue is resolved by authors.
