Add basic stage 2 translation support (guest-physical to host-physical).
RISC-V hypervisor spec[1] introduces new translation schemes: Sv32x4,
Sv39x4, Sv48x4 and Sv57x4.
In each case, the size of the incoming address is widened by 2 bits (e.g.
Sv39 becomes 41-bit system).
To accommodate the 2 extra bits, the root page table (only) is expanded
by a factor of four to be 16 KiB instead of the usual 4 KiB. The rest of
page table system (including PTE format) is similar.
This gives us 4x of memory space in each scheme, but it does not make sense
to support all that memory for now.
Allocate required amount of pages for the top directory in case of stage 2,
but leave it unused.
1. https://github.com/riscv/riscv-isa-manual/blob/main/src/hypervisor.adoc
Reviewed by: mhorne
Sponsored by: UKRI
Differential Revision: https://reviews.freebsd.org/D45481
With 4-level paging enabled, the layout of KVA is identical, but we need
to step through an extra level to find the L1 table.
Reviewed by: markj
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45473
It really doesn't fit here anymore as locore is all about early startup
code. Thus, move it to its own file.
Reviewed by: br
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45320
The optional 'table' pointer is a legacy part of the interface, which
has been replaced by devmap_register_table()/devmap_add_entry(). The few
in-tree callers have already adapted to this, so it can be removed.
The 'l1pt' argument is already entirely unused within the function.
Reviewed by: andrew, markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45319
Implement a simple heuristic to skip pointless promotion attempts by
pmap_enter_quick_locked() and moea64_enter(). Specifically, when
vm_fault() calls pmap_enter_quick() to map neighboring pages at the end
of a copy-on-write fault, there is no point in attempting promotion in
pmap_enter_quick_locked() and moea64_enter(). Promotion will fail
because the base pages have differing protection.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D45431
MFC after: 1 week
DELAY takes microseconds not milliseconds, so 100 was too low. Moreover,
when enabling hw.pci.clear_pcib, PCI emeration would still stop at one
of the first bridges, but by asserting PERST for the rest of the reset
sequence that appears to be reliably addressed.
Fixes: 896e217a0e ("fu740_pci_dw: Add SiFive FU740 PCIe controller driver")
I cannot find a time where the function was not named this.
Reviewed by: kib, markj
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45383
This commit introduces the MINIDUMP_STARTUP_PAGE_TRACKING symbol and
uses it to simplify several instances of a complex preprocessor conditional
for adding pages allocated when bootstraping the kernel to minidumps.
Reviewed by: markj, mhorne
Approved by: markj (mentor)
Differential Revision: https://reviews.freebsd.org/D45085
This commit refactors the UMA small alloc code and
removes most UMA machine-dependent code.
The existing machine-dependent uma_small_alloc code is almost identical
across all architectures, except for powerpc where using the direct
map addresses involved extra steps in some cases.
The MI/MD split was replaced by a default uma_small_alloc
implementation that can be overridden by architecture-specific code by
defining the UMA_MD_SMALL_ALLOC symbol. Furthermore, UMA_USE_DMAP was
introduced to replace most UMA_MD_SMALL_ALLOC uses.
Reviewed by: markj, kib
Approved by: markj (mentor)
Differential Revision: https://reviews.freebsd.org/D45084
It is inherited from arm, where the global exists and is used. No
functional change.
Reviewed by: markj
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45323
No functional change.
Reviewed by: markj
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45322
Implement atomic_load_acq_16, atomic_store_rel_16.
These are needed by bhyve(8) PCIe bus emulation code.
Group 16-bit atomic functions similarly to 32 and 64-bit.
Reviewed by: mhorne
Differential Revision: https://reviews.freebsd.org/D45228
From the SSTC spec:
"If the stimecmp (supervisor-mode timer compare) register is implemented,
then STIP is read-only in mip and reflects the supervisor-level timer
interrupt signal resulting from stimecmp. This timer interrupt signal
is cleared by writing stimecmp with a value greater than the current time
value."
This fixes operation in Spike with sstc extension enabled.
Example:
spike --isa RV64IMAFDCH_zicntr_zihpm_sstc
Reviewed by: mhorne
Differential Revision: https://reviews.freebsd.org/D45226
Per the Privilege Spec, the Accessed (A) or Dirty (D) bits must only be
set for a leaf PTE.
It seems newer versions of QEMU have started to enforce this
requirement, and without this change, pmap_bootstrap() hangs when
switching to Sv48 mode.
Reviewed by: jrtc27, markj
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45210
This is mostly to reduce the diff with CheriBSD which adds additional
constants to enum uio_rw, but also matches the normal style used for
uio_segflg.
Reviewed by: kib, emaste
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D45142
Implement a core clknode driver for the JH7110 (StarFive VisionFive v2)
platform.
Add clock/reset generator drivers for the PLL, SYS, and AON clock
groupings.
Co-authored-by: mhorne
Reviewed by: mhorne
Sponsored by: The FreeBSD Foundation (mhorne's contributions)
Differential Revision: https://reviews.freebsd.org/D43037
Add a variant of the existing dwmmc driver, and enable it in the GENERIC
kernel.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D44026
Implement a core clknode driver for the JH7110 (StarFive VisionFive v2)
platform.
Add clock/reset generator drivers for the PLL, SYS, and AON clock
groupings.
Co-authored-by: mhorne
Reviewed by: mhorne
Sponsored by: The FreeBSD Foundation (mhorne's contributions)
Differential Revision: https://reviews.freebsd.org/D43037
It serves the purpose of attaching syscon devices as early as possible;
this is required for early attachment of the PLL clock driver.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D44270
Typically, when a DMA transaction requires bouncing, we will break up
the request into segments that are, at maximum, page-sized.
However, in the atypical case of a driver whose maximum segment size is
smaller than PAGE_SIZE, we end up inefficiently assigning each segment
its own bounce page. For example, the dwmmc driver has a maximum segment
size of 2048 (PAGE_SIZE / 2); a 4-page transfer ends up requiring 8
bounce pages in the current scheme.
We should attempt to batch segments into bounce pages more efficiently.
This is achieved by pushing all considerations of the maximum segment
size into the new _bus_dmamap_addsegs() function, which wraps
_bus_dmamap_addseg(). Thus we allocate the minimal number of bounce
pages required to complete the entire transfer, while still performing
the transfer with smaller-sized transactions.
For most drivers with a segment size >= PAGE_SIZE, this will have no
impact. For drivers like dwmmc mentioned above, this improves the memory
and performance efficiency when bouncing a large transfer.
Co-authored-by: jhb
Reviewed by: jhb
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45048
It is functionally identical in all implementations, so move the
function to subr_busdma_bounce.c. The KASSERT present in the x86 version
is now enabled for all architectures. It should be universally
applicable.
Reviewed by: jhb
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45047
Since config(8) searches sys/conf by default, there's no need to specify
the full relative path here; replace it by the filename alone.
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1124
The new sys/conf/std.debug contains the list of debugging options
enabled by default in -CURRENT, so they don't need to be listed
individually in every kernel config.
The enabled options are the set of all debug options which were enabled
for the GENERIC kernel on any platform. This means some architectures
now have debugging options enabled in GENERIC which weren't previously
enabled:
- amd64: [1]
- arm64: [2]
- arm: [2]. [3]
- i386: [1], [2]
- powerpc: [1], [2], [3]
- riscv: [2]
[1] ALT_BREAK_TO_DEBUGGER is now enabled.
[2] BUF_TRACKING, FULL_BUF_TRACKING, and QUEUE_MACRO_DEBUG_TRASH are now
enabled.
[3] DEADLKRES is now enabled.
While here, move the documentation for the (commented out) K*SAN options
for amd64 from GENERIC to NOTES.
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1124
Define PTE_TO_VM_PAGE to compose the PHYS_TO_VM_PAGE and PTE_TO_PHYS
macros. Use it where appropriate, and drop some variables that it
makes unnecessary.
Reviewed by: jhb (previous version)
Differential Revision: https://reviews.freebsd.org/D44700
While here, adjust the sample setting for NVME_USE_NVD to use a
non-default setting as is typical in entries in NOTES.
Discussed with: imp
Reviewed by: manu
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D44691
The public bus_activate/deactivate_resource() API still accepts both
forms, but the internal kobj methods no longer pass the arguments.
Implementations which need the rid or type now use rman_get_rid() or
rman_get_type() to fetch the value from the allocated resource.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D44130
The public bus_map/unmap_resource() API still accepts both forms, but
the internal kobj methods no longer pass the argument.
Implementations which need the type now use rman_get_type() to fetch
the value from the allocated resource.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D44129
On this platform early console access is possible via SBI. Follow recent
changes to EARLY_PRINTF option and give it a named constant.
Update the commented option in GENERIC so that it compiles.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D44100
Enable the Synopsis UART driver. Other drivers will be added in the
future.
Reviewed by: jrtc27
Sponsored by: The FreeBSD Foundatino
Differential Revision: https://reviews.freebsd.org/D44105
For small segments (< PAGE_SIZE) there is a mismatch between how
required bounce pages are counted in _bus_dmamap_count_pages() and
bounce_bus_dmamap_load_buffer().
This problem has been observed on the RISC-V VisionFive v2 SoC (and
earlier revisions of the hardware) which has memory physically addressed
above 4GB. This requires some bouncing for the dwmmc driver, which has
has a maximum segment size of 2048 bytes. When attempting to load a
page-aligned 4-page buffer that requires bouncing, we can end up
counting 4 bounce pages for an 8-segment transfer. These pages will be
incorrectly configured to cover only the first half of the transfer (4 x
2048 bytes).
Fix the immediate issue by adding the maxsegsz check to
_bus_dmamap_count_pages(); this is what _bus_dmamap_count_phys() does
already. The result is that we will inefficiently allocate a separate
bounce page for each segment (8 pages for the example above), but the
transfer will proceed in its entirety.
The more complete fix is to address the shortcomings in how small
segments are assigned to bounce pages, so that we opportunistically
batch multiple segments to a page whenever they fit (e.g. two 2048 bytes
segments per 4096 page). This will be addressed more holistically in the
future. For now this change will prevent the (silent) incomplete
transfers that have been observed.
PR: 273694
Reported by: Jari Sihvola <jsihv@gmx.com>
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34118
The #ifdef DDB code in parse_metadata was dead code without opt_ddb.h.
While here, update the call to db_fetch_ksymtab for changes in commit
02bc014a20.
Reviewed by: mhorne
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D43919
This patch introduces support for the RISC-V APLIC interrupt controller
[1]. Currently, it is only supports direct mode, i.e. without an IMSIC
and functionally replacing the legacy RISC-V PLIC. Work on IMSIC support
is in progress.
[1] https://github.com/riscv/riscv-aia/releases/tag/1.0
Reviewed by: mhorne
Discussed with: jrtc27
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D43293
This is a repeat of 63bf2d735c ("Remove the unused arm64_cpu driver.")
for RISC-V, which copied the defunct code from arm64 with no changes
beyond substituting riscv64 for arm64, and made no use of it elsewhere.
It has thus always been entirely superfluous.
Reviewed by: mhorne
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D43672
Currently the local interrupt controller implementation is based on
pre-INTRNG arm/arm64 code, using hand-rolled event code rather than
INTRNG. This then interacts weirdly with the PLIC, and other future
interrupt controllers like the APLIC and IMSICs in the upcoming AIA
specification, since they become the root PIC despite not being the
logical root. Instead, use a real newbus device for it and register
it as the root PIC.
This also adapts the IPI code to make use of the newly-added INTRNG
generic IPI handling framework, adding a new sbi_ipi as the PIC. In
future there will be alternative devices for sending IPIs that will
register with higher priorities, such as the proposed AIA IMSIC and
ACLINT SSWI.
Reviewed by: mhorne
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D35901
This approach is based on the Arm PSCI driver, though that makes more
extensive use of its softc than we do here. This will be used to extract
the SBI IPI code as a real PIC.
Reviewed by: mhorne, imp
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D35900
The static irq_root_ipicount variable is only ever written to (with the
value passed to irq_root_ipicount), never read. Moreover, the bcm2836
driver, as used by the Raspberry Pi 2B and 3A/B (but not 4, which uses a
GIC-400, though does have the legacy interrupt controller present too)
passes 0 as ipicount, despite implementing IPIs. It's thus inaccurate
and serves no purpose, so should be removed.
Reviewed by: mmel, imp, mhorne
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D35897
_Alignof(expression) is a non-standard extension. This is not allowed
in gnu11 and gnu17 which follow the C11 standard _Alignof(type).
Reviewed by: arichardson, imp, jhb
Fixes: 4a9cd9fc22 amd64 db_trace: Reject unaligned frame pointers
Fixes: 7ccaf76a27 riscv db_trace: Ensure trapframe pointer is suitably aligned.
Fixes: 638c68897f arm64 db_trace: Ensure trapframe pointer is suitably aligned.
Differential Revision: https://reviews.freebsd.org/D43409
We've removed kernel option EXT_RESOURCES almost two years ago.
While it was ok to have some code under a common 'extres' subdirectory
at first, we now have a lot of consumer of it and we made it mandatory
so no need to have it under a cryptic name.
Reviewed by: mhorne
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D43196
We've removed kernel option EXT_RESOURCES almost two years ago.
While it was ok to have some code under a common 'extres' subdirectory
at first, we now have a lot of consumer of it and we made it mandatory
so no need to have it under a cryptic name.
Reviewed by: imp
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D43192
We've removed kernel option EXT_RESOURCES almost two years ago.
While it was ok to have some code under a common 'extres' subdirectory
at first, we now have a lot of consumer of it and we made it mandatory
so no need to have it under a cryptic name.
Reviewed by: mhorne
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D43191
Some implementations copy data to userspace, an operation which can in
principle fail. In preparation for adding a __result_use_check
annotation to copyin() and related functions, let implementations of
cpu_set_upcall() return an error, and check for errors when copying data
to user memory.
Reviewed by: kib, jhb
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D43100
After removing filter functionality, the naming doesn't clearly
represent what the function does, so try to address this. Include some
code clarity and style improvements.
Create a common version in subr_busdma_bounce.c, used by most
implementations. powerpc still needs its own version of the function,
due to its dmat->iommu == NULL check.
No functional change intended.
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D42896
Without filter functions, we do not need to keep track of tag ancestry.
All inheritance of the parent tag's parameters occurs when creating the
new child tag.
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D42895
Address filter functions are unused, unsupported, and now rejected.
Simplify some busdma code by removing filter functionality completely.
Note that the chains of parent tags become useless, and will be cleaned
up in the next commit.
No functional change intended.
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D42894
A deprecation notice was added to the bus_dma(9) man page by scottl@ in
September 2020 discouraging the use of filter functions. I've performed
an attentive check of all callers in the tree and everything that exists
today passes NULL for both filtfunc and filtarg. Thus, we should start
returning EINVAL if these arguments are non-NULL to prevent new usages
from popping up. Update the man page to be more clear about this.
The deprecation notice is present since at least 13.0-RELEASE, so this
is the appropriate step for the lifetime of 15, without actually
breaking the driver API. Stable branches will emit a warning instead.
This change enables the removal of a fair amount of unused complexity
across the various busdma implementations.
Reviewed by: jhb
MFC after: never
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D42852
Enable phy and regulator extres devices. These aren't needed for
existing SoC support, but are of general utility to FDT platforms and
enable out-of-tree work.
Similarly, enable sdhci and mmc.
Reviewed by: jrtc27
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D42913
