Most of these already treat it as a proper bool, i.e. using true/false.
Also fix-up callers of OF_install().
No functional change intended.
Reviewed by: andrew, emaste
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45733
Add new SBI implementation IDs including recently allocated one for bhyve.
Reviewed by: mhorne
Sponsored by: UKRI
Differential Revision: https://reviews.freebsd.org/D45696
And from struct riscv_bootparams. It is no longer needed.
Reviewed by: br, markj
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45470
Currently we create the DMAP by mapping the entire range between the
smallest and largest physical memory addresses with L1 superpages. This
is obviously overkill, and we may end up mapping all kinds of ranges that
are not real memory.
In the case of the HiFive Unmatched (obsolete hardware), there is an
errata resulting in faults when a TLB mapping spans PMP (firmware)
protection regions. So, when our DMAP mapping spans into the memory
reserved by OpenSBI, we get a fatal fault. This highlights the need to
be smarter here.
Therefore, let's attempt to build the DMAP a little more correctly by
walking the physmap array and mapping each range individually. It is not
perfect in that we still only respect the range to a 2MB granularity,
but this could be improved in the future.
Reviewed by: markj
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45328
The overall goal of the change is to reduce the amount of work done in
locore assembly, and defer as much as possible until pmap_bootstrap().
Currently, half the setup is done in assembly, and then we pass the l1pt
address to pmap_bootstrap() where it is amended with other mappings.
Inspiration and understanding has been taken from amd64's
create_pagetables() routine, and I try to present the page table
construction in the same way: a linear procedure with commentary
explaining what we are doing and why. Thus the core of the new
implementation is contained in pmap_create_pagetables().
Once pmap_create_pagetables() has finished, we switch to the new
pagetable root and leave the bootstrap ones created by locore behind,
resulting in a minimal 8kB of wasted space.
Having the whole procedure in one place, in C code, allows it to be more
easily understood, while also making it more amenable to future changes
which depend on CPU feature/errata detection.
Note that with this change the size of the early devmap is bumped up
from one to four L2 pages (8MB).
Reviewed by: markj
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45327
This is useful for two reasons. Within this change, it allows the
early DTB mapping to be eliminated, as we can now just dereference the
physical address provided by FW and copy the DTB contents into KVA.
It will also aid in an upcoming change: the larger reworking of page
table bootstrapping on this platform.
Reviewed by: markj, jhb
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45324
The idea here is to avoid a memory access and conditional branch per
probe site. Instead, the probe is represented by an "unreachable"
unconditional function call. asm goto is used to store the address of
the probe site (represented by a no-op sled) and the address of the
function call into a tracepoint record. Each SDT probe carries a list
of tracepoints.
When the probe is enabled, the no-op sled corresponding to each
tracepoint is overwritten with a jmp to the corresponding label. The
implementation uses smp_rendezvous() to park all other CPUs while the
instruction is being overwritten, as this can't be done atomically in
general. The compiler moves argument marshalling code and the
sdt_probe() function call out-of-line, i.e., to the end of the function.
Per gallatin@ in D43504, this approach has less overhead when probes are
disabled. To make the implementation a bit simpler, I removed support
for probes with 7 arguments; nothing makes use of this except a
regression test case. It could be re-added later if need be.
The approach taken in this patch enables some more improvements:
1. We can now automatically fill out the "function" field of SDT probe
names. The SDT macros let the programmer specify the function and
module names, but this is really a bug and shouldn't have been
allowed. The intent was to be able to have the same probe in
multiple functions and to let the user restrict which probes actually
get enabled by specifying a function name or glob.
2. We can avoid branching on SDT_PROBES_ENABLED() by adding the ability
to include blocks of code in the out-of-line path. For example:
if (SDT_PROBES_ENABLED()) {
int reason = CLD_EXITED;
if (WCOREDUMP(signo))
reason = CLD_DUMPED;
else if (WIFSIGNALED(signo))
reason = CLD_KILLED;
SDT_PROBE1(proc, , , exit, reason);
}
could be written
SDT_PROBE1_EXT(proc, , , exit, reason,
int reason;
reason = CLD_EXITED;
if (WCOREDUMP(signo))
reason = CLD_DUMPED;
else if (WIFSIGNALED(signo))
reason = CLD_KILLED;
);
In the future I would like to use this mechanism more generally, e.g.,
to remove branches and marshalling code used by hwpmc, and generally to
make it easier to add new tracepoint consumers without having to add
more conditional branches to hot code paths.
Reviewed by: Domagoj Stolfa, avg
MFC after: 2 months
Differential Revision: https://reviews.freebsd.org/D44483
d0941ed fixed an edge case invloving mlock() and superpage creation
by creating and inserting a leaf pagetable page for mlock'd superpages.
However, the code does not properly release the reference to the
pagetable page in the error handling path.
This commit fixes the issue by adding calls to 'pmap_abort_ptp'
in the error handling path.
Reported by: alc
Approved by: markj (mentor)
Fixes: d0941ed
Differential Revision: https://reviews.freebsd.org/D45580
This commit moves code for releasing pagetable page references
into a separate function. No functional change intended.
Approved by: markj (mentor)
Differential Revision: https://reviews.freebsd.org/D45579
Make it explicit why we must set the trap vector before enabling virtual
memory.
Reviewed by: br, jhb, markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45474
This code path should never be hit, if it does it means we did not
bootstrap correctly. Turn it into a panic like we do on amd64 and arm64.
Reviewed by: markj, jhb
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45326
Make sure we do this BEFORE pmap_bootstrap().
Reviewed by: markj, jhb
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45325
Add basic stage 2 translation support (guest-physical to host-physical).
RISC-V hypervisor spec[1] introduces new translation schemes: Sv32x4,
Sv39x4, Sv48x4 and Sv57x4.
In each case, the size of the incoming address is widened by 2 bits (e.g.
Sv39 becomes 41-bit system).
To accommodate the 2 extra bits, the root page table (only) is expanded
by a factor of four to be 16 KiB instead of the usual 4 KiB. The rest of
page table system (including PTE format) is similar.
This gives us 4x of memory space in each scheme, but it does not make sense
to support all that memory for now.
Allocate required amount of pages for the top directory in case of stage 2,
but leave it unused.
1. https://github.com/riscv/riscv-isa-manual/blob/main/src/hypervisor.adoc
Reviewed by: mhorne
Sponsored by: UKRI
Differential Revision: https://reviews.freebsd.org/D45481
With 4-level paging enabled, the layout of KVA is identical, but we need
to step through an extra level to find the L1 table.
Reviewed by: markj
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45473
It really doesn't fit here anymore as locore is all about early startup
code. Thus, move it to its own file.
Reviewed by: br
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45320
The optional 'table' pointer is a legacy part of the interface, which
has been replaced by devmap_register_table()/devmap_add_entry(). The few
in-tree callers have already adapted to this, so it can be removed.
The 'l1pt' argument is already entirely unused within the function.
Reviewed by: andrew, markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45319
Implement a simple heuristic to skip pointless promotion attempts by
pmap_enter_quick_locked() and moea64_enter(). Specifically, when
vm_fault() calls pmap_enter_quick() to map neighboring pages at the end
of a copy-on-write fault, there is no point in attempting promotion in
pmap_enter_quick_locked() and moea64_enter(). Promotion will fail
because the base pages have differing protection.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D45431
MFC after: 1 week
DELAY takes microseconds not milliseconds, so 100 was too low. Moreover,
when enabling hw.pci.clear_pcib, PCI emeration would still stop at one
of the first bridges, but by asserting PERST for the rest of the reset
sequence that appears to be reliably addressed.
Fixes: 896e217a0e ("fu740_pci_dw: Add SiFive FU740 PCIe controller driver")
I cannot find a time where the function was not named this.
Reviewed by: kib, markj
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45383
This commit introduces the MINIDUMP_STARTUP_PAGE_TRACKING symbol and
uses it to simplify several instances of a complex preprocessor conditional
for adding pages allocated when bootstraping the kernel to minidumps.
Reviewed by: markj, mhorne
Approved by: markj (mentor)
Differential Revision: https://reviews.freebsd.org/D45085
This commit refactors the UMA small alloc code and
removes most UMA machine-dependent code.
The existing machine-dependent uma_small_alloc code is almost identical
across all architectures, except for powerpc where using the direct
map addresses involved extra steps in some cases.
The MI/MD split was replaced by a default uma_small_alloc
implementation that can be overridden by architecture-specific code by
defining the UMA_MD_SMALL_ALLOC symbol. Furthermore, UMA_USE_DMAP was
introduced to replace most UMA_MD_SMALL_ALLOC uses.
Reviewed by: markj, kib
Approved by: markj (mentor)
Differential Revision: https://reviews.freebsd.org/D45084
It is inherited from arm, where the global exists and is used. No
functional change.
Reviewed by: markj
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45323
No functional change.
Reviewed by: markj
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45322
Implement atomic_load_acq_16, atomic_store_rel_16.
These are needed by bhyve(8) PCIe bus emulation code.
Group 16-bit atomic functions similarly to 32 and 64-bit.
Reviewed by: mhorne
Differential Revision: https://reviews.freebsd.org/D45228
From the SSTC spec:
"If the stimecmp (supervisor-mode timer compare) register is implemented,
then STIP is read-only in mip and reflects the supervisor-level timer
interrupt signal resulting from stimecmp. This timer interrupt signal
is cleared by writing stimecmp with a value greater than the current time
value."
This fixes operation in Spike with sstc extension enabled.
Example:
spike --isa RV64IMAFDCH_zicntr_zihpm_sstc
Reviewed by: mhorne
Differential Revision: https://reviews.freebsd.org/D45226
Per the Privilege Spec, the Accessed (A) or Dirty (D) bits must only be
set for a leaf PTE.
It seems newer versions of QEMU have started to enforce this
requirement, and without this change, pmap_bootstrap() hangs when
switching to Sv48 mode.
Reviewed by: jrtc27, markj
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45210
This is mostly to reduce the diff with CheriBSD which adds additional
constants to enum uio_rw, but also matches the normal style used for
uio_segflg.
Reviewed by: kib, emaste
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D45142
Implement a core clknode driver for the JH7110 (StarFive VisionFive v2)
platform.
Add clock/reset generator drivers for the PLL, SYS, and AON clock
groupings.
Co-authored-by: mhorne
Reviewed by: mhorne
Sponsored by: The FreeBSD Foundation (mhorne's contributions)
Differential Revision: https://reviews.freebsd.org/D43037
Add a variant of the existing dwmmc driver, and enable it in the GENERIC
kernel.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D44026
Implement a core clknode driver for the JH7110 (StarFive VisionFive v2)
platform.
Add clock/reset generator drivers for the PLL, SYS, and AON clock
groupings.
Co-authored-by: mhorne
Reviewed by: mhorne
Sponsored by: The FreeBSD Foundation (mhorne's contributions)
Differential Revision: https://reviews.freebsd.org/D43037
It serves the purpose of attaching syscon devices as early as possible;
this is required for early attachment of the PLL clock driver.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D44270
Typically, when a DMA transaction requires bouncing, we will break up
the request into segments that are, at maximum, page-sized.
However, in the atypical case of a driver whose maximum segment size is
smaller than PAGE_SIZE, we end up inefficiently assigning each segment
its own bounce page. For example, the dwmmc driver has a maximum segment
size of 2048 (PAGE_SIZE / 2); a 4-page transfer ends up requiring 8
bounce pages in the current scheme.
We should attempt to batch segments into bounce pages more efficiently.
This is achieved by pushing all considerations of the maximum segment
size into the new _bus_dmamap_addsegs() function, which wraps
_bus_dmamap_addseg(). Thus we allocate the minimal number of bounce
pages required to complete the entire transfer, while still performing
the transfer with smaller-sized transactions.
For most drivers with a segment size >= PAGE_SIZE, this will have no
impact. For drivers like dwmmc mentioned above, this improves the memory
and performance efficiency when bouncing a large transfer.
Co-authored-by: jhb
Reviewed by: jhb
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45048
It is functionally identical in all implementations, so move the
function to subr_busdma_bounce.c. The KASSERT present in the x86 version
is now enabled for all architectures. It should be universally
applicable.
Reviewed by: jhb
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45047
Since config(8) searches sys/conf by default, there's no need to specify
the full relative path here; replace it by the filename alone.
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1124
The new sys/conf/std.debug contains the list of debugging options
enabled by default in -CURRENT, so they don't need to be listed
individually in every kernel config.
The enabled options are the set of all debug options which were enabled
for the GENERIC kernel on any platform. This means some architectures
now have debugging options enabled in GENERIC which weren't previously
enabled:
- amd64: [1]
- arm64: [2]
- arm: [2]. [3]
- i386: [1], [2]
- powerpc: [1], [2], [3]
- riscv: [2]
[1] ALT_BREAK_TO_DEBUGGER is now enabled.
[2] BUF_TRACKING, FULL_BUF_TRACKING, and QUEUE_MACRO_DEBUG_TRASH are now
enabled.
[3] DEADLKRES is now enabled.
While here, move the documentation for the (commented out) K*SAN options
for amd64 from GENERIC to NOTES.
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1124
Define PTE_TO_VM_PAGE to compose the PHYS_TO_VM_PAGE and PTE_TO_PHYS
macros. Use it where appropriate, and drop some variables that it
makes unnecessary.
Reviewed by: jhb (previous version)
Differential Revision: https://reviews.freebsd.org/D44700
While here, adjust the sample setting for NVME_USE_NVD to use a
non-default setting as is typical in entries in NOTES.
Discussed with: imp
Reviewed by: manu
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D44691
The public bus_activate/deactivate_resource() API still accepts both
forms, but the internal kobj methods no longer pass the arguments.
Implementations which need the rid or type now use rman_get_rid() or
rman_get_type() to fetch the value from the allocated resource.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D44130
The public bus_map/unmap_resource() API still accepts both forms, but
the internal kobj methods no longer pass the argument.
Implementations which need the type now use rman_get_type() to fetch
the value from the allocated resource.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D44129
On this platform early console access is possible via SBI. Follow recent
changes to EARLY_PRINTF option and give it a named constant.
Update the commented option in GENERIC so that it compiles.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D44100
Enable the Synopsis UART driver. Other drivers will be added in the
future.
Reviewed by: jrtc27
Sponsored by: The FreeBSD Foundatino
Differential Revision: https://reviews.freebsd.org/D44105
For small segments (< PAGE_SIZE) there is a mismatch between how
required bounce pages are counted in _bus_dmamap_count_pages() and
bounce_bus_dmamap_load_buffer().
This problem has been observed on the RISC-V VisionFive v2 SoC (and
earlier revisions of the hardware) which has memory physically addressed
above 4GB. This requires some bouncing for the dwmmc driver, which has
has a maximum segment size of 2048 bytes. When attempting to load a
page-aligned 4-page buffer that requires bouncing, we can end up
counting 4 bounce pages for an 8-segment transfer. These pages will be
incorrectly configured to cover only the first half of the transfer (4 x
2048 bytes).
Fix the immediate issue by adding the maxsegsz check to
_bus_dmamap_count_pages(); this is what _bus_dmamap_count_phys() does
already. The result is that we will inefficiently allocate a separate
bounce page for each segment (8 pages for the example above), but the
transfer will proceed in its entirety.
The more complete fix is to address the shortcomings in how small
segments are assigned to bounce pages, so that we opportunistically
batch multiple segments to a page whenever they fit (e.g. two 2048 bytes
segments per 4096 page). This will be addressed more holistically in the
future. For now this change will prevent the (silent) incomplete
transfers that have been observed.
PR: 273694
Reported by: Jari Sihvola <jsihv@gmx.com>
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34118
The #ifdef DDB code in parse_metadata was dead code without opt_ddb.h.
While here, update the call to db_fetch_ksymtab for changes in commit
02bc014a20.
Reviewed by: mhorne
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D43919
This patch introduces support for the RISC-V APLIC interrupt controller
[1]. Currently, it is only supports direct mode, i.e. without an IMSIC
and functionally replacing the legacy RISC-V PLIC. Work on IMSIC support
is in progress.
[1] https://github.com/riscv/riscv-aia/releases/tag/1.0
Reviewed by: mhorne
Discussed with: jrtc27
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D43293
