system/freebsd-src
system/freebsd-src
1
0
Fork 0
mirror of https://github.com/freebsd/freebsd-src synced 2024-09-06 17:18:32 +00:00
Code Issues Packages Projects Releases Wiki Activity

x86: handle domains with no CPUs usable for intr delivery

Browse source 
We can end up with a domain having no CPUs capable of receiving I/O
interrupts.  This can occur, for example, when all APIC IDs in a given
domain are 256 or greater, and we have no IOMMU.

In this case disable per-domain interrupt support, effectively reverting
to the behaviour before commit a48de40bcc ("Only use CPUs in the
domain the device is attached to for default").  This has a performance
impact but at least allows the system to be functional.  It is a stop-
gap until we can rely on the presence of an IOMMU on all x86 platforms.

Thanks to AMD for providing the high-thread-count machine I used for
testing this change, and to cperciva for testing on other hardware.

Reviewed by:	jhb
Tested by:	cperciva, emaste
Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D41501
This commit is contained in:
Ed Maste 2023-08-17 23:29:33 -04:00
parent b404e03007
commit 4258eb5a0d
1 changed files with 19 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

20
sys/x86/x86/intr_machdep.c
View file

@ -578,10 +578,17 @@ DB_SHOW_COMMAND(irqs, db_show_irqs)
/*
* Support for balancing interrupt sources across CPUs. For now we just
* allocate CPUs round-robin.
*
* XXX If the system has a domain with without any usable CPUs (e.g., where all
* APIC IDs are 256 or greater and we do not have an IOMMU) we use
* intr_no_domain to fall back to assigning interrupts without regard for
* domain. Once we can rely on the presence of an IOMMU on all x86 platforms
* we can revert this.
*/
cpuset_t intr_cpus = CPUSET_T_INITIALIZER(0x1);
static int current_cpu[MAXMEMDOM];
static bool intr_no_domain;
static void
intr_init_cpus(void)
@ -589,7 +596,15 @@ intr_init_cpus(void)
int i;
for (i = 0; i < vm_ndomains; i++) {
if (CPU_OVERLAP(&cpuset_domain[i], &intr_cpus) == 0) {
intr_no_domain = true;
printf("%s: unable to route interrupts to CPUs in domain %d\n",
__func__, i);
}
current_cpu[i] = 0;
if (intr_no_domain && i > 0)
continue;
if (!CPU_ISSET(current_cpu[i], &intr_cpus) ||
!CPU_ISSET(current_cpu[i], &cpuset_domain[i]))
intr_next_cpu(i);
@ -615,6 +630,8 @@ intr_next_cpu(int domain)
return (PCPU_GET(apic_id));
#endif
if (intr_no_domain)
domain = 0;
mtx_lock_spin(&icu_lock);
apic_id = cpu_apic_ids[current_cpu[domain]];
do {
@ -622,7 +639,8 @@ intr_next_cpu(int domain)
if (current_cpu[domain] > mp_maxid)
current_cpu[domain] = 0;
} while (!CPU_ISSET(current_cpu[domain], &intr_cpus) ||
!CPU_ISSET(current_cpu[domain], &cpuset_domain[domain]));
(!CPU_ISSET(current_cpu[domain], &cpuset_domain[domain]) &&
!intr_no_domain));
mtx_unlock_spin(&icu_lock);
return (apic_id);
}