linux/Documentation/x86/topology.rst
Linus Torvalds 1fb3b526df Some late arriving documentation changes. In particular, this contains the
conversion of the x86 docs to RST, which has been in the works for some
 time but needed a couple of final tweaks.
 -----BEGIN PGP SIGNATURE-----
 
 iQFDBAABCAAtFiEEIw+MvkEiF49krdp9F0NaE2wMflgFAlzVlVoPHGNvcmJldEBs
 d24ubmV0AAoJEBdDWhNsDH5YPWgH/1z+HO4QiLZ72kVxLf2U5r6FAo4CtQYLymL/
 GiDabC7Jt7hobXdFQmDXhFnLOR/ibMnawJw2JAgWXDo33KenKGbE2OiW8ecsebSb
 hd1F3pU6P3gVTYItcuM8dZ6/0C/F98/J/O3O3sOhZ0Uup2WPxW5XdNOp7LjFQScc
 ENkgm2C5trs1wGjVswXWztGxSTcYrF7ehhjpWsFr9MUnUOI6ghvXX1akN3cEo7eo
 7D8nvG2/HWOkf9Oq87/1uQxF6lERRqOQE+HN1J80XUsNTV5Hn40RP40FeebVv1rr
 1GjUu+mKk/5uV+OlRWFqLbt10cU4+TKKfNTqfEchHyDOMpJD+S0=
 =hfly
 -----END PGP SIGNATURE-----

Merge tag 'docs-5.2a' of git://git.lwn.net/linux

Pull more documentation updates from Jonathan Corbet:
 "Some late arriving documentation changes. In particular, this contains
  the conversion of the x86 docs to RST, which has been in the works for
  some time but needed a couple of final tweaks"

* tag 'docs-5.2a' of git://git.lwn.net/linux: (29 commits)
  Documentation: x86: convert x86_64/machinecheck to reST
  Documentation: x86: convert x86_64/cpu-hotplug-spec to reST
  Documentation: x86: convert x86_64/fake-numa-for-cpusets to reST
  Documentation: x86: convert x86_64/5level-paging.txt to reST
  Documentation: x86: convert x86_64/mm.txt to reST
  Documentation: x86: convert x86_64/uefi.txt to reST
  Documentation: x86: convert x86_64/boot-options.txt to reST
  Documentation: x86: convert i386/IO-APIC.txt to reST
  Documentation: x86: convert usb-legacy-support.txt to reST
  Documentation: x86: convert orc-unwinder.txt to reST
  Documentation: x86: convert resctrl_ui.txt to reST
  Documentation: x86: convert microcode.txt to reST
  Documentation: x86: convert pti.txt to reST
  Documentation: x86: convert amd-memory-encryption.txt to reST
  Documentation: x86: convert intel_mpx.txt to reST
  Documentation: x86: convert protection-keys.txt to reST
  Documentation: x86: convert pat.txt to reST
  Documentation: x86: convert mtrr.txt to reST
  Documentation: x86: convert tlb.txt to reST
  Documentation: x86: convert zero-page.txt to reST
  ...
2019-05-10 13:24:53 -04:00

222 lines
6.8 KiB
ReStructuredText

.. SPDX-License-Identifier: GPL-2.0
============
x86 Topology
============
This documents and clarifies the main aspects of x86 topology modelling and
representation in the kernel. Update/change when doing changes to the
respective code.
The architecture-agnostic topology definitions are in
Documentation/cputopology.txt. This file holds x86-specific
differences/specialities which must not necessarily apply to the generic
definitions. Thus, the way to read up on Linux topology on x86 is to start
with the generic one and look at this one in parallel for the x86 specifics.
Needless to say, code should use the generic functions - this file is *only*
here to *document* the inner workings of x86 topology.
Started by Thomas Gleixner <tglx@linutronix.de> and Borislav Petkov <bp@alien8.de>.
The main aim of the topology facilities is to present adequate interfaces to
code which needs to know/query/use the structure of the running system wrt
threads, cores, packages, etc.
The kernel does not care about the concept of physical sockets because a
socket has no relevance to software. It's an electromechanical component. In
the past a socket always contained a single package (see below), but with the
advent of Multi Chip Modules (MCM) a socket can hold more than one package. So
there might be still references to sockets in the code, but they are of
historical nature and should be cleaned up.
The topology of a system is described in the units of:
- packages
- cores
- threads
Package
=======
Packages contain a number of cores plus shared resources, e.g. DRAM
controller, shared caches etc.
AMD nomenclature for package is 'Node'.
Package-related topology information in the kernel:
- cpuinfo_x86.x86_max_cores:
The number of cores in a package. This information is retrieved via CPUID.
- cpuinfo_x86.phys_proc_id:
The physical ID of the package. This information is retrieved via CPUID
and deduced from the APIC IDs of the cores in the package.
- cpuinfo_x86.logical_proc_id:
The logical ID of the package. As we do not trust BIOSes to enumerate the
packages in a consistent way, we introduced the concept of logical package
ID so we can sanely calculate the number of maximum possible packages in
the system and have the packages enumerated linearly.
- topology_max_packages():
The maximum possible number of packages in the system. Helpful for per
package facilities to preallocate per package information.
- cpu_llc_id:
A per-CPU variable containing:
- On Intel, the first APIC ID of the list of CPUs sharing the Last Level
Cache
- On AMD, the Node ID or Core Complex ID containing the Last Level
Cache. In general, it is a number identifying an LLC uniquely on the
system.
Cores
=====
A core consists of 1 or more threads. It does not matter whether the threads
are SMT- or CMT-type threads.
AMDs nomenclature for a CMT core is "Compute Unit". The kernel always uses
"core".
Core-related topology information in the kernel:
- smp_num_siblings:
The number of threads in a core. The number of threads in a package can be
calculated by::
threads_per_package = cpuinfo_x86.x86_max_cores * smp_num_siblings
Threads
=======
A thread is a single scheduling unit. It's the equivalent to a logical Linux
CPU.
AMDs nomenclature for CMT threads is "Compute Unit Core". The kernel always
uses "thread".
Thread-related topology information in the kernel:
- topology_core_cpumask():
The cpumask contains all online threads in the package to which a thread
belongs.
The number of online threads is also printed in /proc/cpuinfo "siblings."
- topology_sibling_cpumask():
The cpumask contains all online threads in the core to which a thread
belongs.
- topology_logical_package_id():
The logical package ID to which a thread belongs.
- topology_physical_package_id():
The physical package ID to which a thread belongs.
- topology_core_id();
The ID of the core to which a thread belongs. It is also printed in /proc/cpuinfo
"core_id."
System topology examples
========================
.. note::
The alternative Linux CPU enumeration depends on how the BIOS enumerates the
threads. Many BIOSes enumerate all threads 0 first and then all threads 1.
That has the "advantage" that the logical Linux CPU numbers of threads 0 stay
the same whether threads are enabled or not. That's merely an implementation
detail and has no practical impact.
1) Single Package, Single Core::
[package 0] -> [core 0] -> [thread 0] -> Linux CPU 0
2) Single Package, Dual Core
a) One thread per core::
[package 0] -> [core 0] -> [thread 0] -> Linux CPU 0
-> [core 1] -> [thread 0] -> Linux CPU 1
b) Two threads per core::
[package 0] -> [core 0] -> [thread 0] -> Linux CPU 0
-> [thread 1] -> Linux CPU 1
-> [core 1] -> [thread 0] -> Linux CPU 2
-> [thread 1] -> Linux CPU 3
Alternative enumeration::
[package 0] -> [core 0] -> [thread 0] -> Linux CPU 0
-> [thread 1] -> Linux CPU 2
-> [core 1] -> [thread 0] -> Linux CPU 1
-> [thread 1] -> Linux CPU 3
AMD nomenclature for CMT systems::
[node 0] -> [Compute Unit 0] -> [Compute Unit Core 0] -> Linux CPU 0
-> [Compute Unit Core 1] -> Linux CPU 1
-> [Compute Unit 1] -> [Compute Unit Core 0] -> Linux CPU 2
-> [Compute Unit Core 1] -> Linux CPU 3
4) Dual Package, Dual Core
a) One thread per core::
[package 0] -> [core 0] -> [thread 0] -> Linux CPU 0
-> [core 1] -> [thread 0] -> Linux CPU 1
[package 1] -> [core 0] -> [thread 0] -> Linux CPU 2
-> [core 1] -> [thread 0] -> Linux CPU 3
b) Two threads per core::
[package 0] -> [core 0] -> [thread 0] -> Linux CPU 0
-> [thread 1] -> Linux CPU 1
-> [core 1] -> [thread 0] -> Linux CPU 2
-> [thread 1] -> Linux CPU 3
[package 1] -> [core 0] -> [thread 0] -> Linux CPU 4
-> [thread 1] -> Linux CPU 5
-> [core 1] -> [thread 0] -> Linux CPU 6
-> [thread 1] -> Linux CPU 7
Alternative enumeration::
[package 0] -> [core 0] -> [thread 0] -> Linux CPU 0
-> [thread 1] -> Linux CPU 4
-> [core 1] -> [thread 0] -> Linux CPU 1
-> [thread 1] -> Linux CPU 5
[package 1] -> [core 0] -> [thread 0] -> Linux CPU 2
-> [thread 1] -> Linux CPU 6
-> [core 1] -> [thread 0] -> Linux CPU 3
-> [thread 1] -> Linux CPU 7
AMD nomenclature for CMT systems::
[node 0] -> [Compute Unit 0] -> [Compute Unit Core 0] -> Linux CPU 0
-> [Compute Unit Core 1] -> Linux CPU 1
-> [Compute Unit 1] -> [Compute Unit Core 0] -> Linux CPU 2
-> [Compute Unit Core 1] -> Linux CPU 3
[node 1] -> [Compute Unit 0] -> [Compute Unit Core 0] -> Linux CPU 4
-> [Compute Unit Core 1] -> Linux CPU 5
-> [Compute Unit 1] -> [Compute Unit Core 0] -> Linux CPU 6
-> [Compute Unit Core 1] -> Linux CPU 7